NLQ Routing Test ①　Runbook (SQL vs Cypher/DeciKG Stub)

Purpose (this test only)

Verify that /nlq/execute correctly represents the selected route:

• SQL_ONLY → returns tabular result and no graph
• CYPHER_ONLY → returns graph stub (DeciKG not connected) and no SQL result

We are not testing “Cypher then SQL” or any hybrid behavior.

---

Endpoints

• POST http://localhost:8081/nlq/execute
• POST http://localhost:8081/nlq/plan (optional / informational)

---

Test Case A — Force SQL route (SQL_ONLY)

Command

curl -sS -X POST http://localhost:8081/nlq/execute \
  -H 'Content-Type: application/json' \
  -d '{
    "route": "SQL_ONLY",
    "lang": "ja",
    "session_id": "dev",
    "turn_no": 1,
    "sql_params": {
      "question": "今月の売上合計を出して",
      "top_k": 0,
      "locale": "ja_JP"
    }
  }' | jq .

PASS criteria (must all be true)

1. Route is SQL

• route == "SQL_ONLY"
• execution_kind == "sql"

2. Tabular payload exists

• row_count > 0
• rows is an array and rows.length == row_count

3. Graph is not used

• graph == null
• decikg_params == null
• cypher_result == null

4. Reason/UI-friendly messaging exists

• narration exists (string)
• trace.timeline contains at least:
  • one item with phase == "route_decide" and its note mentions SQL
  • one item with phase == "sql_only" and its note == "ok" (or similar success)

5. Provider metadata exists

• providers.devportal.mode exists (e.g. "dummy" now)
• providers.decikg.mode exists (e.g. "dummy" now)

Notes:

• stub_used == true is acceptable in dummy mode.
• sql and sql_result.sql may be empty in this phase; this is OK for the routing test.

---

Test Case B — Force Cypher route (CYPHER_ONLY → DeciKG Stub)

Command

curl -sS -X POST http://localhost:8081/nlq/execute \
  -H 'Content-Type: application/json' \
  -d '{
    "route": "CYPHER_ONLY",
    "lang": "ja",
    "session_id": "dev",
    "turn_no": 1,
    "decikg_params": {
      "question": "粗利率の低下要因を分解して",
      "kpi": "gross_margin_rate"
    }
  }' | jq .

PASS criteria (must all be true)

1. Route is Cypher/Graph

• route == "CYPHER_ONLY"
• execution_kind == "graph"

2. Stub is clearly indicated (DeciKG not connected)

• stub_used == true
• warnings contains "decikg_stub" (or equivalent)

3. Graph payload exists and is structurally valid

• graph is not null
• graph.nodes is an array
• graph.edges is an array
• graph.summary exists (string)
• graph.meta.stub == true OR graph.meta._nlq_stub_used == true
• graph.meta.note == "decikg_unavailable" (or equivalent reason)

4. SQL result is not used

• rows.length == 0
• row_count == 0
• sql_result == null

5. Reason/UI-friendly messaging exists

• narration exists and explains DeciKG is unavailable
• Optional: stats.analysis.note == "decikg_unavailable" is present (good)

---

Optional — /nlq/plan output (informational only)

Your current plan output is:

• route == "sql"
• candidates[0].route == "sql"

So, for UI, do not rely on /nlq/plan for route decision yet.
Use /nlq/execute.route + narration/trace as the “final decision” for branch display.

---

What the Frontend should display (minimal mapping)

For the “chosen branch + reason” UI:

• Chosen branch label: route
  • examples: "SQL_ONLY" or "CYPHER_ONLY"
• Reason text (priority order):
  1. narration
  2. trace.timeline[0].note (typically route_decide)
  3. for Cypher stub: stats.analysis.note or graph.meta.note
• Badge:
  • show “STUB” when stub_used == true

---

Files used (for offshore reference)

• api/app/main.py (router registration)
• api/app/routers/nlq_execute.py (thin endpoint)
• api/app/services/nlq_execute.py (route normalization + stub graph + response fields)
• api/app/schemas/nlq.py (request/response contract)
• api/app/services/executors/route_executor.py (internal route execution)

---

If you want, I can also add a one-liner jq “PASS/FAIL checker” for each case (so offshore can run it in CI), but the above is the concise manual runbook version.

NLQ Routing Test ②

Goal

Verify that nlq-dev forwards options.top_k to Dev-Portal and Dev-Portal retrieval becomes enabled (not retrieval_disabled).

Preconditions

1. Dev-Portal is reachable at:

• DEVPORTAL_URL=http://192.168.0.44:30080
• DEVPORTAL_ANALYTICS_PATH=/analytics/query

2. nlq-dev API is reachable at:

• NLQ_BASE=http://127.0.0.1:8081

3. Environment variables are set (either host shell export or docker compose env):

export NLQ_BASE="http://127.0.0.1:8081"
export DEVPORTAL_MODE="http"
export DEVPORTAL_URL="http://192.168.0.44:30080"
export DEVPORTAL_ANALYTICS_PATH="/analytics/query"

Test Command (copy/paste)

SID="$(uuidgen)"

curl -sS -X POST "${NLQ_BASE}/nlq/execute" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d "{
    \"session_id\": \"${SID}\",
    \"turn_no\": 1,
    \"question\": \"2011年の受注データで、月別の受注件数（重複なし）と合計金額を出して。1月から12月まで。\",
    \"route\": \"SQL_ONLY\",
    \"locale\": \"ja_JP\",
    \"options\": { \"top_k\": 10, \"dry_run\": false },
    \"devportal_request\": { \"join_planner_enabled\": true }
  }" \
| python -c 'import sys,json; d=json.load(sys.stdin);
m=(d.get("meta") or {});
tm=(m.get("tabular_meta") or {});
dp=(tm.get("devportal") or {});
req=(tm.get("request") or {});
diag=(dp.get("diagnostics") or {});
ret=(diag.get("retrieval") or {});
print("stub_used:", d.get("stub_used"));
print("providers:", d.get("providers"));
print("tabular_meta.url:", tm.get("url"));
print("tabular_meta.request.top_k:", req.get("top_k"));
print("devportal.status:", dp.get("status"));
print("devportal.reason_code:", dp.get("reason_code"));
print("diagnostics.retrieval.enabled:", ret.get("enabled"), "top_k_requested:", ret.get("top_k_requested"));
rows=(d.get("rows") or []);
print("row_count:", d.get("row_count"), "rows_len:", len(rows));
print("rows_head:", rows[:3]);
'

Expected Result (Pass Criteria)

The output must show all of the following:

• stub_used: False
• providers: {'devportal': {'mode': 'http'}, ...}
• tabular_meta.url: http://192.168.0.44:30080/analytics/query
• tabular_meta.request.top_k: 10
• devportal.status: ok
• devportal.reason_code: ok
• diagnostics.retrieval.enabled: True top_k_requested: 10
• row_count: 12 (or at least rows_len: 12) and rows contain month 1..12

Failure Patterns & Quick Checks

1. curl: (3) URL using bad/illegal format
   → NLQ_BASE is empty or not exported. Run: echo $NLQ_BASE
2. HTTP 404 from Apache / HTML response
   → you hit the wrong port (8000). Use 8081.
3. HTTP 422 missing session_id / turn_no
   → include session_id and turn_no.
4. devportal.reason_code: retrieval_disabled / top_k_requested: 0
   → forwarding is broken (top_k not passed). Must be top_k: 10 and retrieval enabled.