docs: scouting loop diagrams + backlog/architecture corrections

Add scouting_loop.md + PlantUML state/sequence diagrams describing
the intended pending->scouting->filtering->confirmed loop and what's
actually built. Correct backlog.md/architecture.md drift (stale
Todo/Done status, unverified scraper claims).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

docs/architecture.md

@@ -57,7 +57,7 @@ Adapter layer for electronic component marketplaces. All adapters implement the
 ```python
 class SupplierAdapter(ABC):
     def __init__(self, api_token: str | None = None): ...
-    def search(self, category_id: int, params: NormalizedParams, page: int = 1) -> SearchResult: ...
+    def search(self, category_id: int, params: NormalizedParams, filters: dict[str, Any] | None = None, page: int = 1) -> SearchResult: ...
 ```
 
 #### 4a. LCSC Category Resolution
@@ -94,8 +94,11 @@ Before searching, each `BomRow` must be mapped to a specific LCSC subcategory (n
 Example: `category=capacitor, package=6.3x5.8, voltage=25V` → LLM picks `"Aluminum Electrolytic Capacitors"` → looked up → id `1140`
 
 **LCSC Search** (`lcsc/lcsc.py`):
-- Scraper mode (default): GET any LCSC category page to acquire CSRF cookie + session, then POST to `https://lcsc.com/api/products/search` with `catalog_id=<subcategory_id>` + parameter filters
-- Token mode: if `api_token` provided, use official LCSC API (`https://ips.lcsc.com/rest/wmsc2agent/`) with HMAC-signed requests
+- Live API mode (default, no credentials needed): LCSC's real internal JSON API, reverse-engineered from live browser traffic (the site is a Nuxt SPA — the previously assumed `lcsc.com/api/products/search` scraper endpoint does not exist and was never actually verified working). No CSRF/session dance required, plain POST with `User-Agent`/`Origin`/`Referer` is sufficient:
+  - `POST https://wmsc.lcsc.com/ftps/wm/product/query/list` — paginated product search. Request: `{catalogIdList:[id], brandIdList, encapValueList, paramNameValueMap, isStock, ..., currentPage, pageSize}`. Response items carry LCSC part number (`productCode`), manufacturer (`brandNameEn`), package (`encapStandard`), tiered pricing (`productPriceList`), stock, per-part spec list (`paramVOList`), datasheet URL.
+  - `POST https://wmsc.lcsc.com/ftps/wm/product/query/param/group` — facet/filter query, same request shape (minus pagination). Response: `{Package: [...], Manufacturer: [...], Packaging: [...], paramNameValueMap: {<param name>: {paramDetailList, unitList, unitConversionMap}}}`, scoped to the category and any filters already applied — this is the source for building filter chips each scouting round, no separate scraping or client-side aggregation needed.
+  - Verified live: `categories.json`/`category_resolver.py`'s subcategory ids (e.g. `1142` = "Ceramic Capacitors") are directly usable as `catalogIdList` values against both endpoints — no id remapping layer required.
+- Token mode: if `api_token` provided, use official LCSC API (`https://ips.lcsc.com/rest/wmsc2agent/`) with HMAC-signed requests — not yet implemented (`NotImplementedError`).
 
 ### 5. Scouting Loop *(not yet built)*
 Drives each `BomRow` from `pending` to `confirmed` through an iterative user-guided process.

docs/backlog.md

@@ -3,6 +3,7 @@
 ## Notes
 - First-time setup: `fastapi`/`uvicorn`/`starlette` are listed in `requirements.txt` but are not installed by default in a fresh `.venv` — run `pip install -r requirements.txt` before running the API.
 - `ComponentCategory.other` maps to an empty subcategory list in `suppliers/lcsc/categories.py:89` (`_SUBCATEGORY_NAMES[other] = []`) — rows classified as `other` can never resolve to an LCSC subcategory. Needs a taxonomy decision (which LCSC categories represent the catch-all), not a code fix.
+- `suppliers/lcsc/lcsc.py`'s original scraper (`lcsc.com/api/products/search` via CSRF session) was previously marked Done but was never actually verified live — it returns the Nuxt SSR shell, not JSON. Replaced with LCSC's real internal API (`wmsc.lcsc.com/ftps/wm/product/query/*`), found via live browser traffic capture and verified end-to-end.
 
 ## Done
 
@@ -26,6 +27,18 @@
 - [x] `suppliers/lcsc/category_cache.json` — file-backed resolution cache (69 entries)
 - [x] `suppliers/lcsc/product_lookup.py`, `scout_ground_truth.py`, `eval_ground_truth.py` — ground-truth harness (60 rows from real LCSC part numbers) + accuracy eval
   - Current accuracy: **top-1 68% (41/60), top-3 88% (53/60)**
+- [x] Verified live: `categories.json` subcategory ids are directly usable as `catalogIdList` values against LCSC's real search/facet API — no remapping layer needed
+
+### API + Frontend (backbone)
+- [x] `api/main.py` — FastAPI app entry point
+- [x] `POST /upload` — ingest file, return BomSession (HTMX-rendered row table)
+- [x] `GET /session/{id}/row/{row_id}/scout` — LLM category inference, HTMX-rendered candidate list
+- [x] `bom_assistant/scouting/` — framework-agnostic orchestrator (`build_session`, `scout_row`) backing the above
+- [x] HTMX-based test page (`api/templates/`, server-rendered by FastAPI, no separate frontend process) for interactively testing category inference
+
+### LCSC Search Adapter
+- [x] `suppliers/lcsc/lcsc.py` — `search()` against LCSC's real internal API (`wmsc.lcsc.com/ftps/wm/product/query/list`), reverse-engineered from live browser traffic; previous `lcsc.com/api/products/search` scraper endpoint was stale/never worked (returned the Nuxt SSR shell, not JSON) — no CSRF/session dance needed for the real endpoint
+- [x] `suppliers/lcsc/lcsc.py` — `query_facets()` against `wmsc.lcsc.com/ftps/wm/product/query/param/group`: returns Package/Manufacturer/Packaging + per-param value lists, scoped to category + already-applied filters — this is the source for building filter chips each scouting round, no scraping or manual aggregation needed
 
 ### Bug Fixes
 - [x] Double-space header normalization (`_normalize_key` collapses whitespace) — fixed `Ref Des  (Multi-4)` mismatch
@@ -43,20 +56,16 @@
 - [ ] Persian-header BOM support — AI column mapping for non-Latin headers (currently returns 0 rows)
 
 ### LCSC Search Adapter
-- [x] `suppliers/lcsc/lcsc.py` — scraper mode: CSRF session + POST to `lcsc.com/api/products/search`
 - [ ] `suppliers/lcsc/lcsc.py` — official API mode (`_search_official`) still a stub, raises `NotImplementedError`; needs HMAC signing + LCSC API key
 
 ### Scouting Loop
-- [ ] Scouting orchestrator — drive pending rows through scouting → filtering → confirmed
-- [ ] BomRow state machine transitions wired to category resolver + search adapter
+- [ ] BomRow state machine transitions for `filtering`/`confirmed` wired to `query_facets()` + `search()` (data layer for both is ready and live-verified, see Done)
 - [ ] Retry / re-scout flow when user rejects results or picks a different category
 
 ### API
-- [ ] FastAPI app entry point (`api/main.py`)
-- [ ] `POST /upload` — ingest file, return BomSession
 - [ ] `GET /session/{id}` — return session state
-- [ ] `GET /session/{id}/row/{row_id}/scout` — LLM category inference, return `{subcategory, alternatives[]}`
-- [ ] `POST /session/{id}/row/{row_id}/confirm-category` — lock LCSC subcategory, advance state to filtering — **deferred** until category inference is validated interactively (only inference itself has been tested so far, not the full loop)
+- [ ] `GET /session/{id}/row/{row_id}/facets` — call `query_facets()`, return filter chip groups — **deferred** until category inference is validated interactively (only inference itself has been tested so far, not the full loop)
+- [ ] `POST /session/{id}/row/{row_id}/confirm-category` — lock LCSC subcategory, advance state to filtering — **deferred**, same reason
 - [ ] `POST /session/{id}/row/{row_id}/search` — search within confirmed category + param filters — **deferred**, same reason
 - [ ] `POST /session/{id}/row/{row_id}/confirm` — lock part pick, advance state to confirmed — **deferred**, same reason
 - [ ] `GET /session/{id}/export` — collate confirmed picks → downloadable BOM — **deferred**, same reason
@@ -67,9 +76,7 @@
 - [ ] Nexar adapter
 
 ### Frontend
-- [ ] HTMX-based test page (server-rendered by FastAPI, no separate frontend process) for interactively testing category inference — backbone, in progress
-- [ ] React scaffold + build setup — deferred until the scouting loop is validated; HTMX test page covers interim needs
-- [ ] Upload page
+- [ ] React scaffold + build setup — deferred until the scouting loop is validated; HTMX test page (see Done) covers interim needs
 - [ ] Row-by-row scouting UI (category dropdown, results grid, confirm/reject)
 - [ ] Session progress view (rows confirmed vs pending)
 - [ ] Export button

docs/diagrams/scouting_loop_sequence.puml

@@ -0,0 +1,42 @@
+@startuml
+actor User
+participant "API" as API
+participant "Orchestrator" as Orch
+participant "LcscCategoryResolver" as Resolver
+participant "LcscAdapter" as Adapter
+
+== Implemented ==
+
+User -> API : POST /upload (file)
+API -> Orch : build_session(content, filename)
+Orch --> API : BomSession (rows: pending)
+API --> User : row table (HTMX)
+
+User -> API : GET .../row/{id}/scout
+API -> Orch : scout_row(session, row_id)
+Orch -> Resolver : resolve(row)
+Resolver --> Orch : [LcscCategory] (cache hit or Haiku call)
+Orch -> Orch : row.scout_candidates = [...]\nrow.state = scouting
+Orch --> API : row
+API --> User : candidate list (HTMX swap)
+
+== Designed, not yet built ==
+
+User --> API : POST .../confirm-category {category_id}
+API --> Orch : confirm_category(session, row_id, category_id)
+Orch --> Orch : row.resolved_category_id = ...\nrow.state = filtering
+
+User --> API : POST .../search {filters, page}
+API --> Orch : search_row(session, row_id, filters, page)
+Orch --> Adapter : search(category_id, params, filters, page)
+Adapter --> Orch : SearchResult(items, count)
+Orch --> API : SearchResult
+API --> User : results grid
+
+User --> API : POST .../confirm {pick}
+API --> Orch : confirm_pick(session, row_id, pick)
+Orch --> Orch : row.confirmed_pick = pick\nrow.state = confirmed
+
+User --> API : GET .../export
+API --> User : collated BOM (CSV/XLSX)
+@enduml

docs/diagrams/scouting_loop_state.puml

@@ -0,0 +1,15 @@
+@startuml
+[*] --> pending
+
+pending --> flagged : DNP / non-electronic\n(normalizer.py, built)
+
+pending --> scouting : GET .../scout\n(resolve() -> candidates, built)
+scouting --> scouting : re-scout\n(GET .../scout again, built)
+
+scouting -[dotted]-> filtering : POST .../confirm-category\n**not yet built**
+filtering -[dotted]-> confirmed : POST .../confirm\n**not yet built**
+filtering -[dotted]-> scouting : user rejects all results,\nre-scout with new params\n**not yet built**
+
+confirmed --> [*]
+flagged --> [*]
+@enduml

docs/scouting_loop.md

@@ -0,0 +1,74 @@
+# Scouting Loop
+
+The scouting loop drives each `BomRow` from `pending` to `confirmed` through a
+user-in-the-loop cycle: AI infers an LCSC subcategory → user confirms or
+overrides it → parametric search within that subcategory → user picks a part.
+Design source of truth: `docs/architecture.md` (§5 "Scouting Loop", "User
+Flow", "System Flow"). Build status source of truth: `docs/backlog.md`.
+
+**Current reality:** only the first hop (`pending → scouting`, i.e. category
+inference) is implemented, via `GET /session/{id}/row/{row_id}/scout`. The
+rest of the loop (`confirm-category`, `search`, `confirm`, `export`) is
+designed but intentionally deferred until inference itself is trusted through
+interactive testing (see `docs/backlog.md` → API section, "deferred" notes).
+
+## State diagram
+
+`RowState` per `bom_assistant/session/models.py`. Solid edges are built,
+dashed edges are designed but not yet implemented.
+
+Source: [`diagrams/scouting_loop_state.puml`](diagrams/scouting_loop_state.puml)
+
+Note: today's `orchestrator.scout_row` always transitions to `scouting`, even
+when `resolve()` returns zero candidates (it does not auto-flag the row) —
+the empty-result case is left as `scout_candidates: []` for the UI to show
+"no candidates found," rather than kicking the row to `flagged`. Auto-flagging
+on empty results, and a `flag_reason` field to distinguish that from
+normalizer-level flagging, was scoped out as part of the deferred loop work.
+
+## Sequence diagram
+
+Actors: User, API (`bom_assistant/api/routes.py`), Orchestrator
+(`bom_assistant/scouting/orchestrator.py`), LcscCategoryResolver
+(`bom_assistant/suppliers/lcsc/category_resolver.py`), LcscAdapter
+(`bom_assistant/suppliers/lcsc/lcsc.py`). Dashed arrows = not yet built.
+
+Source: [`diagrams/scouting_loop_sequence.puml`](diagrams/scouting_loop_sequence.puml)
+
+## Walkthrough
+
+1. **Upload** *(built)* — user uploads a BOM file. `POST /upload` runs
+   `parse_bom` → `map_columns` → `normalize`, returns a `BomSession` with all
+   rows `pending` (or `flagged` for DNP/non-electronic rows).
+2. **Scout** *(built)* — for a `pending` row, `GET .../scout` calls
+   `LcscCategoryResolver.resolve(row)` (file-cached, falls back to a Claude
+   Haiku call) and returns up to 3 ranked LCSC subcategory candidates. Row
+   moves to `scouting`. Can be re-triggered freely.
+3. **Confirm category** *(deferred)* — user accepts the top candidate or
+   picks an alternative; row locks in `resolved_category_id` and moves to
+   `filtering`.
+4. **Facet query + Search** *(data layer built, not yet wired to an
+   endpoint)* — `LcscAdapter.query_facets(category_id, filters)` calls LCSC's
+   real facet API (`wmsc.lcsc.com/ftps/wm/product/query/param/group`) to get
+   manufacturer/package/param filter groups scoped to the category and
+   whatever's already selected; `LcscAdapter.search(category_id, params,
+   filters, page)` calls the matching product-list endpoint. Both were
+   reverse-engineered from live traffic and verified end-to-end against real
+   category-resolver output — see `docs/backlog.md`. No `GET .../facets` or
+   `POST .../search` route exists yet.
+5. **Confirm pick** *(deferred)* — user selects a specific part; row locks
+   `confirmed_pick` and moves to `confirmed`.
+6. **Retry** *(deferred)* — if the user rejects all search results, the row
+   can go back to `scouting` with adjusted parameters rather than being a
+   dead end.
+7. **Export** *(deferred)* — once rows are `confirmed` (or explicitly
+   skipped), `GET .../export` collates them into an order-ready BOM.
+
+Steps 3–7 are why the API currently exposes only `/upload` and `/scout`: the
+loop is built incrementally, and the next slice only gets built once category
+inference (step 2) has been exercised enough through the HTMX test page to
+trust building on top of it.
+
+## See also
+- `docs/architecture.md` — full intended module design
+- `docs/backlog.md` — current Done/Todo status per component