instant-bom/docs/backlog.md

Backlog

Notes

• First-time setup: fastapi/uvicorn/starlette are listed in requirements.txt but are not installed by default in a fresh .venv — run pip install -r requirements.txt before running the API.
• ComponentCategory.other maps to an empty subcategory list in suppliers/lcsc/categories.py:89 (_SUBCATEGORY_NAMES[other] = []) — rows classified as other can never resolve to an LCSC subcategory. Needs a taxonomy decision (which LCSC categories represent the catch-all), not a code fix.
• suppliers/lcsc/lcsc.py's original scraper (lcsc.com/api/products/search via CSRF session) was previously marked Done but was never actually verified live — it returns the Nuxt SSR shell, not JSON. Replaced with LCSC's real internal API (wmsc.lcsc.com/ftps/wm/product/query/*), found via live browser traffic capture and verified end-to-end.

Done

Ingestion Pipeline

• session/models.py — BomSession, BomRow, NormalizedParams, ComponentCategory, RowState
• ingestion/parser.py — CSV + XLSX + XLS, preamble skip, footer strip, delimiter detection
• ingestion/column_mapper.py — heuristic synonym lookup, AI fallback, synonym persistence
• ingestion/column_synonyms.json — file-backed synonym table, auto-updated on AI discovery
• ingestion/normalizer.py — designator parsing, range expansion, DNP detection, row state
• classification/classifier.py — designator prefix map + value heuristics → ComponentCategory
• classification/param_extractor.py — value parsing, description fallback, package extraction, AI fallback
• classification/package_patterns.json — file-backed regex patterns + known_footprints dict

LCSC Category Resolution

• suppliers/base.py — abstract SupplierAdapter(ABC), search(category_id, params, filters=None, page=1) -> SearchResult
• suppliers/category_resolver.py — generic AI category resolver, caches by (category, package, value)
• suppliers/lcsc/category_resolver.py — LCSC-specific wrapper over generic resolver
• suppliers/lcsc/fetch_categories.py + suppliers/lcsc/categories.json — scraped LCSC category tree, 600 subcategories committed
• suppliers/lcsc/categories.py — load_categories(), internal ComponentCategory → curated LCSC subcategory lists
• suppliers/lcsc/enrich_categories.py + make_enrich_input.py / apply_enrich_output.py — scrape example products + LLM one-line descriptions per subcategory, applied to all 600/600
• suppliers/lcsc/category_cache.json — file-backed resolution cache (69 entries)
• suppliers/lcsc/product_lookup.py, scout_ground_truth.py, eval_ground_truth.py — ground-truth harness (60 rows from real LCSC part numbers) + accuracy eval
  • Current accuracy: top-1 68% (41/60), top-3 88% (53/60)
• Verified live: categories.json subcategory ids are directly usable as catalogIdList values against LCSC's real search/facet API — no remapping layer needed

API + Frontend (backbone)

• api/main.py — FastAPI app entry point
• POST /upload — ingest file, return BomSession (HTMX-rendered row table)
• GET /session/{id}/row/{row_id}/scout — LLM category inference, HTMX-rendered candidate list
• bom_assistant/scouting/ — framework-agnostic orchestrator (build_session, scout_row) backing the above
• HTMX-based test page (api/templates/, server-rendered by FastAPI, no separate frontend process) for interactively testing category inference

LCSC Search Adapter

• suppliers/lcsc/lcsc.py — search() against LCSC's real internal API (wmsc.lcsc.com/ftps/wm/product/query/list), reverse-engineered from live browser traffic; previous lcsc.com/api/products/search scraper endpoint was stale/never worked (returned the Nuxt SSR shell, not JSON) — no CSRF/session dance needed for the real endpoint
• suppliers/lcsc/lcsc.py — query_facets() against wmsc.lcsc.com/ftps/wm/product/query/param/group: returns Package/Manufacturer/Packaging + per-param value lists, scoped to category + already-applied filters — this is the source for building filter chips each scouting round, no scraping or manual aggregation needed

Bug Fixes

• Double-space header normalization (_normalize_key collapses whitespace) — fixed Ref Des (Multi-4) mismatch
• Comment → part_number disambiguation — passive value sampling prevents pn='100nF'
• Description fallback scanners — extracts values from long strings like "CAP CER 0.1UF 100V X7R 0603"
• \b → (?<![A-Za-z]) lookbehind — fixes underscore-prefixed footprints (Texas_HTSOP-8-1EP)
• HTSOP overcapture fix — suffix pattern (?:-[A-Z0-9]+)* stops at _digit

---

Todo

Ingestion

• Strip Elec_ prefix from package — store "6.3x5.8" not "Elec_6.3x5.8" (change capture group in package_patterns.json)
• Persian-header BOM support — AI column mapping for non-Latin headers (currently returns 0 rows)

LCSC Search Adapter

• suppliers/lcsc/lcsc.py — official API mode (_search_official) still a stub, raises NotImplementedError; needs HMAC signing + LCSC API key

Scouting Loop

• BomRow state machine transitions for filtering/confirmed wired to query_facets() + search() (data layer for both is ready and live-verified, see Done)
• Retry / re-scout flow when user rejects results or picks a different category

API

• GET /session/{id} — return session state
• GET /session/{id}/row/{row_id}/facets — call query_facets(), return filter chip groups — deferred until category inference is validated interactively (only inference itself has been tested so far, not the full loop)
• POST /session/{id}/row/{row_id}/confirm-category — lock LCSC subcategory, advance state to filtering — deferred, same reason
• POST /session/{id}/row/{row_id}/search — search within confirmed category + param filters — deferred, same reason
• POST /session/{id}/row/{row_id}/confirm — lock part pick, advance state to confirmed — deferred, same reason
• GET /session/{id}/export — collate confirmed picks → downloadable BOM — deferred, same reason

Future: Other Suppliers

• DigiKey adapter
• Mouser adapter
• Nexar adapter

Frontend

• React scaffold + build setup — deferred until the scouting loop is validated; HTMX test page (see Done) covers interim needs
• Row-by-row scouting UI (category dropdown, results grid, confirm/reject)
• Session progress view (rows confirmed vs pending)
• Export button