# Architecture

## Modules

### 1. Session
Shared data schemas used across all other modules.

- `ComponentCategory` enum: capacitor, resistor, inductor, diode, led, ic, connector, crystal, mosfet, sensor, other
- `RowState` enum: pending → scouting → filtering → confirmed | flagged
- `NormalizedParams`: package, value (SI float), value_str, unit, voltage_rating, current_rating, power_rating, tolerance, part_number, manufacturer
- `BomRow`: row_id, designators, quantity, raw_value, footprint, category, normalized_params, state, applied_filters, facet_groups, confirmed_pick
- `BomSession`: session_id, created_at, filename, preferred_suppliers, rows

### 2. Ingestion
Converts raw file bytes into a list of `BomRow` objects.

**Parser** — detects format by extension, handles:
- CSV: utf-8-sig decode, delimiter detection (`,` vs `;`), footer stripping, blank row skipping
- XLSX: preamble skip via header-row detection (scans rows 0–14 for synonym matches), column-offset detection, phantom column stripping, footer/totals detection
- XLS: same as XLSX via xlrd

**Column Mapper** — maps raw column names to canonical fields:
- Heuristic lookup against `column_synonyms.json` (exact normalized match)
- Comment disambiguation: if BOM has both `Comment` and a richer value column, `Comment` → `part_number` (unless values look like passive component values)
- AI fallback (Claude Haiku) when required `value` field is still unmapped
- New AI-discovered synonyms persisted to `column_synonyms.json` to avoid future API calls
- Unknown columns passed through with `_raw_` prefix

**Normalizer** — per-row processing:
- Skip empty / KiCad `~` rows
- Parse and expand designators (`C1-5` → `[C1, C2, C3, C4, C5]`)
- Parse quantity; qty=0 → flagged
- DNP detection (value string, boolean columns, Mounting column)
- Non-electronic detection (PCB, screw, nut, heatsink, etc.) → flagged
- Calls classifier + param_extractor → emits `BomRow`

### 3. Classification
Determines component category and extracts normalized parameters.

**Classifier** — `(designators, raw_value)` → `ComponentCategory`:
- Designator prefix map: `C`→capacitor, `R/RV`→resistor, `L`→inductor, `D`→diode, `U/IC`→ic, `J/P/CON/USB`→connector, `Q/T`→mosfet, `X/Y`→crystal, `NTC/RT`→sensor
- Special: D + LED keyword in value → led
- Fallback (no designators): capacitance/resistance regex, alphanumeric part-number pattern → ic, else → other

**Param Extractor** — `(raw_value, footprint, category)` → `NormalizedParams`:
- Value parsing: capacitance (bare prefix, European notation), resistance (European 2k2/4k7, standard), inductance, frequency, IC/MOSFET (entire string → part_number)
- Description fallback: scans full string for embedded values (`"CAP CER 0.1UF 100V X7R 0603"`)
- Cross-category extractions: voltage_rating, current_rating, power_rating, tolerance
- Package extraction cascade:
  1. Exact match in `known_footprints` dict
  2. Regex cascade from `package_patterns.json`
  3. AI fallback (Claude Haiku) → result persisted to `known_footprints`

### 4. Suppliers
Adapter layer for electronic component marketplaces. All adapters implement the abstract `SupplierAdapter` base class, which supports both scraper mode (default, no credentials) and token mode (official API).

```python
class SupplierAdapter(ABC):
    def __init__(self, api_token: str | None = None): ...
    def search(self, category_id: int, params: NormalizedParams, filters: dict[str, Any] | None = None, page: int = 1) -> SearchResult: ...
```

#### 4a. LCSC Category Resolution
Before searching, each `BomRow` must be mapped to a specific LCSC subcategory (name + integer ID). Generic keyword search is unreliable — LCSC expects category-scoped queries.

**One-time setup** (`lcsc/fetch_categories.py`):
- Scrapes ~10 LCSC category pages (Capacitors, Resistors, ICs, etc.) with 1.5s grace period between requests
- Extracts subcategory name + ID from each page
- Writes `lcsc/categories.json` — committed to repo, refreshed quarterly

**Category JSON format:**
```json
[
  {
    "group": "Passives",
    "category": "Capacitors",
    "category_id": 11,
    "subcategories": [
      {"name": "Aluminum Electrolytic Capacitors", "id": 1140},
      {"name": "Multilayer Ceramic Capacitors (MLCC)", "id": 1142},
      {"name": "Film Capacitors", "id": 1144},
      {"name": "Tantalum Capacitors", "id": 1150}
    ]
  }
]
```

**LLM Category Resolver** (`lcsc/category_resolver.py`):
- Cache key: `f"{internal_category}|{package}|{value_str}"` → check `lcsc/category_cache.json`
- On miss: pre-filter category list to relevant subtree → build Claude Haiku prompt → parse JSON response
- Prompt passes component info + subcategory name list; LLM returns one exact name
- Persist result to `category_cache.json` (same file-backed learning pattern as `package_patterns.json`)

Example: `category=capacitor, package=6.3x5.8, voltage=25V` → LLM picks `"Aluminum Electrolytic Capacitors"` → looked up → id `1140`

**LCSC Search** (`lcsc/lcsc.py`):
- Live API mode (default, no credentials needed): LCSC's real internal JSON API, reverse-engineered from live browser traffic (the site is a Nuxt SPA — the previously assumed `lcsc.com/api/products/search` scraper endpoint does not exist and was never actually verified working). No CSRF/session dance required, plain POST with `User-Agent`/`Origin`/`Referer` is sufficient:
  - `POST https://wmsc.lcsc.com/ftps/wm/product/query/list` — paginated product search. Request: `{catalogIdList:[id], brandIdList, encapValueList, paramNameValueMap, isStock, ..., currentPage, pageSize}`. Response items carry LCSC part number (`productCode`), manufacturer (`brandNameEn`), package (`encapStandard`), tiered pricing (`productPriceList`), stock, per-part spec list (`paramVOList`), datasheet URL.
  - `POST https://wmsc.lcsc.com/ftps/wm/product/query/param/group` — facet/filter query, same request shape (minus pagination). Response: `{Package: [...], Manufacturer: [...], Packaging: [...], paramNameValueMap: {<param name>: {paramDetailList, unitList, unitConversionMap}}}`, scoped to the category and any filters already applied — this is the source for building filter chips each scouting round, no separate scraping or client-side aggregation needed.
  - Verified live: `categories.json`/`category_resolver.py`'s subcategory ids (e.g. `1142` = "Ceramic Capacitors") are directly usable as `catalogIdList` values against both endpoints — no id remapping layer required.
- Token mode: if `api_token` provided, use official LCSC API (`https://ips.lcsc.com/rest/wmsc2agent/`) with HMAC-signed requests — not yet implemented (`NotImplementedError`).

### 5. Scouting Loop
Drives each `BomRow` from `pending` to `confirmed` through an iterative user-guided process (`bom_assistant/scouting/orchestrator.py`).

```
pending
  → scouting   (LLM infers LCSC subcategory, user confirms or picks different)          — built
  → filtering  (facet query + search within confirmed subcategory, user narrows/browses) — built (Package + Manufacturer filters only)
  → confirmed  (user picks a specific part)                                              — built
  → flagged    (DNP / non-electronic / user skips)                                       — built (ingestion-time only)
```

Not yet built: per-component-param filters (Capacitance/Tolerance/Voltage Rating/... — `paramNameValueMap` request shape unverified live), pagination past page 1, removing/toggling off an applied filter, and the retry path (user rejects all results → back to scouting with modified parameters).

### 6. API Layer
FastAPI HTTP interface (`bom_assistant/api/routes.py`).

- `POST /upload` — accept BOM file, run ingestion, return `BomSession` (built)
- `GET /session/{id}/row/{row_id}/scout` — trigger LLM category inference, sets `row.scout_candidates` (built)
- `POST /session/{id}/row/{row_id}/confirm-category` — user accepts/overrides category → state: filtering, populates facets + initial search results (built)
- `POST /session/{id}/row/{row_id}/apply-filter` — apply a Package/Manufacturer facet value, re-run facets + search (built)
- `POST /session/{id}/row/{row_id}/confirm` — lock part pick, advance state to confirmed (built)
- `GET /session/{id}` — current session state (not yet built)
- `GET /session/{id}/export` — collate confirmed picks → CSV/XLSX download (not yet built)

### 7. Frontend *(not yet built)*
React SPA.

- Upload page → triggers `POST /upload`
- Row-by-row scouting view: category confirmation dropdown + results grid + confirm/reject buttons
- Progress bar across all rows
- Export button

---

## User Flow

```
User uploads BOM file
  └─ Ingestion pipeline runs
       ├─ Parser reads file → raw rows
       ├─ Column Mapper normalizes column names
       └─ Normalizer classifies + extracts params → BomSession

For each pending BomRow:
  └─ Scout
       ├─ LLM infers LCSC subcategory from component info
       │    (cache hit → instant; miss → Claude Haiku call → cached)
       ├─ UI shows inferred subcategory + dropdown of all alternatives
       ├─ User confirms or picks a different subcategory
       ├─ Search within confirmed subcategory + param filters
       ├─ User browses results, adjusts filters if needed
       └─ User confirms pick  ──→  row state: confirmed
                     └─ rejects  ──→  retry with different subcategory/filters

All rows confirmed
  └─ Export → order-ready BOM (CSV/XLSX)
```

---

## System Flow

```
POST /upload
  parse_bom(bytes, filename)
    → map_columns(raw_rows)
    → normalize(mapped_rows)
    → BomSession stored in memory

GET /session/{id}/row/{row_id}/scout
  lcsc_category_resolver.resolve(row)          ← cache hit or Haiku call
    → [LcscCategory("Alum. Electrolytic...", 1140), ...]   (best-first, up to 3)
  row.scout_candidates = [{name, id}, ...]
  row.state → scouting

POST /session/{id}/row/{row_id}/confirm-category  {category_id: 1140, category_name: "..."}
  row.resolved_category_id = 1140
  row.resolved_category_name = "..."
  row.applied_filters = {}          ← reset; kept strictly as an LCSC filter payload fragment
  row.state → filtering
  _refresh_filtering(row):
    row.facet_groups = lcsc.query_facets(1140, row.applied_filters)
    result = lcsc.search(1140, row.normalized_params, row.applied_filters, page=1)
    row.search_results = result.items
    row.search_meta = {count, page, page_size, error}

POST /session/{id}/row/{row_id}/apply-filter  {group: "Package"|"Manufacturer", value: ...}
  row.applied_filters[<encapValueList|brandIdList>].append(value)
  _refresh_filtering(row)           ← same as above, narrows facets + results

POST /session/{id}/row/{row_id}/confirm  {product_code: "C602037"}
  row.confirmed_pick = <item from row.search_results matching product_code>
  row.state → confirmed

GET /session/{id}/export
  [confirmed rows] → assemble BOM → return file
```

---

## Physical File Map

```
instant-bom/
├── bom_assistant/
│   ├── session/
│   │   └── models.py                      — Pydantic schemas
│   │
│   ├── ingestion/
│   │   ├── parser.py                      — raw bytes → list[dict[str, str]]
│   │   ├── column_mapper.py               — raw keys → canonical keys
│   │   ├── column_synonyms.json           — file-backed synonym table (auto-updated by AI)
│   │   └── normalizer.py                  — mapped rows → list[BomRow]
│   │
│   ├── classification/
│   │   ├── classifier.py                  — designator prefix + value heuristics → ComponentCategory
│   │   ├── param_extractor.py             — value string + footprint → NormalizedParams
│   │   └── package_patterns.json          — file-backed regex + known_footprints (auto-updated by AI)
│   │
│   ├── suppliers/
│   │   ├── base.py                        — abstract SupplierAdapter
│   │   └── lcsc/
│   │       ├── fetch_categories.py        — one-time CLI: scrape LCSC → categories.json
│   │       ├── categories.json            — committed static file (refresh quarterly)
│   │       ├── categories.py              — load + filter category tree
│   │       ├── category_resolver.py       — LLM category inference + file-backed cache
│   │       ├── category_cache.json        — auto-updated by resolver
│   │       └── lcsc.py                    — search adapter (live wmsc.lcsc.com API + token mode stub)
│   │
│   ├── scouting/
│   │   ├── ingest.py                      — bytes → BomSession (parser → column_mapper → normalizer)
│   │   ├── orchestrator.py                — scout_row, confirm_category, apply_filter, confirm_pick
│   │   └── errors.py                      — SessionNotFoundError, RowNotFoundError, ScoutingValidationError
│   │
│   └── api/
│       ├── main.py                        — FastAPI app + exception handlers
│       ├── routes.py                      — HTTP routes (HTMX-rendered partials)
│       ├── store.py                       — in-memory SessionStore
│       └── templates/                     — Jinja2 partials for the HTMX test page
│
├── frontend/                              (not yet built — React SPA; HTMX test page above covers interim needs)
│   └── src/
│
├── examples/
│   ├── full/                              — real BOM test files (CSV, XLSX, XLS)
│   └── reduced/                           — same files, trimmed
│
└── requirements.txt
```