Architecture

Modules

1. Session

Shared data schemas used across all other modules.

ComponentCategory enum: capacitor, resistor, inductor, diode, led, ic, connector, crystal, mosfet, sensor, other
RowState enum: pending → scouting → filtering → confirmed | flagged
NormalizedParams: package, value (SI float), value_str, unit, voltage_rating, current_rating, power_rating, tolerance, part_number, manufacturer
BomRow: row_id, designators, quantity, raw_value, footprint, category, normalized_params, state, applied_filters, facet_groups, confirmed_pick
BomSession: session_id, created_at, filename, preferred_suppliers, rows

2. Ingestion

Converts raw file bytes into a list of BomRow objects.

Parser — detects format by extension, handles:

CSV: utf-8-sig decode, delimiter detection (, vs ;), footer stripping, blank row skipping
XLSX: preamble skip via header-row detection (scans rows 0–14 for synonym matches), column-offset detection, phantom column stripping, footer/totals detection
XLS: same as XLSX via xlrd

Column Mapper — maps raw column names to canonical fields:

Heuristic lookup against column_synonyms.json (exact normalized match)
Comment disambiguation: if BOM has both Comment and a richer value column, Comment → part_number (unless values look like passive component values)
AI fallback (Claude Haiku) when required value field is still unmapped
New AI-discovered synonyms persisted to column_synonyms.json to avoid future API calls
Unknown columns passed through with _raw_ prefix

Normalizer — per-row processing:

Skip empty / KiCad ~ rows
Parse and expand designators (C1-5 → [C1, C2, C3, C4, C5])
Parse quantity; qty=0 → flagged
DNP detection (value string, boolean columns, Mounting column)
Non-electronic detection (PCB, screw, nut, heatsink, etc.) → flagged
Calls classifier + param_extractor → emits BomRow

3. Classification

Determines component category and extracts normalized parameters.

Classifier — (designators, raw_value) → ComponentCategory:

Designator prefix map: C→capacitor, R/RV→resistor, L→inductor, D→diode, U/IC→ic, J/P/CON/USB→connector, Q/T→mosfet, X/Y→crystal, NTC/RT→sensor
Special: D + LED keyword in value → led
Fallback (no designators): capacitance/resistance regex, alphanumeric part-number pattern → ic, else → other

Param Extractor — (raw_value, footprint, category) → NormalizedParams:

Value parsing: capacitance (bare prefix, European notation), resistance (European 2k2/4k7, standard), inductance, frequency, IC/MOSFET (entire string → part_number)
Description fallback: scans full string for embedded values ("CAP CER 0.1UF 100V X7R 0603")
Cross-category extractions: voltage_rating, current_rating, power_rating, tolerance
Package extraction cascade:
1. Exact match in known_footprints dict
2. Regex cascade from package_patterns.json
3. AI fallback (Claude Haiku) → result persisted to known_footprints

4. Suppliers

Adapter layer for electronic component marketplaces. All adapters implement the abstract SupplierAdapter base class, which supports both scraper mode (default, no credentials) and token mode (official API).

class SupplierAdapter(ABC):
    def __init__(self, api_token: str | None = None): ...
    def search(self, category_id: int, params: NormalizedParams, filters: dict[str, Any] | None = None, page: int = 1) -> SearchResult: ...

4a. LCSC Category Resolution

Before searching, each BomRow must be mapped to a specific LCSC subcategory (name + integer ID). Generic keyword search is unreliable — LCSC expects category-scoped queries.

One-time setup (lcsc/fetch_categories.py):

Scrapes ~10 LCSC category pages (Capacitors, Resistors, ICs, etc.) with 1.5s grace period between requests
Extracts subcategory name + ID from each page
Writes lcsc/categories.json — committed to repo, refreshed quarterly

Category JSON format:

[
  {
    "group": "Passives",
    "category": "Capacitors",
    "category_id": 11,
    "subcategories": [
      {"name": "Aluminum Electrolytic Capacitors", "id": 1140},
      {"name": "Multilayer Ceramic Capacitors (MLCC)", "id": 1142},
      {"name": "Film Capacitors", "id": 1144},
      {"name": "Tantalum Capacitors", "id": 1150}
    ]
  }
]

LLM Category Resolver (lcsc/category_resolver.py):

Cache key: f"{internal_category}|{package}|{value_str}" → check lcsc/category_cache.json
On miss: pre-filter category list to relevant subtree → build Claude Haiku prompt → parse JSON response
Prompt passes component info + subcategory name list; LLM returns one exact name
Persist result to category_cache.json (same file-backed learning pattern as package_patterns.json)

Example: category=capacitor, package=6.3x5.8, voltage=25V → LLM picks "Aluminum Electrolytic Capacitors" → looked up → id 1140

LCSC Search (lcsc/lcsc.py):

Live API mode (default, no credentials needed): LCSC's real internal JSON API, reverse-engineered from live browser traffic (the site is a Nuxt SPA — the previously assumed lcsc.com/api/products/search scraper endpoint does not exist and was never actually verified working). No CSRF/session dance required, plain POST with User-Agent/Origin/Referer is sufficient:
- POST https://wmsc.lcsc.com/ftps/wm/product/query/list — paginated product search. Request: {catalogIdList:[id], brandIdList, encapValueList, paramNameValueMap, isStock, ..., currentPage, pageSize}. Response items carry LCSC part number (productCode), manufacturer (brandNameEn), package (encapStandard), tiered pricing (productPriceList), stock, per-part spec list (paramVOList), datasheet URL.
- POST https://wmsc.lcsc.com/ftps/wm/product/query/param/group — facet/filter query, same request shape (minus pagination). Response: {Package: [...], Manufacturer: [...], Packaging: [...], paramNameValueMap: {<param name>: {paramDetailList, unitList, unitConversionMap}}}, scoped to the category and any filters already applied — this is the source for building filter chips each scouting round, no separate scraping or client-side aggregation needed.
- Verified live: categories.json/category_resolver.py's subcategory ids (e.g. 1142 = "Ceramic Capacitors") are directly usable as catalogIdList values against both endpoints — no id remapping layer required.
Token mode: if api_token provided, use official LCSC API (https://ips.lcsc.com/rest/wmsc2agent/) with HMAC-signed requests — not yet implemented (NotImplementedError).

5. Scouting Loop (not yet built)

Drives each BomRow from pending to confirmed through an iterative user-guided process.

pending
  → scouting   (LLM infers LCSC subcategory, user confirms or picks different)
  → filtering  (search within confirmed subcategory + param filters, user browses results)
  → confirmed  (user picks a specific part)
  → flagged    (DNP / non-electronic / user skips)

Retry path: user rejects all results → back to scouting with modified parameters.

6. API Layer (not yet built)

FastAPI HTTP interface.

POST /upload — accept BOM file, run ingestion, return BomSession
GET /session/{id} — current session state
GET /session/{id}/row/{row_id}/scout — trigger LLM category inference, return {category, alternatives[]}
POST /session/{id}/row/{row_id}/confirm-category — user accepts/overrides category → state: filtering
POST /session/{id}/row/{row_id}/search — search within confirmed category + params, return page 1
POST /session/{id}/row/{row_id}/confirm — lock part pick, advance state to confirmed
GET /session/{id}/export — collate confirmed picks → CSV/XLSX download

7. Frontend (not yet built)

React SPA.

Upload page → triggers POST /upload
Row-by-row scouting view: category confirmation dropdown + results grid + confirm/reject buttons
Progress bar across all rows
Export button

User Flow

User uploads BOM file
  └─ Ingestion pipeline runs
       ├─ Parser reads file → raw rows
       ├─ Column Mapper normalizes column names
       └─ Normalizer classifies + extracts params → BomSession

For each pending BomRow:
  └─ Scout
       ├─ LLM infers LCSC subcategory from component info
       │    (cache hit → instant; miss → Claude Haiku call → cached)
       ├─ UI shows inferred subcategory + dropdown of all alternatives
       ├─ User confirms or picks a different subcategory
       ├─ Search within confirmed subcategory + param filters
       ├─ User browses results, adjusts filters if needed
       └─ User confirms pick  ──→  row state: confirmed
                     └─ rejects  ──→  retry with different subcategory/filters

All rows confirmed
  └─ Export → order-ready BOM (CSV/XLSX)

System Flow

POST /upload
  parse_bom(bytes, filename)
    → map_columns(raw_rows)
    → normalize(mapped_rows)
    → BomSession stored in memory

GET /session/{id}/row/{row_id}/scout
  lcsc_category_resolver.resolve(row)          ← cache hit or Haiku call
    → {subcategory: "Alum. Electrolytic...", id: 1140, alternatives: [...]}
  row.state → scouting

POST /session/{id}/row/{row_id}/confirm-category  {subcategory_id: 1140}
  row.applied_filters["lcsc_category_id"] = 1140
  row.state → filtering

POST /session/{id}/row/{row_id}/search  {filters: {voltage_min: 25, ...}, page: 1}
  lcsc.search(category_id=1140, params=row.normalized_params, filters=..., page=1)
    → {count: N, results: [...]}

POST /session/{id}/row/{row_id}/confirm  {pick: {...}}
  row.confirmed_pick = pick
  row.state → confirmed

GET /session/{id}/export
  [confirmed rows] → assemble BOM → return file

Physical File Map

instant-bom/
├── bom_assistant/
│   ├── session/
│   │   └── models.py                      — Pydantic schemas
│   │
│   ├── ingestion/
│   │   ├── parser.py                      — raw bytes → list[dict[str, str]]
│   │   ├── column_mapper.py               — raw keys → canonical keys
│   │   ├── column_synonyms.json           — file-backed synonym table (auto-updated by AI)
│   │   └── normalizer.py                  — mapped rows → list[BomRow]
│   │
│   ├── classification/
│   │   ├── classifier.py                  — designator prefix + value heuristics → ComponentCategory
│   │   ├── param_extractor.py             — value string + footprint → NormalizedParams
│   │   └── package_patterns.json          — file-backed regex + known_footprints (auto-updated by AI)
│   │
│   ├── suppliers/
│   │   ├── base.py                        — abstract SupplierAdapter
│   │   └── lcsc/
│   │       ├── fetch_categories.py        — one-time CLI: scrape LCSC → categories.json
│   │       ├── categories.json            — committed static file (refresh quarterly)
│   │       ├── categories.py              — load + filter category tree
│   │       ├── category_resolver.py       — LLM category inference + file-backed cache
│   │       ├── category_cache.json        — auto-updated by resolver
│   │       └── lcsc.py                    — search adapter (scraper + token mode)
│   │
│   └── api/                               (not yet built)
│       └── main.py                        — FastAPI app + routes
│
├── frontend/                              (not yet built)
│   └── src/
│
├── examples/
│   └── bom/                               — 12 real BOM test files (CSV, XLSX, XLS)
│
└── requirements.txt

architecture.md 12 KB History Raw