Where AI Actually Belongs in Production Systems

Most AI demos put the model at the center. In production, the model is step 3 of 4.

FoodNet matches raw user input — "chkn brst", "organic free-range chicken", a blurry receipt photo — to a canonical database of 7,003 USDA food items. This is the core of the entire system. Every feature downstream (nutrition, recipes, recommendations) depends on this match being correct.

Here's how the pipeline actually works.

The 4-Tier Matching Pipeline

Every food input runs through four steps, in order. The moment one succeeds, the rest are skipped:

1. Redis Canonical Lookup → confidence 1.0
2. Postgres Alias Lookup → confidence 0.95
3. Fuzzy Match (RapidFuzz) → confidence 0.85–0.95
4. No Match → confidence 0.0

The first two steps are deterministic. No model, no API call, no latency. A Redis lookup takes <1ms. The Postgres alias check takes ~3ms. Together, these handle 90%+ of all matches.

Step 3 — fuzzy matching — uses RapidFuzz's token_set_ratio with word-count adaptive thresholds:

# Word-count adaptive thresholds
2-word inputs: 92%  (e.g., "chicken breast")
3-word inputs: 89%  (e.g., "low fat milk")
4+ words:      91%  (e.g., "extra virgin olive oil")

These thresholds were tuned through failure. More on that in a later post.

Where AI Enters the Picture

AI doesn't match foods. AI prepares the input so the deterministic pipeline can match it.

Before the matching pipeline ever sees an input, two things happen:

1. TextCleaner detects the input modality (receipt OCR, photo OCR, speech, text) using regex signals, then applies modality-specific noise removal. Prices, filler words, nutrition labels — all stripped before the LLM sees anything.

2. LLM Parser takes the cleaned text and returns structured data: generic_name, quantity, unit. It also expands vague categories ("spices" becomes [salt, pepper, garlic powder, onion powder, paprika, cumin, oregano, chili powder]) and marks ambiguous items for the UI to surface as suggestions.

The LLM call costs ~500ms and ~$0.001. The deterministic lookup that follows costs 0ms and $0.

The Persistence Threshold: A $0 Bug Fix

Early on, every successful fuzzy match was cached as an alias for faster future lookups. This seemed smart until we discovered that "apple" had been fuzzy-matched to a cheese variant at 87% confidence and cached. Every subsequent "apple" query returned cheese.

The fix: only persist fuzzy matches with scores ≥99 as aliases. Below that, the match is returned but never cached. One line of code, zero model calls, and the system stopped poisoning its own data.

Source Tier Ranking

Not all food data is equal. Our 7,003 items come from multiple sources, ranked:

Tier	Source	Count	Quality
0	Curated (manual)	~50	Highest — hand-verified
1	USDA Foundation	361	Gold standard
2	USDA SR Legacy	6,642	Comprehensive
3	Open Food Facts	(search fallback)	Variable
4	USDA Branded	(search fallback)	Lowest signal

Every query orders by source_tier ASC, manually_verified DESC, usage_count DESC. The best data always wins.

The Lesson

The AI in this system is powerful — it handles OCR noise, expands vague inputs, and structures messy text. But it's not the source of truth. The source of truth is a deterministic lookup against verified data.

If your AI system can answer "what happens when the model is wrong?", you're building for production. If it can't, you're building a demo.

---

Next up: LLMs Are Not Data Cleaners (Until You Force Them to Be) — how we handle 4 input modalities with 40+ regex patterns before the LLM ever sees the text.