Captures the architectural research surfaced during the 2026-05-25
SLM-failure diagnostic session: RouteLLM treats routing as
classification, ModernBERT is well-suited to that classification, and
FunctionGemma fits as an optional JSON-sanity layer rather than the
primary classifier. The current decoder-SLM-as-classifier design is
the wrong shape (100% failure rate observed across two model swaps).
Five-phase plan:
1. Embedding feature scaffold (near-term, additive, opt-in)
2. Contextual bandit (LinUCB / Thompson) over the feature set
3. Retire the decoder-SLM classifier once 2 outperforms
4. ModernBERT fine-tune on the accumulated labelled data
5. FunctionGemma JSON sanity layer (optional final stage)
Phase 1 is the only piece scoped for near-term implementation; the
rest is multi-month and hinges on the strategic 'EMA vs SLM'
question already tracked in TODO.
Cross-references the existing tool-router-specialization plan so a
reader of either lands on both. Updates the TODO entry for the
bandit selector to note the supersession path.
vikingowl
24945b1eb2