session-eval
A dual-mode AI evaluation harness that compares model and speech-to-text outputs on real session data to drive a defensible engine-selection decision.
/ The Challenge
Make a defensible, data-backed recommendation on which ASR (speech-to-text) engine to adopt, and evaluate LLM outputs against real session transcripts — reproducibly, and without depending on live services during a demo.
/ Approach
- 01
Built a Streamlit harness that runs in two modes: live evaluation, and fully offline replay from cached runs for reproducible demos.
- 02
Structured side-by-side comparisons across models and ASR engines using consistent, transparent metrics.
- 03
Packaged the analysis into an ASR-selection deliverable spanning a Boostlingo interview scenario and the SanaMente use case.
/ Outcomes
-
A reproducible, offline-capable evaluation that runs the same way on any machine.
-
A clear, data-backed ASR-selection recommendation instead of a gut call.
-
A reusable harness that can be pointed at new models or engines as they emerge.