About Bolna Bolna is a YC-backed voice AI orchestration platform built for the Indian market—powering multilingual, vernacular voice agents across Hindi, Hinglish, Tamil, and 10+ languages at sub-500ms latency across collections, recruitment, sales, and e-commerce use cases. We are an orchestration layer, not a model company: our moat is outcome-labelled vernacular data, rigorous evaluation infrastructure, and a growing taxonomy of how Indian enterprise voice AI fails in production. Why This Role Exists Product decisions at Bolna increasingly hinge on rigorous, code-mixed-aware data analysis—and not just one kind. On one side, there is model and evaluation rigor: LLM benchmarking for post-call intelligence, ASR/WER evaluation, inter-rater reliability on human-labelled calls, and routing and latency economics. On the other, there is product and growth insight: understanding where self-serve users drop off in their journey, what patterns emerge across lakhs of monthly calls, and which use cases and configurations are actually working. Both currently sit with the Head of Product alongside strategy and roadmap ownership. We need a dedicated analyst to own the execution and recurring cadence across both-freeing product leadership to act on findings rather than produce them. What You’ll Do Model and Evaluation Analysis LLM and model benchmarking: Run structured comparisons across model providers such as Sarvam, DeepSeek, Gemini, and Claude variants for tasks including post-call extraction and LLM-as-judge scoring. Evaluate cost, accuracy, fill rate, and TTR, with particular attention to Hinglish and code-mixed content. Evaluation infrastructure: Build and maintain LLM-as-judge pipelines using tools such as DeepEval, design and track evaluation metrics, and run inter-rater reliability analysis such as Krippendorff’s alpha across human call reviewers. Golden dataset creation: Support the construction of golden datasets for ASR and transcript labelling, including flagging co