Senior AI Evals Engineer

P-926 The Applied AI team at Databricks sits at the forefront of advancing AI/ML-powered products. Databricks’ customers are continuously creating new assets (tables, notebooks, dashboards, datarooms, pipelines, sql queries, ml models etc.) on the platform. Finding an asset is a critical user journey for Databricks’ customers which helps them accomplish their tasks. As our Search product continues to evolve, we are seeking multiple senior engineers to drive enhancements to our Search Quality. In 2026, we will focus on enhancing search ranking, improving query understanding, building robust evals and growing the coverage of assets to enable seamless search at scale. Key Responsibilities Drive the development and deployment of ML based search and discovery relevance models and systems integrated with Databricks' products and services. Design and implement automated ML and NLP pipelines for data preprocessing, query understanding and rewrite, ranking and retrieval, and model evaluation, enabling rapid experimentation and iteration. Collaborate with product managers and cross-functional teams to drive technology-first initiatives that enable novel business strategies and product roadmaps for the search and discovery experience. Contribute to building a robust framework for evaluating search ranking improvements - both offline and online. What We’re Looking For BS+ (M.S. or PhD preferred) in Computer Science, or a related field. 5+ years experience developing search relevance systems at scale in production or in high-impact research environments. Experience applying LLM to search relevance Experience in one or more of the following: Query understanding NLP Text mining Recommendations Personalization Discovery Conversational AI Strong understanding of computer science fundamentals. Contributions to well-used open-source projects. Why Join Us? At Databricks, we are building state-of-the-art AI solutions that red

Who We Are Simpplr is the AI-powered intranet for unifying the digital workplace. It brings people, trusted knowledge, apps, and agents into a coherent digital experience. Powered by a proprietary EX Knowledge Graph, Simpplr synthesizes signals and context across connected systems to deliver personalized information and actions. The platform serves as a digital hub supporting communications, engagement, employee services, and work. With low-code extensibility and enterprise-grade security and governance, Simpplr enables confident operation at scale. More than 1,000 organizations — including AAA, the NHS, Penske, and Moderna — trust Simpplr to keep their workforce informed, aligned, and productive. Learn more at simpplr.com . The Opportunity We're looking for a highly skilled Senior Product Manager – Enterprise Search to join our Product Management team . In this role, you'll own the strategy, definition, and execution of enterprise search across Simpplr — powering both Magnus (our AI assistant) and the Simpplr platform. You'll shape how employees find the right knowledge, content, and answers across the entire enterprise, quickly, accurately, and securely. Search is the connective tissue of the employee experience: it is what makes Magnus's answers trustworthy and what makes the Simpplr platform feel intelligent. You'll be responsible for the end-to-end search experience — from how content is ingested, chunked, and indexed, to how it is retrieved, ranked, and grounded into high-quality, permission-aware answers via retrieval-augmented generation (RAG). You'll work closely with AI engineers, search/ML engineers, data scientists, designers, and cross-functional teams to deliver a search foundation that is fast, relevant, and enterprise-grade. This role requires strong technical fluency in modern search and AI concepts — indexing, chunking, embeddings, vector and hybrid retrieval, ranking, and RAG — along with the product judgment to translate them into measurabl

For over 20 years, Smartsheet has empowered teams to manage work seamlessly and scale solutions smarter. Now, in our most ambitious chapter yet, we are uniting human teams with AI agents. By orchestrating the work agents do best, automating manual tasks and uncovering insights at scale, we create the space for people to focus on what truly matters: judgment, creativity, and big thinking. That is magic at work, and it’s what we show up for every day. For over 20 years, Smartsheet has empowered teams to manage work seamlessly and scale solutions smarter. Now, in our most ambitious chapter yet, we are uniting human teams with AI agents. By orchestrating the work agents do best, automating manual tasks and uncovering insights at scale, we create the space for people to focus on what truly matters: judgment, creativity, and big thinking. That is magic at work, and it’s what we show up for every day. Job Description/ Responsibilities: 7-12 Years of Experience in Designing, developing and maintaining stable and reliable AI/ML Ops platforms / pipelines Model Deployment: Package and deploy AI/ML services to production, ensuring they are reproducible and interpretable CI/CD Pipeline Development: Design and implement automated CI/CD (Continuous Integration/Continuous Deployment) pipelines to accelerate model deployment using tools Infrastructure Management: Provision and optimize infrastructure for training and serving, utilizing Docker, Kubernetes, or serverless platforms Monitoring & Observability : Implement post-deployment monitoring for model performance, data drift, and latency using tools. Experience in Monte Carlo is preferable Automation: Automate retraining and data pipeline workflows to ensure models stay accurate over time. Manage the deployment of foundation models, fine-tuning workflows, and Retrieval-Augmented Generation (RAG) stacks (Vector DBs, Knowledge Graph. Experience with AWS Bedrock is preferable Resource Optimization: Manage GPU/CPU utilization to m

Title: Senior AI QA Engineer Location: Bengaluru (Bangalore) Opportunity: As a Senior AI QA Engineer for our Precision Patient Care Pipeline , you will go beyond traditional functional testing. You will be responsible for building the framework that ensures our clinical insights are accurate, safe, and reliable. This role requires a unique blend of high-level software testing and data engineering to validate complex, non-deterministic medical outputs using a hybrid of automated grading methodologies . Key Responsibilities: Architect Multi-Layered Validation Frameworks: Design and implement structured testing strategies that combine deterministic checks, semantic similarity metrics, and model-based evaluations. Automated Model Grading: Develop systems to evaluate clinical pipeline outputs for faithfulness, safety, and hallucination detection using various automated scoring techniques (e.g., BERTScore, ROUGE, or custom heuristics). Vibe-Driven Development: Leverage agentic AI tools to rapidly prototype complex test harnesses, "red-team" clinical logic, and build internal validation utilities at high velocity. Data Pipeline Integrity: Execute integration and regression tests for data-heavy backend processes, ensuring medical data remains consistent from ingestion to insight generation. Collaborative Strategy: Work closely with Data Scientists and Product Managers to define "Ground Truth" datasets and clinical evaluation rubrics. Root Cause Analysis: Deep-dive into complex system failures to identify whether issues stem from code logic, data drift, or model behavior. Requirements: 6+ years of technical experience in Quality Assurance, with a strong focus on system architecture and backend data validation. Advanced Python Proficiency: Expert-level skills in Python for building custom test scripts and working within AI/ML ecosystems. AI/ML Validation Experience: Proven experience testing model outputs using diverse metrics (e.g., Semantic Similarity, NLP metrics, an

ABOUT SINGLESTORE & THE TEAM SingleStore is the Intelligence Platform powering real-time data, analytics, and AI experiences at enterprise scale. Internally, we practice what we preach through our AI & Automation Center of Excellence (CoE): a unit that partners with cross-functional teams to identify high-friction business problems and solve them using AI, automation, and data. As a Senior AI & Automation Manager, you will be overseeing the AI CoE and working directly with executive leadership to tackle the company's highest-impact problems. WHAT YOU'LL DO ● AI agent development: Building and refining AI agents (e.g., Glean or custom agents) to automate knowledge retrieval, extract intelligence, and support decisions. ● Automation workflows: Design and build automation pipelines to eliminate manual and repetitive processes. ● Data analysis & pipelines: Write Python and SQL to extract, transform, and analyse data from internal systems (Salesforce, data warehouse, etc.), including building scoring and prioritization models. ● Stakeholder discovery: Engage with cross-functional teams to understand pain points, document requirements, and translate them into structured problem statements. ● Process improvement: Identify workflow inefficiencies, propose solutions, and drive implementation end-to-end with strong ownership. ● Project management: Independently scope, sequence, and deliver technical projects. ● Enablement/Team management: Guide or mentor junior team members on technical execution, and act as the go-to resource for AI/automation capability across the org QUALIFICATIONS / SKILLS Background 4-6 years of experience in AI, automation, data science or analytics Bachelor's degree in Computer Science Engineering, or a related technical field Preferred background in SaaS, particularly working with cross-functional teams Technical Competencies Proficient in Python (Pandas, NumPy, Scikit-learn), able to

For over 20 years, Smartsheet has empowered teams to manage work seamlessly and scale solutions smarter. Now, in our most ambitious chapter yet, we are uniting human teams with AI agents. By orchestrating the work agents do best, automating manual tasks and uncovering insights at scale, we create the space for people to focus on what truly matters: judgment, creativity, and big thinking. That is magic at work, and it’s what we show up for every day. Job Description/ Responsibilities: Designing, developing and maintaining stable and reliable AI/ML Ops platforms / pipelines Model Deployment: Package and deploy AI/ML services to production, ensuring they are reproducible and interpretable CI/CD Pipeline Development: Design and implement automated CI/CD (Continuous Integration/Continuous Deployment) pipelines to accelerate model deployment using tools Infrastructure Management: Provision and optimize infrastructure for training and serving, utilizing Docker, Kubernetes, or serverless platforms Monitoring & Observability : Implement post-deployment monitoring for model performance, data drift, and latency using tools. Experience in Monte Carlo is preferable Automation: Automate retraining and data pipeline workflows to ensure models stay accurate over time. Manage the deployment of foundation models, fine-tuning workflows, and Retrieval-Augmented Generation (RAG) stacks (Vector DBs, Knowledge Graph. Experience with AWS Bedrock is preferable Resource Optimization: Manage GPU/CPU utilization to minimize cloud costs while maintaining low-latency inference for users Collaboration: Work closely with data scientists, data engineers, and software engineers to bridge the gap between model development and production. Version Control & Governance: Manage versioning for data, code, and models using tools like MLflow. Security & Compliance: Implementing data security measures, ensuring compliance with data governance policies, and protecting sensitive data Technology Eva