Senior Program Manager Eval Operations in San Francisco

About the Role At Sigma, we’re not just adding AI—we’re building the future of how people work with data. Our platform already lets users explore billions of rows of data in seconds with a spreadsheet-like interface, analyze and present their data in workbooks, and build data apps and workflows. Now we’re pushing further, applying AI to reshape how people build in Sigma, discover insights, and make smarter decisions—fast. That’s where you come in. As an AI/ML Engineer, you’ll join a growing team focused on building the AI foundation that will power Sigma for the future. Your work will become an integral part of the workflow for the thousands of enterprises that run on Sigma. What You’ll Do Partner with product, design, and engineering teams to identify high-impact AI/ML opportunities Prototype and productionize AI systems that feel intuitive but do a lot under the hood—recommendations, natural language interfaces, agentic workflows, and more Develop and scale AI/ML infrastructure that powers both internal tooling and customer-facing features Tackle novel UX problems at the intersection of AI, BI, and apps What You Bring Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field (required) 10+ years of experience building and deploying production-grade AI/ML systems Deep knowledge of machine learning, deep learning, and applied AI Experience across the full ML lifecycle: data curation, training, deployment, monitoring A track record of building things that ship—whether it’s recommendations, search, machine translation, or something equally complex Experience adapting or training foundation models (language or multimodal) for novel domains Bonus Points (or skills you’ll build here) You've built agents that can plan, reason, and use tools You know your way around cloud infrastructure (AWS, GCP, Azure) You’ve worked in a fast-moving startup or high-growth environment Additional Job details The base salary range for this posit

Sigma is transforming how businesses allow customers to build apps, agents and dashboards on top of governed enterprise data. Hence, we are growing the design team and looking for designers who are excited to solve challenging problems, deliver impactful capabilities throughout our stack to build world-class technology. You will be part of a talented team of designers with a shared mission to make data easily accessible for all users. We're looking for a Senior Product Designer / Design Engineer who sits at the intersection of interaction design and AI engineering: someone who uses AI to ship faster, builds the skills and evals that make AI more effective, and invents new interaction paradigms for how people work alongside intelligent systems. This isn't a traditional design role. Yes you'll be using Figma, but also writing code with AI, training it, evaluating it, and questioning every assumption about what a "UI" can be when the interface itself reasons. Please note this is a 4 day on-site role in our San Francisco office. What You'll Do Start with AI, stay with AI. Use LLMs to clarify scope, draft specs, surface edge cases, and align your team before committing to a direction, use AI coding tools to build and iterate on the solution itself, and merge code to prod when fits. Prototype in code. Build working interfaces with Cursor and Claude Code, guiding structure, behavior, interaction, motion and UX quality while AI handles implementation. Partner directly with engineering to decide what moves into the product and what stays as a validated spike. Bring it to production. Fix small interaction and refinement issues directly on prod code. Design new AI interaction paradigms for conversational interfaces. Invent and validate novel patterns for how users converse with, direct, and trust AI systems - especially in data contexts where precision and confidence matter. Write evals, skills, and help on tools. Build the scaffolding that makes AI reliabl

Senior Software Engineer - Observability and Reliability About the Role We are growing the engineering team and looking for engineers who have the chops to build and deliver world-class technology. You will be part of a talented team of engineers with a shared mission to make data easily accessible. What You Will Be Doing Build observability tools and platforms, including: metrics, logging, distributed tracing, dashboarding, alerting, application performance management Build with modern tools and languages like Go, Open Telemetry and Kubernetes Participate in on-call rotation and ensure uptime of services Create runtime tools/processes that optimize cloud triaging and limit downtime Define best practices around making our systems and services measurable Collaborate with peers and stakeholders through design and code reviews to ensure best practices amongst available technologies. We expect successful candidates to be coding a majority of their time Qualifications We Need Strong Computer Science fundamentals 5+ years industry experience building and maintaining high-quality software, especially software other engineers use You apply a product mindset to infrastructure systems and feel accomplished enabling others Desire to be a great teammate and have fun at work Strong sense of craftsmanship, and a healthy academic curiosity Qualifications We Want (also, skills you’ll learn!) Experience building systems for data analytics Distributed systems monitoring and profiling skills Knowledge of cloud application security models Administered cloud service infrastructure (GCP, AWS, Azure) Startup experience Additional Job details Additional Job details The base salary range for this position is $170k - $240k annually. Compensation may vary outside of this range depending on a number of factors, including a candidate’s qualifications, skills, competencies and experience. Base pay is one part of the Total Package that is provided to compensate and recognize e

About the Role Sigma is transforming how businesses run by delivering a high performance platform on modern data architecture. Hence, we are growing the engineering team and looking for engineers who are excited to solve challenging problems, deliver impactful capabilities throughout our stack to build world-class technology. You will be part of a talented team of engineers with a shared mission to make data easily accessible for all users. What You Will Be Doing You will be responsible for developing elegant and responsive user experience using the latest front-end technologies. You'll own substantial pieces of the product, from design to launch Working with our product, UX design, and backend development teams, you will develop new features and technologies that make our product experience awesome and radically simplify the user experience for non-technical users You will leverage your technical expertise in front-end application development in the creation of novel visualizations for structured and unstructured data and develop new techniques for improving the performance and interactivity of the application Use modern frontend frameworks like React, GraphQL, TypeScript and Node.js Qualifications We Need 5+ years industry experience building and maintaining high-quality software An eye for great design and a passion for building products that provide a great user experience The ability to make the right trade-offs between functionality and delivery speed that supports delivering value to customers, all the while iterating based on feedback and roadmap priorities Desire to be a great teammate and have fun at work without compromising ownership towards your work Strong sense of craftsmanship, and a healthy academic curiosity to solve challenges at sigma Strong computer science fundamentals Qualifications We Want (also, skills you’ll learn!) Experience building software capabilities for analyzing large scale data web applications Prior exp

About the Role Sigma is transforming how businesses run by delivering a high performance platform on the modern data architecture. Hence, we are growing the engineering team and looking for engineers who are excited to solve challenging problems, deliver impactful capabilities throughout our stack to build world-class technology. You will be part of a talented team of engineers with a shared mission to make data easily accessible for all users. What You Will Be Doing Solve challenging problems that arise in providing an interactive experience on data warehouses for data exploration and analysis Build with modern tools and languages like Rust, Go, GraphQL, Node, and Kubernetes Build backend distributed services, new algorithms and modern API to support a cloud application Triage product or system issues and debug/track/resolve by analyzing the sources of issues Design and implement new software features to support our fast growing user base Collaborate with peers and stakeholders through design and code reviews to ensure best practices amongst available technologies Qualifications We Need 5+ years industry experience building and maintaining high-quality software Experience building and deploying robust and secure web applications in a continuous deployment environment Desire to be a great teammate and have fun at work without compromising ownership towards your work Strong sense of craftsmanship, and a healthy academic curiosity to solve challenges at sigma Strong Computer Science fundamentals Qualifications We Want (also, skills you’ll learn!) Data driven aptitude and its application to solve distributed system problems Data model design, and API development experience SQL query optimization and database internals Administered cloud service infrastructure (GCP, AWS, Azure) Prior experience working at high growth company solving technical problems to enable continued success Additional Job details The base salary range for this position is $170k - $240

About Scale AI Scale AI is the data foundation for AI, helping organizations build and deploy reliable production AI applications. We partner with leading enterprises and government organizations to accelerate their AI initiatives through our data annotation platform, generative AI solutions, and enterprise AI capabilities. Role Overview As a Senior Staff Frontier Agents Engineer on our Enterprise team, you'll be the technical bridge between Scale AI's cutting-edge AI capabilities and our most strategic customers. You'll work with enterprise clients to understand their unique challenges, architect custom AI solutions, and ensure successful deployment and adoption of AI systems in production environments. This is a hands-on technical role that combines deep engineering expertise with customer-facing problem solving. You'll work directly with customer engineering teams to integrate AI into their critical workflows. Key Responsibilities Customer Integration & Deployment Partner directly with enterprise customers to understand their technical infrastructure, data pipelines, and business requirements Design and implement custom integrations between Scale AI's platform and customer data environments (cloud platforms, data warehouses, internal APIs) Build robust data connectors and ETL pipelines to ingest, process, and prepare customer data for AI workflows Deploy and configure AI models and agents within customer security and compliance boundaries AI Agent Development Develop production-grade AI agents tailored to customer use cases across domains like customer support, data analysis, content generation, and workflow automation Architect multi-agent systems that orchestrate between different models, tools, and data sources Implement evaluation frameworks to measure agent performance and iterate toward business objectives Design human-in-the-loop workflows and feedback mechanisms for continuous agent improvement Prompt Engineering & Optimization Create sophisticate

The Public Sector software engineers (SWEs) create the core product building blocks forward-deployed teams use to develop agentic capabilities that function across multiple domains. SWEs responsibilities include building the systems required to ingest and process federal datasets to support real-time decision-making in contested environments. We develop novel agentic enabling capabilities that includes: Create multi-layered guardrails around agents Optimize data retrieval for agents Orchestrate fleets of asynchronous agents Automatically alerts users to deviations in data Illustrating how an agent reached a decision As a Senior Software Engineer, you will lead the development of a vertical feature or a horizontal capability to include defining requirements with stakeholders and implementation until it is accepted by the stakeholders. You will: Lead the design and implementation of scalable backend systems and distributed architectures for Federal customers. Manage the full lifecycle of feature development from requirement definition to deployment on classified networks. Direct the orchestration of asynchronous agent fleets to meet mission requirements. Lead customer engagements to translate mission needs into technical requirements. Own the communication with stakeholders to ensure implementation meets defined acceptance criteria. Conduct technical reviews and identify risks within machine learning infrastructure and model serving. Drive the platform roadmap by providing technical specifications for Federal product offerings. Ideally you will have: Full Stack Development: Proficiency in front-end, back-end development and infrastructure, including experience with modern web development frameworks, programming languages, and databases Cloud-Native Technologies: Familiarity with cloud platforms (e.g., AWS, Azure, GCP) and experience in developing and deploying applications in a cloud-native environment. Understanding of containerization (e.g., Docker) and contai

Software is eating the world, but AI is eating software. We live in unprecedented times – AI has the potential to exponentially augment human intelligence. Every person will have a personal tutor, coach, assistant, personal shopper, travel guide, and therapist throughout life. As the world adjusts to this new reality, leading platform companies are scrambling to build LLMs at billion scale, while large enterprises figure out how to add it to their products. To make them safe, aligned and actually useful, these models need human eval and reinforcement learning through human feedback (RLHF) during pre-training, fine-tuning, and production evaluations. This is the main innovation that’s enabled ChatGPT to get such a large headstart among competition. At Scale, our products include the Generative AI Data Engine, SGP, Donovan, and others that power the most advanced LLMs and generative models in the world through world-class RLHF, human data generation, model evaluation, safety, and alignment. The data we are producing is some of the most important work for how humanity will interact with AI. At the foundation of these products is the Platform Engineering team. In this role, you will support the design and development of shared platforms used across Scale. This includes designing our foundational data platforms and lifecycle, architecting Scale’s core cloud infrastructure and orchestration stack, and redefining how engineers develop, build, test, and deploy software at Scale. You’ll also get widespread exposure to the forefront of the AI race as Scale sees it in enterprises, startups, governments, and large tech companies. You will: Drive the design, and implementation of our foundational platforms and systems, working closely with stakeholders and internal customers to understand and refine requirements. Collaborating with cross-functional teams to define, design, and deliver new features. Proactively identifying opportunities for, and driving improvements to, current p

About Scale At Scale AI, our mission is to accelerate the development of AI applications. For 8 years, Scale has been the leading AI data foundry, helping fuel the most exciting advancements in AI, including: generative AI, defense applications, and autonomous vehicles. With our recent Series F round, we’re accelerating the abundance of frontier data to pave the road to Artificial General Intelligence (AGI), and building upon our prior model evaluation work with enterprise customers and governments, to deepen our capabilities and offerings for both public and private evaluations. About Data Engine Our Generative AI Data Engine powers the world’s most advanced LLMs and generative models through world-class RLHF (Reinforcement Learning with Human Feedback), human data generation, model evaluation, safety, and alignment. The data we are producing is some of the most important work for how humanity will interact with AI. Our Approach As part of the interview process, you’ll be considered for opportunities across several teams within the GenAI Engineering organization, based on your interests, expertise, and business needs. Potential team placements include Allocation, Growth, Frontier Data, Trust & Safety, Pay, Operator, or Tasking Experience. Together, these teams power Scale’s AI data operations - from building high-impact datasets that push the boundaries of LLM capabilities, to optimizing contributor onboarding and incentives, to safeguarding data integrity through advanced trust, safety, and security measures. They work at the intersection of ML, operations, and analytics to ensure we deliver the highest-quality data at scale. Responsibilities: Design, build, and maintain robust, scalable systems across the full stack, including front-end, back-end, and infrastructure layers Implement high-impact features using modern technologies such as TypeScript, React, Node.js, MongoDB, Elasticsearch, and Temporal Collaborate closely with internal operators (your use

Scale GP (Scale Generative AI Platform) is an enterprise-grade Generative AI platform providing APIs for knowledge retrieval, inference, evaluation, and more. We are seeking a strong Senior Full-Stack Engineer to help us build, scale, and refine our rapidly growing product. The ideal candidate is deeply grounded in software engineering best practices and experienced in developing and scaling modern web applications end-to-end. You will work across the stack—from React/TypeScript frontends to Python-based backends—while integrating with LLMs and machine learning systems. You will solve complex challenges in scalability, reliability, and product experience while owning significant product areas in a fast-paced environment. What You’ll Do Own major full-stack product areas , driving features from design through production deployment. Build modern frontend experiences using React and TypeScript, ensuring performance, usability, and responsiveness. Develop reliable backend services in Python, working with distributed systems, data pipelines, and ML/LLM components. Integrate with LLMs, vector databases, and AI infrastructure to power intelligent product experiences. Deliver experiments and new features quickly , maintaining high quality and tight feedback loops with customers. Collaborate across product, ML, and infrastructure teams to shape the direction of Scale GP. Adapt quickly —learning new technologies, frameworks, and tools as needed across the stack. Ideal Experience 5+ years of full-time engineering experience , post-graduation. Strong experience developing full-stack applications using React, TypeScript, and Python . Experience scaling or shipping products at high-growth startups . Familiarity with LLMs, vector databases, embeddings, or other modern AI tooling (tinkering or production experience welcome). Proficiency with SQL and modern API development. Experience with Kubernetes , containerization, and microservice architectures. Experience working with at leas

Scale AI is the data foundation for AI, helping organizations build and deploy reliable production AI applications. We partner with leading enterprises and government organizations to accelerate their AI initiatives through our data annotation platform, generative AI solutions, and enterprise AI capabilities. About the General Agents Team The General Agents team, part of Scale’s Enterprise organization, builds robust general agents for customer use cases and applications. The team sits at the intersection of frontier agent development and real-world deployment, translating state-of-the-art reasoning and agentic capabilities into reliable, production-grade systems that drive real economic value. Our agents are scalable systems built around recurring enterprise problem domains, with a strong emphasis on generalization, extensibility, and deployment across many customers. About the Role As a Senior/Staff Machine Learning Engineer (MLE) on the General Agents team, you’ll play a critical role in designing, building, and deploying production-ready AI agents that solve high-impact enterprise problems. You will work across the full agent lifecycle—from model and system design to evaluation, deployment, and iteration—bridging cutting-edge agentic techniques with the constraints and requirements of real customer environments. You will: Design and implement end-to-end agent systems that combine LLM reasoning, tool use, memory, and control logic to solve recurring enterprise use cases. Build scalable, reliable agent architectures that can be deployed across many customers with varying data, tools, and constraints. Develop evaluation frameworks, datasets, environments, and metrics to measure agent performance, reliability, and business impact in production settings. Collaborate closely with product managers, customers, data annotators, and other engineering teams to translate enterprise requirements into robust agent designs. Productionize frontier agent techniques (e.g.,

Software is eating the world, but AI is eating software. We live in unprecedented times – AI has the potential to exponentially augment human intelligence. Every person will have a personal tutor, coach, assistant, personal shopper, travel guide, and therapist throughout life. As the world adjusts to this new reality, leading platform companies are scrambling to build LLMs at billion scale, while large enterprises figure out how to add it to their products. To make them safe, aligned and actually useful, these models need human eval and reinforcement learning through human feedback (RLHF) during pre-training, fine-tuning, and production evaluations. This is the main innovation that’s enabled ChatGPT to get such a large headstart among competition. At Scale, our products include the Generative AI Data Engine, SGP, Donovan, and others that power the most advanced LLMs and generative models in the world through world-class RLHF, human data generation, model evaluation, safety, and alignment. The data we are producing is some of the most important work for how humanity will interact with AI. At the foundation of these products is the Identity Engineering team. In this role, you will help support the design and development of core software systems specifically focused on identity, access management, authorization, and authentication. You’ll also get widespread exposure to the forefront of the AI race as Scale sees it in enterprises, startups, governments, and large tech companies. You will: Drive the design, and implementation of our identity infrastructure to ensure secure authentication and authorization across enterprise systems. Build software for authentication mechanisms such as Single Sign-On (SSO), Multi-Factor Authentication (MFA), and federated identity solutions (SAML, OAuth, OpenID Connect). Build software for authorization mechanisms such as Relation-based access control (ReBAC), Attribute-based access control (ABAC), Role-based access cont

About Scale AI Scale AI is the data foundation for AI, helping organizations build and deploy reliable production AI applications. We partner with the world's leading enterprises and government organizations to accelerate their AI transformation through frontier AI systems that solve real business problems. Every day, we work with organizations across finance, healthcare, manufacturing, media and telecommunications to build production AI agents that automate complex workflows, help humans, reason over enterprise knowledge, and operate safely at scale. The Opportunity Applied AI is moving faster than ever. New foundation models, reasoning techniques, agent architectures, and research papers emerge every week. Yet building AI systems that reliably solve real-world problems remains one of the hardest engineering challenges. As a Senior Frontier Agent Engineer (Applied AI) , you'll bridge the gap between cutting-edge AI research and production deployment. You'll work directly with enterprise customers to design, evaluate, and deploy intelligent systems that combine frontier models with structured knowledge, retrieval, traditional machine learning, and enterprise software. Unlike traditional ML roles that focus on a single model or product, you'll work across a diverse portfolio of AI challenges spanning multiple industries and use cases. You may build a multi-agent research system and then participate in designing a customer intelligence platform, a healthcare copilot, or an autonomous workflow for a Fortune 100 company. If you enjoy reading new AI papers, experimenting with the latest models, and shipping production systems that create measurable business impact, you'll fit right in. What You'll Build Frontier AI Systems Design and deploy production AI agents that leverage the latest advances in large language models, reasoning, retrieval, memory, and tool use. Architect intelligent systems that combine LLMs, traditional machine learning, structured knowledge, enterpri

Scale AI is seeking a highly motivated Senior Accountant to join our growing accounting team and assist in preparing day to day corporate accounting operations, supporting the month end close process, and helping to implement systems and processes that will support Scale as we continue to grow. You will also gain exposure to working with our international entities as we continue our expansion globally, working on some of the highest value aspects of the busienss. The ideal candidate thrives in a high-growth start-up, is detail-oriented, and has excellent interpersonal and communication skills. Additionally, the candidate has demonstrated the ability to build scalable cross-functional relationships through systems and process implementation. We hope you will join our team! You Will: Prepare journal entries and day to day corporate accounting activities, support the month end close process, and provide timely and accurate month-end close financials that are U.S. GAAP compliant Build or enhance balance sheet account reconciliation workpaper including reviewing and performing some clean-up of historical reconciliations and related balances Collaborate within Accounting and Finance teams on metrics, flux analysis, forecast, and projections and support preparation of the monthly reporting package Prepare documents supporting internal and external audits and ensure the successful completion of those audits Support the implementation of new systems, tools, and processes to streamline close and build scalable solutions to support the growth of the Company Identify and drive process improvements to gain efficiencies and reduce close timeline Develop, maintain and improve internal controls which relate to assigned areas Ideally You Have: Bachelor’s degree in Accounting; CPA or in the process of working towards one is preferred. 3+ years of relevant accounting experience; Combination of public accounting and industry experience preferred. Strong knowledge of U.S. GAAP. Ex

Scale has been the leading AI data foundry, helping fuel the most exciting advancements in AI, including frontier model training, enterprise adoption, defense applications, and autonomous vehicles. Our mission is to develop reliable AI systems for the world’s most important decisions. We’re looking for an AI Product Manager to own the Finance vertical within our Agents Data & Reinforcement Learning Environments team. In this role, you’ll own both the development of RL environments (the realistic, high-fidelity simulations of financial software and workflows that labs use to train and evaluate agents) and the “data as a product” strategy that powers them. You’ll understand where AI is being used in the Finance industry, decide what financial tasks are worth modeling, how to source and structure the underlying data, and how to turn deep domain knowledge into a defensible product. The ideal candidate has lived inside the Finance industry, and is able to pair that domain understanding with a sense for AI research and current agent capabilities in Finance workflows. You’ll translate that expertise into environments and datasets that teach AI agents to perform real financial work, and you’ll be the domain expert Scale’s most important customers and their leading researchers turn to. A strong entrepreneurial & go-to-market mindset will be necessary. What You’ll Do Own the Finance AI roadmap & data strategy: Set product direction for the Finance agents training stack and the data strategy behind it. Establish a vision for where AI is continuing to transform the Finance industry (including investment banking, private equity, public markets, corporate finance, FP&A, etc), driving execution across engineering, operations, and go-to-market teams. Build partnerships with research teams at frontier labs: Work directly with researchers at leading AI labs to understand where their Finance agentic capabilities fall short and shape new product lines and competit