Quality Operator in San Francisco

Scale works with the industry’s leading AI labs to provide high quality data and accelerate progress in GenAI research. We are looking for Research Scientists and Research Engineers with expertise in LLM post-training (SFT, RLHF, reward modeling). This role will focus on optimizing data curation and eval to enhance LLM capabilities in both text and multimodal modalities. In this role, you will develop novel methods to improve the alignment and generalization of large-scale generative models. You will collaborate with researchers and engineers to define best practices in data-driven AI development. You will also partner with top foundation model labs to provide both technical and strategic input on the development of the next generation of generative AI models. You will: Research and develop novel post-training techniques, including SFT, RLHF, and reward modeling, to enhance LLM core capabilities in both text and multimodal modalities. Design and experiment new approaches to preference optimization. Analyze model behavior, identify weaknesses, and propose solutions for bias mitigation and model robustness. Publish research findings in top-tier AI conferences. Ideally you’d have: Ph.D. or Master's degree in Computer Science, Machine Learning, AI, or a related field. Deep understanding of deep learning, reinforcement learning, and large-scale model fine-tuning. Experience with post-training techniques such as RLHF, preference modeling, or instruction tuning. Excellent written and verbal communication skills Published research in areas of machine learning at major conferences (NeurIPS, ICML, ICLR, ACL, EMNLP, CVPR, etc.) and/or journals Previous experience in a customer facing role. Compensation packages at Scale for eligible roles include base salary, equity, and benefits. The range displayed on each job posting reflects the minimum and maximum target for new hire salaries for the position and may be inclusive of several career levels at Scale; it will be determined du

Scale works with the industry's leading AI labs to provide high quality data and accelerate progress in GenAI research. We are looking for Research Scientists and Research Engineers with expertise in LLM post-training (SFT, RLHF, reward modeling) and evaluation. This role is on the evaluation pod within the GenAI Research Organization and will focus on building benchmarks and diagnosing model failure modes in both text and multimodal modalities. In this role, you will develop rigorous evaluations and diagnostic methods that reveal where frontier models fail and why. You will collaborate with researchers and engineers to define best practices in evaluation-driven AI development. You will also partner with top foundation model labs to translate failure analysis into technical and strategic input on the next generation of generative AI models. You will: Analyze model behavior to identify, characterize, and diagnose failure modes in frontier LLMs and Agents. You’ll identify everything from capability gaps and reasoning errors to robustness and alignment issues, all focusing on RCA. Design and build benchmarks and evaluation methods that measure LLM capabilities in both text and multimodal modalities. Apply post-training expertise (SFT, RLHF, reward modeling) to connect observed failures to the data and training interventions that address them. Publish research findings in top-tier AI conferences. Ideally you’d have: Ph.D. or Master's degree in Computer Science, Machine Learning, AI, or a related field. Deep understanding of deep learning, reinforcement learning, and large-scale model fine-tuning. Experience with post-training techniques such as RLHF, preference modeling, or instruction tuning, and with LLM evaluation or benchmark development. Excellent written and verbal communication skills. Published research in areas of machine learning at major conferences (NeurIPS, ICML, ICLR, ACL, EMNLP, CVPR, etc.) and/or journals. Previous experience in a customer facing r

At Scale, our mission is to develop reliable AI systems for the world's most important decisions. Our products provide the high-quality data and full-stack technologies that power the world's leading models, and help enterprises and governments build, deploy, and oversee AI applications that deliver real impact. Scale Frontier Data is the organization behind the training and evaluation data that frontier labs depend on. We build the systems, tooling, and expert workflows that turn hard human expertise into signals that models can learn from, across reasoning, coding, agentic tool use, and domain expertise. About our Customer Platform team: Our Customer Platform Team plays a pivotal role in integrating our platform with external systems and ensuring seamless, reliable connectivity for both internal users and customers. As the leader of this team, you’ll drive the strategy, architecture, and development of our connectivity solutions, focusing on API integration, distributed systems, and a robust data platform. Your role will be crucial in maintaining and enhancing our platform’s ability to meet the needs of both our internal and external stakeholders. Responsibilities: Own large areas within our product Comfortable working cross functionally, whether that be internal or external customers Build features end-to-end: front-end, back-end, system design, debugging and testing Deliver experiments at a high velocity and level of quality to engage our customers Work across the entire product lifecycle from conceptualization through production Influence the culture, values, and processes of a growing engineering team Inspire and mentor less experienced engineers Collaborating with cross-functional teams to define, design, and ship new product features and experiences. Requirements: At least 7-10 years of relevant experience is preferred Track record of shipping high-quality products and features at scale Desire to work in a very fast-paced environment Abil

About Snorkel At Snorkel, we believe meaningful AI doesn’t start with the model, it starts with the data. We’re on a mission to help enterprises transform expert knowledge into specialized AI at scale. The AI landscape has gone through incredible changes since 2015, when Snorkel started as a research project in the Stanford AI Lab, to the generative AI breakthroughs of today. But one thing has remained constant: the data you use to build AI is the key to achieving differentiation, high performance, and production-ready systems. We work with some of the world’s largest organizations to empower scientists, engineers, financial experts, product creators, journalists, and more to build custom AI with their data faster than ever before. Excited to help us redefine how AI is built? Apply to be the newest Snorkeler! In September 2026 we raised a $350 million Series E at a $3.5 billion valuation , and we are scaling our engineering and research teams to meet demand. The role Frontier AI data is expensive to make and hard to measure. Every task we deliver is tested against the strongest models, often through many long-running agent rollouts. Your job is to make that process faster, cheaper, and more rigorous with ML and AI You will be one of the early members of ML & Research Engineering at Snorkel. You will study how frontier-grade data is generated and evaluated, form hypotheses, validate them against real production data, and ship the winners at scale. You will shape the discipline's direction, its standards, and the team that grows around it. What you'll work on Efficient agentic evals. Cut the cost of long-horizon agent evaluation with adaptive sampling, statistically grounded early stopping, model cascades, caching, and cheap-first gating. AI model routing. Route every eval and judge call to the cheapest model that clears the quality bar, with fallback, monitoring, and cost attribution. Fine-tuned small models. Fine-tune and serve open-weight models (LoRA and other

About Scale Scale’s mission is to develop reliable AI systems for the world’s most important decisions. As the leading AI data foundry, we provide the high-quality data and full-stack technologies that power the world’s most advanced models — fueling breakthroughs in generative AI, defense, and autonomous vehicles. We partner with leading enterprises and governments to bring AI into production that performs when it matters most, combining rigorous evaluation with full-stack deployment so our customers can build AI they can trust. About the Team Applied Intelligence Systems (AIS) is part of the Scale Generative AI Platform (SGP), focused on pushing the frontier of what agentic applications can do across diverse enterprise and government use cases. We build the infrastructure and tooling that power agentic AI in production, paired with applied ML research, design, and evaluation to ensure these systems perform reliably at the scale our customers demand. AIS spans multiple workstreams — agent evaluation and oversight, orchestration and tool-use infrastructure, model and systems optimization, and applied research on new agent capabilities — and this role is not scoped to any single one of them. We’re growing fast, with increasing traction across both commercial and public sector customers, and we’re just getting started — this team will define what dependable, production-grade agentic AI looks like. About the Role As a Staff Machine Learning Research Engineer, you will operate across the full breadth of AIS’s technical needs — wherever the hardest ML problem in agentic AI happens to be that quarter. This could mean training and fine-tuning models, designing evaluation and observability systems, building improvement loops from production data, prototyping novel agent architectures, or designing internal systems and tooling that boost productivity across teams. You’re not tied to one team’s roadmap; you’re expected to move to where the technical leverage is highest, and t

About Scale AI At Scale, our mission is to develop reliable AI systems for the world's most important decisions. Our products provide the high-quality data and full-stack technologies that power the world's leading models, and help enterprises and governments build, deploy, and oversee AI applications that deliver real impact. Scale Frontier Data is the organization behind the training and evaluation data that frontier labs depend on. We build the systems, tooling, and expert workflows that turn hard human expertise into signals that models can learn from, across reasoning, coding, agentic tool use, and domain expertise. Reinforcement learning environments are now the center of gravity for that work: the difference between a model that demos well and a model that reliably completes long-horizon work is almost always the quality of the environments and reward signals it was trained against. Responsibilities As a Staff Software Engineer, RL Environments, you'll own the technical foundation for how Scale builds, runs, verifies, and delivers RL environments at scale. An RL environment is a real piece of software: a containerized world with real dependencies, real state, real tools, and a grader that has to be correct even when the agent is creative about breaking it. Building one is a full-stack engineering problem. Building thousands of them reproducibly, cheaply, with trustworthy reward signals and throughput measured in millions of rollouts is a systems problem that very few people have solved. You'll work on both. You'll design the platform: sandboxed execution, environment packaging and versioning, rollout orchestration, trajectory capture, verifier frameworks, and the authoring surfaces that let engineers and domain experts produce environments without reinventing infrastructure each time. And you'll go deep on the environments themselves by instrumenting real applications, designing task suites that expose specific capability gaps, and building graders that

Scale's mission is to develop reliable AI systems for the world's most important decisions. We provide the high-quality data that powers the world's AI models, and we help enterprises and governments build, deploy, and oversee AI applications that create real impact. As Communications Manager, Corporate and Product, you will help shape how Scale shows up when it comes to our work building enterprise AI, spanning company narrative, product launches, partner communications, media presence, and executive visibility. This role is centered on translating complex enterprise AI deployments and partnerships into clear, compelling narratives that resonate with business and vertical industry audiences. A core focus of the role is pitching and positioning Scale's enterprise wins, product launches, and customer partnerships, including how organizations across industries are using Scale's Generative AI Platform to build, deploy, and oversee AI applications that create real impact. You will also work closely with communications counterparts at partner companies to align messaging and coordinate joint announcements. You will help tell the story of how Scale's enterprise business powers mission-critical AI programs for the world's most consequential organizations, from Fortune 500 companies to leaders across every major industry. In this role, you will partner closely with enterprise product leaders and go-to-market teams, as well as teams across communications, social, and marketing. You will play a key role in strengthening Scale's reputation with enterprise customers, industry analysts, and the broader technology ecosystem. This position reports directly to the Head of Corporate & Product Communications. You Will Support corporate and product communications initiatives with a strong emphasis on Scale's enterprise business, including customer and partnership announcements, product launches, executive visibility, and industry positioning. Help translate co

As a member of our Frontier Tech Consultant team, you will play a critical role in advancing cutting-edge AI innovations by conducting high-impact experiments and ensuring seamless execution at the highest quality standards. Your work will directly contribute to Scale AI’s growth, shaping the future of artificial intelligence. In this role, you will be working on various types of projects, including but not limited to: research experiments, dataset generation, data quality improvements, and in-depth technical analysis. You will tackle complex, technical and operational challenges while collaborating closely with Scale’s ML research scientists and SPM team. The ideal candidate is analytical, detail-oriented, and results-driven, with strong problem-solving abilities and excellent communication skills. We are looking for someone who thrives in a fast-paced environment, is proactive in overcoming challenges, and is committed to delivering exceptional outcomes. If you are eager to contribute to the forefront of AI innovation, we encourage you to apply. You will be responsible for: Design and execute research experiments Build and evaluate frontier LLM datasets Develop training and testing material for frontier pipelines Improve quality of existing and new products Ideally you’d have: Strong machine learning knowledge, either by being in the final years of a ML PhD career or having already graduated Strong writing and verbal communication skills An action-oriented mindset that balances creative problem solving with the scrappiness to ultimately deliver results Analytical, planning, and process improvement capability Experience working in a fast-paced, entrepreneurial environment Technical skills including familiarity with Python, GPU, AWS, API, LLM, ML, and SQL Pay: $60-80/hr Commitment: This is a fully remote, US-based part-time (10-20 hours per week), on-going contract position staffed via HireArt. HireArt values diversity and is an Equal Opportunity E

At Scale, our mission is to develop reliable AI systems for the world's most important decisions. For 10 years, Scale has provided the high-quality data and full-stack technologies that power the world's leading models, and has helped enterprises and governments build, deploy, and oversee AI applications that deliver real impact. We work closely with industry leaders like Meta, Ernst & Young, Mayo Clinic, Time Inc., the Government of Qatar, and U.S. government agencies including the Army and Air Force. Scale's internship is not a side project. Interns own real, shipped work on the same roadmaps as full-time engineers, with mentorship from world-class talent and a culture that values ownership, speed, and truth-seeking. Many of our interns return as full-time Scaliens. Example Projects Build reinforcement learning and post-training data pipelines that power frontier model development Develop evaluation infrastructure that measures model reliability for enterprise and public sector customers Ship agentic AI applications and the tooling that makes them observable, testable, and safe to deploy Ship tools that accelerate the growth of new qualified contributors on Scale's platform Build fraud-detection systems that remove bad actors and keep Scale's contributor base safe and trusted Use models to estimate the quality of tasks and contributors, and guarantee quality on requests at large scale Devise advanced matching algorithms that pair contributors to customers for optimal turnaround and accuracy Create optimized and efficient UI/UX tooling, in combination with ML algorithms, for 100k+ contributors completing billions of complex tasks Develop new AI infrastructure products to visualize, query, and explore Scale data Requirements A graduation date in Fall 2027 or Spring 2028 with a Bachelor's degree (or equivalent) in a relevant field (Computer Science, EECS, Computer Engineering, Statistics) Available for a Summer 2027 internship (May/June start dates) in San Franci

About the Team DoorDash’s GenAI Platform team sits within Machine Learning Platform and builds the shared infrastructure that helps DoorDash, Wolt, and Deliveroo teams safely bring GenAI-powered products, agents, automation, and personalization to production. Our mission is to increase the velocity of business impact from GenAI. A central pillar of that work is our evaluation platform — the unified evals backbone that lets teams measure, trace, and trust the quality of LLM and agent systems across the company, powering trace/score ingestion, LLM-as-judge workflows, agent simulations, and LLM observability for the tens of millions of daily requests flowing through our LLM Gateway. We also own core platform surfaces including the Agent Gateway, open-weights model serving and batch inference, guardrails, and cost attribution. About the Role You will join a small, high-leverage team building production infrastructure for Generative AI at DoorDash, with a primary focus on our evals and LLM observability platform: the systems that let teams evaluate, trace, and continuously improve the quality of LLM and agent products. You’ll work across evaluation frameworks and SDKs, OpenTelemetry-based trace/score ingestion, LLM-as-judge and offline/online eval pipelines, agent simulations, data pipelines, backend services, and observability. This role is ideal for an engineer who enjoys building reliable measurement and quality primitives in a fast-moving technical area where product needs, model capabilities, vendor ecosystems, and evaluation methodologies are evolving quickly. You’re excited about this opportunity because you will… Build the infrastructure that helps DoorDash teams move GenAI ideas from prototype to production, increasing the velocity of business impact from AI across the company. Work on our unified evals platform — evaluation SDKs, OpenTelemetry trace/score ingestion, LLM-as-judge, offline and online eval pipelines, and agent simulations — alongside the LLM Gatew

About the Team DashMart is a local-fulfillment center owned and operated by DoorDash, offering customers household essentials and other items to their doorsteps with speed, reliability, and quality. Customers order their convenience, grocery, retail, and prepared foods in the DoorDash app, and our team members fulfill orders in a real, brick-and-mortar store for Dashers to deliver. We’re open early and close late - some sites even run 24/7! About the Role DashMart is looking for a motivated and experienced individual that excels in fast-paced, physical environments and is excited to roll up their sleeves and actively engage in day-to-day operations. In this role, you will work within a local fulfillment center supporting Site Management running great shifts, and delegating tasks. As a Shift Lead, you will have shift responsibility for fulfilling orders in a warehouse environment and maintaining inventory, and in some locations, this involves preparing food in a light-prep kitchen. You’re excited about this opportunity because you will… Be an Owner: Take ownership of your assigned shifts, including warehouse and kitchen processes, safety/cleanliness, quality, and training. Maintain accountability for inventory, equipment, and other company assets to ensure they are properly handled, stored, and protected from loss or theft. Delight Customers: Ensure customer orders are delivered with high quality by executing orders accurately, communicating with customers when issues arise, and making sure Dasher pickups go smoothly. Lead: Guide Operations Associates through their shift by ensuring the team works safely and productively and serving as the point of escalation for daily operations. One Team One Fight: Support operations in both the warehouse and kitchen, assist with day-to-day tasks, and lead by example. You will be expected to engage in professional and respectful interactions with team members and customers, ensuring a positive and safe

About the Team Our team exists to empower DoorDash Mobile engineers. We are the champions of three core pillars: Quality, Velocity, and Efficiency. Ultimately, our goal is to build the supportive infrastructure that allows our fellow engineers to build, ship, and operate apps at massive scale—with confidence and ease. We believe that a great developer experience leads to a great customer experience. About the Role We are looking for an Engineering Manager to lead our Mobile Developer Experience team (also known internally as the Mobile Foundations team). In this role, you’ll not only set the technical vision but also nurture the culture required to build world-class mobile applications. Your team will concentrate on one vertical—ZeroKit, which enables rapid mobile prototyping via agents—and two horizontal infrastructure pillars: the iOS and Android monorepos. You will partner with senior engineers and stakeholders across the company to design systems that make our platform faster, more reliable, and more efficient. You will be a partner to your customers—your fellow engineers—working side-by-side to understand their hurdles and solve their immediate challenges. You’ll also look to the future, anticipating needs so we can deliver solutions before they become blockers. Crucially, this role will support our growing international engineering footprint and unified technology stack, meaning you and your team will work across our DoorDash, Wolt, and Deliveroo brands. This is a unique opportunity to lead a team through a mix of exciting greenfield initiatives and the refinement of established, successful tools. You must be located in the following locations for this opportunity: San Francisco, CA; Sunnyvale, CA; Seattle, WA; Los Angeles, CA; New York, New York. You’re excited about this opportunity because you will… Develop and maintain foundational components to enable DoorDash Engineers to excel at mobile engineering. Lead the development and strategy for ZeroKit to enabl

About the Team Merchant Analytics helps DoorDash make better product, business, and go-to-market decisions through high-quality analytics, predictive modeling, experimentation, and strategic thought partnership. We work across some of DoorDash’s most important merchant and marketplace priorities, building the measurement, insights, and decision frameworks that improve outcomes for merchants and drive company impact. About the Role We’re hiring two Data Science Managers, each to lead a pod within Merchant Analytics and help shape high-priority product and business decisions. In this role, you will lead a team of data scientists, partner closely with Strategy & Operations, Product, Engineering, and business leaders, and turn ambiguous questions into clear recommendations that influence roadmap and plan outcomes. Success in this role means building a high-performing team, raising the quality and speed of decision-making, and ensuring analytics work is tightly connected to measurable business impact. You will report into Director, Data Science on our Merchant Analytics team in our Analytics organization. You’re excited about this opportunity because you will… Lead and develop a team of data scientists responsible for high-impact analytics, predictive modeling, and decision support tied to DoorDash’s most important product, business, and GTM priorities. Partner closely with Strategy & Operations, Product, Engineering, and business leaders to shape decisions, influence roadmaps, and improve plan-critical metrics. Define success metrics, build measurement frameworks and predictive models, and use experimentation and analysis to connect product and operational levers to business outcomes. Build a high-performing pod that balances analytical rigor, strong prioritization, and clear storytelling in a fast-moving environment. Scale reusable analytics frameworks, tools, models, and best practices that make the broader organization more effective over time. We’re excited

About the Team As DoorDash continues to expand rapidly, our Core Consumer team plays a pivotal role in shaping the consumer experience through enhancing personalization, search relevance, merchandising strategy, app quality, and the overall ordering experience. Our mission is to implement scalable data solutions and provide insights that directly influence our strategic product direction. We are looking for an Analytics Senior Manager to lead our Discovery team. About the Role As a Senior Manager of Data Science/Analytics, you’ll be leading a team of data scientists who are working on improving the consumer experience. You will develop strategic insights and work closely with the product, engineering, strategy and operations teams to actively build and measure the impact of new features. You will oversee our metrics and analytics strategy to inform strategic product direction, offer technical leadership and build processes to support velocity, and partner with data and product engineers to build robust data foundations. You're excited about this opportunity because you will… Lead and develop a team of Data Scientists in investigating complex issues and uncovering key drivers of our business, along with your own contributions as an Individual Contributor Influence the Product and Operations roadmap by making actionable recommendations based on data Interface frequently with senior leadership to showcase your team’s work and tackle complex business problems Drive measurement strategy for the area under scope, defining success metrics and implementing best practices around experiment design and statistical analysis Develop a strategic learning roadmap based on data observations, strategic questions, and hypotheses We're excited about you because you have… A degree in Math, Physics, Statistics, Economics, Computer Science, or a similar domain Experience managing a team of data scientists, and a track record of delivering impactful analyses 8+ years of experience i

Airbnb was born in 2007 when two hosts welcomed three guests to their San Francisco home, and has since grown to over 5 million hosts who have welcomed over 2 billion guest arrivals in almost every country across the globe. Every day, hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way. The Community You Will Join: Providing world-class customer service for our community of guests and hosts is a critical mission at Airbnb. The Ambassador Routing & Channels (ARC) team sits within Airbnb Community Support and is responsible for powering intelligent, differentiated service experiences, ensuring every support request, across any channel, is addressed by the right expert at the right time. Our work directly drives faster resolutions, higher quality interactions, and exceptional customer satisfaction at scale. The Difference You Will Make: As a Senior Backend Engineer on the ARC team, you will contribute meaningfully across our core routing and differentiated service programs: Contribute to and lead engineering delivery on programs that enable intelligent agent matching, personalized support experiences for high-value customers, and next-generation routing and communication channel capabilities. Own and evolve core routing infrastructure, including decision engines, workflow orchestration, and routing rule management. Ensuring high availability, correctness, and scalability across millions of daily support interactions. Help drive consolidation of fragmented routing logic spread across multiple platforms and channels into a centralized, auditable, and maintainable system. Collaborate cross-functionally with Product, Operations, Data Science, and partner engineering teams to align on priorities and deliver outcomes that improve the customer support experience. Contribute to technical quality through design reviews, code reviews, and pragmatic architectural decision-making. Support the team's tech