Human Evaluator Jobs

Software is eating the world, but AI is eating software. We live in unprecedented times – AI has the potential to exponentially augment human intelligence. Every person will have a personal tutor, coach, assistant, personal shopper, travel guide, and therapist throughout life. As the world adjusts to this new reality, leading platform companies are scrambling to build LLMs at billion scale, while large enterprises figure out how to add it to their products. To make them safe, aligned and actually useful, these models need human eval and reinforcement learning through human feedback (RLHF) during pre-training, fine-tuning, and production evaluations. This is the main innovation that’s enabled ChatGPT to get such a large headstart among competition. At Scale, our products include the Generative AI Data Engine, SGP, Donovan, and others that power the most advanced LLMs and generative models in the world through world-class RLHF, human data generation, model evaluation, safety, and alignment. The data we are producing is some of the most important work for how humanity will interact with AI. At the foundation of these products is the Identity Engineering team. In this role, you will help support the design and development of core software systems specifically focused on identity, access management, authorization, and authentication. You’ll also get widespread exposure to the forefront of the AI race as Scale sees it in enterprises, startups, governments, and large tech companies. You will: Drive the design, and implementation of our identity infrastructure to ensure secure authentication and authorization across enterprise systems. Build software for authentication mechanisms such as Single Sign-On (SSO), Multi-Factor Authentication (MFA), and federated identity solutions (SAML, OAuth, OpenID Connect). Build software for authorization mechanisms such as Relation-based access control (ReBAC), Attribute-based access control (ABAC), Role-based access cont

Software is eating the world, but AI is eating software. We live in unprecedented times – AI has the potential to exponentially augment human intelligence. Every person will have a personal tutor, coach, assistant, personal shopper, travel guide, and therapist throughout life. As the world adjusts to this new reality, leading platform companies are scrambling to build LLMs at billion scale, while large enterprises figure out how to add it to their products. To make them safe, aligned and actually useful, these models need human eval and reinforcement learning through human feedback (RLHF) during pre-training, fine-tuning, and production evaluations. This is the main innovation that’s enabled ChatGPT to get such a large headstart among competition. At Scale, our products include the Generative AI Data Engine, SGP, Donovan, and others that power the most advanced LLMs and generative models in the world through world-class RLHF, human data generation, model evaluation, safety, and alignment. The data we are producing is some of the most important work for how humanity will interact with AI. At the foundation of these products is the Identity Engineering team. In this role, you will help support the design and development of core software systems specifically focused on identity, access management, authorization, and authentication. You’ll also get widespread exposure to the forefront of the AI race as Scale sees it in enterprises, startups, governments, and large tech companies. You will: Drive the design, and implementation of our identity infrastructure to ensure secure authentication and authorization across enterprise systems. Build software for authentication mechanisms such as Single Sign-On (SSO), Multi-Factor Authentication (MFA), and federated identity solutions (SAML, OAuth, OpenID Connect). Build software for authorization mechanisms such as Relation-based access control (ReBAC), Attribute-based access control (ABAC), Role-based access cont

Software is eating the world, but AI is eating software. We live in unprecedented times – AI has the potential to exponentially augment human intelligence. Every person will have a personal tutor, coach, assistant, personal shopper, travel guide, and therapist throughout life. As the world adjusts to this new reality, leading platform companies are scrambling to build LLMs at billion scale, while large enterprises figure out how to add it to their products. To make them safe, aligned and actually useful, these models need human eval and reinforcement learning through human feedback (RLHF) during pre-training, fine-tuning, and production evaluations. This is the main innovation that’s enabled ChatGPT to get such a large headstart among competition. At Scale, our products include the Generative AI Data Engine, SGP, Donovan, and others that power the most advanced LLMs and generative models in the world through world-class RLHF, human data generation, model evaluation, safety, and alignment. The data we are producing is some of the most important work for how humanity will interact with AI. At the foundation of these products is the Platform Engineering team. In this role, you will support the design and development of shared platforms used across Scale. This includes designing our foundational data platforms and lifecycle, architecting Scale’s core cloud infrastructure and orchestration stack, and redefining how engineers develop, build, test, and deploy software at Scale. You’ll also get widespread exposure to the forefront of the AI race as Scale sees it in enterprises, startups, governments, and large tech companies. You will: Drive the design, and implementation of our foundational platforms and systems, working closely with stakeholders and internal customers to understand and refine requirements. Collaborating with cross-functional teams to define, design, and deliver new features. Proactively identifying opportunities for, and driving improvements to, current p

About the Team Safety Systems manages the complete lifecycle of safety efforts for OpenAI’s frontier models, ensuring our models are deployed responsibly and have a positive impact on society. Our work spans diverse research and engineering initiatives—from system-level safeguards and model training to evaluation and red-teaming—all aimed at mitigating misuse, misalignment, and maintaining our high bar for safety. We lead OpenAI's commitment to developing and deploying safe Artificial General Intelligence (AGI), fostering a culture of trust, responsibility, and transparency. Our goal is to continuously learn from deployments, distribute AI’s benefits widely, and ensure that powerful tools remain aligned with human values and safety considerations. About the Role We are hiring a Product Manager to focus on risk related to multimodal models. In this role, you will drive initiatives which ensure that OpenAI’s audio, image, and video deployments are safe, impactful, and aligned with user needs and technical innovation. You will clarify strategic priorities, develop safety-focused product roadmaps, and collaborate closely with AI researchers, software engineers, policy experts, and cross-functional partners. This role suits a proactive, technically skilled product manager adept at adversarial thinking and excited to tackle challenging, ambiguous problems through structured analysis and collaborative decision-making. This position is based in San Francisco, CA, with relocation assistance available. In this role, you will: Partner closely with AI research, engineering, data science, policy teams, and other stakeholders to embed safety throughout the development and deployment of multimodal AI models - such as GPT-Live and ChatGPT Images - as well as multimodal capabilities in frontier AI models. Develop comprehensive frameworks for understanding and mitigating deployment safety risks, drawing on data analysis, expert consultation, and adversarial assessments. Define strate

Become a part of our caring community You have shipped AI products before. You understand the difference between a demo and a production system. You have strong opinions about evaluation frameworks because you have experienced the consequences of operating without them. You are at your best when you own architecture decisions while continuing to build and deliver critical code yourself. We build the platform that transforms millions of clinical documents into trusted, actionable data. Our systems use large language models (LLMs) to read medical records, extract structured facts, answer complex questions with citations back to source documents, and route complex cases to human experts. The output of these systems supports healthcare decisions that impact real members. As a Lead AI Applied Engineer, you will provide technical leadership for AI-enabled products and platforms, define architectural direction, establish engineering standards, and personally design and build the most critical components of our systems. You will lead through both technical expertise and execution, helping the team deliver reliable, scalable, and auditable AI solutions in a highly regulated healthcare environment. Why Join Us Lead the architecture of production AI systems where LLMs are foundational to the product experience. Make key technical decisions regarding model selection, system boundaries, platform architecture, and build-versus-buy strategies. Own the highest-risk and highest-impact technical challenges involving reliability, explainability, and correctness. Influence engineering culture and establish standards that shape how the team builds and ships AI products. Work on systems operating at meaningful scale, processing millions of documents and supporting healthcare decisions across a large member population. Partner

About the Team The Health team, within OpenAI’s broader Personal AGI organization, has a mission to ensure AGI improves health for all humanity. Improving human health will be one of the defining impacts of AGI. Hundreds of millions of people already turn to ChatGPT for questions about their health and millions of clinicians use it weekly to support care delivery. Increasingly capable models create an opportunity to make high-quality medical intelligence more accessible across patients and clinicians—raising the floor of human health—and accelerate the new capabilities and scientific advances that raise the ceiling of human health. Our job is to make those benefits real. We work across the full model stack—pretraining, midtraining, reinforcement learning, post-training, evaluations, harnessing, and deployment—and connect that research to the patients, clinicians, and real-world outcomes we aim to improve. About the Role We’re looking for an exceptional, hands-on researcher who wants to build frontier health capabilities and turn them into impact at scale. This is a role for someone who can take an important, underdefined problem from 0→1: identify the right bet, build what’s needed to test it, and drive it all the way to a measurable improvement in the models and products we actually ship. We’re especially excited about two kinds of people: researchers with the technical depth to move the frontier in pretraining, reinforcement learning (RL) / post-training, or evals; and researchers with real depth in developing frontier biomedical AI capabilities. Prior experience in healthcare is helpful but not required. Research excellence, velocity, ownership, and alignment with the mission are most important to us. This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees. In this role, you will: Own a high-leverage research direction end to end—from deciding which problem matters and h

Position: Assistant Professor Master of Business Administration (MBA) Department: Management Studies Location: College Campus Key Responsibilities Deliver lectures and practical sessions in MBA subjects such as Marketing, Finance, Human Resource Management, Operations, Business Analytics, and Entrepreneurship. Prepare lesson plans, teaching materials, assignments, case studies, and assessments. Mentor students in academic projects, internships, seminars, and research activities. Evaluate student performance through assignments, presentations, internal assessments, and examinations. Organize workshops, guest lectures, industry visits, and management development programs. Participate in curriculum development, accreditation activities, and departmental administration. Encourage research, innovation, entrepreneurship, and industry collaboration. Perform academic and administrative responsibilities assigned by the Head of Department. Eligibility Qualification: MBA/PGDM with First Class in the relevant specialization from a recognized university or institution. Ph.D. is preferred as per applicable AICTE/UGC norms.

For over 20 years, Smartsheet has empowered teams to manage work seamlessly and scale solutions smarter. Now, in our most ambitious chapter yet, we are uniting human teams with AI agents. By orchestrating the work agents do best, automating manual tasks and uncovering insights at scale, we create the space for people to focus on what truly matters: judgment, creativity, and big thinking. That is magic at work, and it’s what we show up for every day. We are looking for a VP of Product Management to lead AI product strategy at Smartsheet. You will lead a product team responsible for defining our comprehensive AI vision and roadmap, ensuring we build a cohesive, coordinated approach to how AI transforms our product. Your team will establish the strategic direction for specialized AI agents, evaluation infrastructure, and AI-powered capabilities, from platform foundations through to customer-facing features. You will report to our Chief Product and Technology Officer located in our Bellevue, WA office, or you may work remotely from anywhere in the US where Smartsheet is a registered employer. You Will: Lead a high-performing product team focused on defining and executing Smartsheet's comprehensive Applied AI product strategy. Define the strategic roadmap for AI product initiatives — from conversational experiences, MCP, automations, specialized agents, and evaluation infrastructure through to AI-powered features in customer-facing products — ensuring a cohesive, coordinated vision. Own the business outcomes of the AI portfolio — define the success metrics (adoption, retention, expansion, and revenue impact), set the targets, and hold the team accountable to them. Instrument, measure, and iterate: build the data and evaluation foundation that tells us whether AI features are actually working, and use it to make hard prioritization calls — including what to sunset. Move fast in a fast-moving space — ship, learn, and adjust in short cycles rather than waiting for per

About the Team The Safety Systems team is at the forefront of OpenAI's mission to build and deploy safe AGI, driving our commitment to AI safety and fostering a culture of trust and transparency. The Model Policy team aligns model behavior with desired human values and norms. We co-design policy with models and for models by driving rapid policy taxonomy iteration based on data and defining evaluation criteria for foundational models’ ability to reason about safety. Key focus areas include: catastrophic risk, mental health, teen safety and multimodal safety. About the Role Providing access to frontier AI systems raises complex questions around dual-use science and catastrophic risk. How should models respond to requests involving chemical synthesis, biological experimentation, or pathogen research? Where is the boundary between legitimate scientific inquiry and information that could enable misuse? How do we design policies that meaningfully reduce risk without unnecessarily restricting beneficial research? This is a senior role in which you’ll help shape policy creation and development at OpenAI for addressing biological and chemical risks. You will develop structured policy frameworks and taxonomies to guide safe model behavior. This role sits at the intersection of biosecurity expertise, AI safety research, and policy design. You will help ensure that frontier AI systems can support beneficial life sciences research, such as drug discovery, public health, and biosafety, while reducing the risk that these capabilities could be misused. This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees. In this role, you’ll: Design and maintain model policies governing chemical and biological risk, defining how models should safely handle dual-use scenarios. Develop structured taxonomies of chemical and biological risk that inform model training data, evaluation benchmarks, and safet

For over 20 years, Smartsheet has empowered teams to manage work seamlessly and scale solutions smarter. Now, in our most ambitious chapter yet, we are uniting human teams with AI agents. By orchestrating the work agents do best, automating manual tasks and uncovering insights at scale, we create the space for people to focus on what truly matters: judgment, creativity, and big thinking. That is magic at work, and it’s what we show up for every day. We are looking for a Senior Product Manager to join our Applied AI organization at Smartsheet. You'll own the roadmap for specialized AI agents and evaluation infrastructure, ship complex agent capabilities end-to-end, and define the standards that enable teams across the company to build agents successfully. You'll report to our Director of Product Management for Applied AI located in our Bellevue, WA office, or you may work remotely from anywhere in the US where Smartsheet is a registered employer. You Will: Own the strategy, roadmap, and execution for specialized AI agents and evaluation infrastructure Ship large, technically complex agent capabilities end-to-end with with a high degree of autonomy Define agentic development standards that enable teams beyond your own to build high-quality agents Mentor and develop junior PMs on your team Build strong cross-functional partnerships with engineering, design, and data science to drive alignment and move work forward Support other duties as assigned You Have: 8-10+ years of product management experience with technically complex platforms Fluency in AI/ML concepts -model evaluation, agent architectures, RAG, prompt engineering, quality measurement Track record of shipping enterprise software from inception to launch Experience building or managing evaluation and quality infrastructure for AI or ML systems (strongly preferred) Comfort operating in high-ambiguity environments with significant autonomy Exceptional communication skills and the ability to influence without auth

For over 20 years, Smartsheet has empowered teams to manage work seamlessly and scale solutions smarter. Now, in our most ambitious chapter yet, we are uniting human teams with AI agents. By orchestrating the work agents do best, automating manual tasks and uncovering insights at scale, we create the space for people to focus on what truly matters: judgment, creativity, and big thinking. That is magic at work, and it’s what we show up for every day. Smartsheet is looking for a Senior Solutions Engineer who has a passion for solving complex challenges. In this role, you will partner directly with Sales to provide technical leadership throughout the sales process, navigating intricate deal cycles, providing value-based demonstrations, and serving as a trusted advisor to prospects and clients throughout their evaluations. You will be a force multiplier within the Solutions Engineering organization that is on a mission to grow revenue and to empower anyone to drive meaningful change — for themselves, their businesses, and the world. This role is part of the Worldwide Field Operations Commercial Solutions Engineering team and can be based remotely from anywhere in the US where Smartsheet is a registered employer. You Will: Identify, build, and demonstrate innovative solutions to solve complex business challenges for high visibility Smartsheet customers Lead technical sales engagements, collaborate closely with sales teams to identify customer requirements, summarize challenges, design solutions, and provide technical guidance that demonstrates the value of Smartsheet's offerings Conduct discovery sessions to uncover business processes and challenges to inform the sales and solution delivery strategy Establish yourself as a strategic partner to Sales, Product, Customer Success, and Professional Services to provide a great overall customer experience and drive revenue growth Proactively grow and maintain deep knowledge for all Smartsheet products, services, and compe

For over 20 years, Smartsheet has empowered teams to manage work seamlessly and scale solutions smarter. Now, in our most ambitious chapter yet, we are uniting human teams with AI agents. By orchestrating the work agents do best, automating manual tasks and uncovering insights at scale, we create the space for people to focus on what truly matters: judgment, creativity, and big thinking. That is magic at work, and it’s what we show up for every day. The Opportunity Asia-Pacific is one of the fastest-moving enterprise AI markets in the world — and the buyers, competitive dynamics, and adoption patterns in the region are distinct from any other geography Smartsheet serves. Enterprises in APJ are evaluating AI capabilities with urgency, but their buying criteria, integration ecosystems, and trust frameworks differ meaningfully from those in North America or Europe. Smartsheet needs an AI product marketing leader embedded in the region who can translate our global AI narrative into locally resonant positioning that wins. As a Sr Product Marketing Manager on the AI GTM Strategy team, you will be the in-region AI product marketing partner for Smartsheet's APJ sales organization. You will own the regional activation of Smartsheet's AI narrative — including organizational intelligence, agentic workflows, and Model Context Protocol (MCP) positioning — ensuring APJ field teams can lead differentiated AI conversations with enterprise buyers across the region. This is not a generalist PMM role. You will own the intersection of AI product marketing and regional field activation across APJ, working at the pace of a rapidly evolving product surface and a field organization operating in one of the most competitive enterprise software markets globally. This remote eligible role is based in APJ and will report to the VP, AI GTM Strategy. What You Will Own AI Narrative Activation — APJ Execute the regional activation of Smartsheet's global AI narrative across APJ field teams, a

For over 20 years, Smartsheet has empowered teams to manage work seamlessly and scale solutions smarter. Now, in our most ambitious chapter yet, we are uniting human teams with AI agents. By orchestrating the work agents do best, automating manual tasks and uncovering insights at scale, we create the space for people to focus on what truly matters: judgment, creativity, and big thinking. That is magic at work, and it’s what we show up for every day. Smartsheet is looking for a Sr. Principal Data Scientist to set the technical direction for the ML models and AI sub-agents that drive growth, monetization, efficiency, and retention across the customer lifecycle. You’ll architect the systems, define the modeling and evaluation standards, and shape the roadmap for how Smartsheet uses applied ML and agentic AI to serve millions of customers. The data is unusually rich: petabyte-scale execution data spanning two decades of how real work gets done. You are a recognized technical leader who moves fluidly between architecture, prototype, and production; raises the bar for the data scientists around you; and partners with senior product and engineering leaders to make decisions that compound across the org. You will work primarily with Product and Engineering and will be a part of Smartsheet’s Business Intelligence team. This full-time position reports to the Director of Data Science and is based in Smartsheet’s Bengaluru, India office. You Will: Set the technical direction for how Smartsheet uses applied ML and agentic AI across the customer lifecycle — what gets built, what doesn’t, and what good looks like Architect the systems behind the sub-agents: how they ground themselves in evidence, how risk and confidence are calibrated, how decisions are evaluated, and how all of it stays safe and trustworthy at scale Define the modeling, evaluation, and experimentation standards the team follows — from offline metrics to online rollout to production monitoring Personally ship the

Ready to do the most impactful work of your career? At Coinbase , we are uncompromising on our mission to increase economic freedom. The bar is high, the environment is intense, and we like it that way. This isn't a place for complacency, it’s a place to be pushed past your perceived limits. If you're ready to build the future of finance alongside people who refuse to settle for "good enough," you belong here. Coinbase is a remote-first, but not remote-only company. Expect to get together quarterly for intense in-person working sessions called “surges.” learn more about working at Coinbase . We're hiring a Group Product Manager to lead the Compliance Agent Experience product team within our EAA organization. This team builds the internal tooling and AI-powered investigation experiences used by Compliance Operations to execute Coinbase's financial crime programs, including AML Transaction Monitoring, KYC, Enhanced Due Diligence, and Sanctions/Screening. You'll own the automation-first roadmap that enables investigators to triage alerts, assess risk, and document decisions faster and more accurately, so humans focus on high-judgment work while the platform handles repeatable tasks at scale. What you'll do: Own the Compliance Agent Experience product area end-to-end, from vision and strategy through execution, covering AML/TMS, KYC, EDD, and Sanctions/Screening investigation surfaces. Drive a multi-year roadmap for AI-assisted, human-in-the-loop investigation workflows that measurably reduce cycle time, improve decision quality, and increase automation adoption. Lead build-vs-buy and vendor strategy for investigation enablement (OSINT, case enrichment, LLM tooling), including evaluations and RFPs in partnership with Compliance, Legal, Security, and Procurement. Partner with Engineering, Data Science/ML, Compliance Program owners, and CX Operations to deliver integrated tooling that reduces context-switching and improves end-to-end investigation throughput.

About the Team Our Safety Systems team is at the forefront of OpenAI's mission to build and deploy safe AGI, driving our commitment to AI safety and fostering a culture of trust and transparency. The Model Policy team aligns model behavior with desired human values and norms. We co-design policy with models and for models by driving rapid policy taxonomy iteration based on data and defining evaluation criteria for foundational models’ ability to reason about safety. Key focus areas include: catastrophic risk, mental health, teen safety and multimodal safety. About the Role Providing access to frontier AI systems raises complex questions around dual-use science and catastrophic risk. How should models respond to requests involving chemical synthesis, biological experimentation, or pathogen research? Where is the boundary between legitimate scientific inquiry and information that could enable misuse? How do we design policies that meaningfully reduce risk without unnecessarily restricting beneficial research? This is a senior role in which you’ll help shape policy creation and development at OpenAI for addressing biological and chemical risks. You will develop structured policy frameworks and taxonomies to guide safe model behavior. This role sits at the intersection of biosecurity expertise, AI safety research, and policy design. You will help ensure that frontier AI systems can support beneficial life sciences research, such as drug discovery, public health, and biosafety, while reducing the risk that these capabilities could be misused. Our relevant publications: Preparedness framework Preparing for future AI capabilities in biology Safety evaluations hub OpenAI GPT5 System Card Evaluating Fairness in ChatGPT Improving Model Safety Behavior with Rule-Based Rewards OpenAI Model Spec Your Responsibilities: Design and maintain model policies governing chemical and biological risk, defining how models should safely handle dual-use scenarios. Develop structured taxonomi