Human Data Quality Analyst in San Francisco

About the Team OpenAI's Human Data Team creates custom data solutions driving groundbreaking research. Our work enhances and evaluates our flagship models and products like ChatGPT, GPT-5, and Sora, and contributes to safety initiatives through collaboration with our Preparedness and Safety Systems teams. About the Role As a Research Program Manager (RPM) in the Human Data team you will partner with research and engineering to design and implement pragmatic solutions for collecting high-quality data. You will be a key interface between our research roadmap, external vendors, AI trainers, and the Human Data engineering team. This role is based in our San Francisco HQ. In this role, you will: Collaborate with Research: Partner with researchers to scope data collection needs, define success metrics, and establish quality measurement frameworks. Design & Execute Data Collection Campaigns: Translate research needs into actionable plans and accelerate execution by leveraging existing tooling and iterating to reach the desired outcome. In many cases, you will need to implement scrappy new solutions while partnering with engineering to design robust/scalable solutions. Unblock Yourself: You must be deeply uncomfortable with the idea of sitting around waiting for external dependencies, and have the technical acumen and drive to figure out how to achieve at least partial success in the interim. Optimize Systems & Processes: Build and optimize dashboards to track campaign performance, leveraging SQL and Python for data analysis and actionable insights. Drive Technical Roadmaps: Collaborate with engineers to enhance data platforms, resolve blockers, and ensure security best practices such as access management. Scale Your Impact : Advise and empower program managers and vendors to drive day-to-day execution so that you can focus on addressing high priority opportunities. You might thrive in this role if you: Are proficient in SQL and Python for data analysis, including q

About the Team OpenAI’s People team hires, engages, and retains world-class talent to safely build and deploy AGI that benefits all of humanity. The People Analytics team helps leaders make rigorous, evidence-based talent decisions and ensures that the systems supporting those decisions are valid, reliable, fair, and accountable. About the Role As a People Data Scientist focused on AI fairness and bias testing, you will help establish how OpenAI evaluates AI-assisted People systems and high-impact talent processes. You will design and conduct rigorous assessments to identify, measure, and mitigate potential bias across the lifecycle of models, agents, decision-support tools, and automated workflows. Your work will span the entire employee life-cycle, such as hiring, performance, promotion, employee development, workforce planning, etc. You will evaluate both technical systems and the broader human-AI decision processes in which they operate, examining not only model performance but also data quality, measurement validity, differential outcomes, human oversight, and unintended consequences. We’re looking for an experienced data scientist or applied researcher who can translate complex fairness questions into defensible evaluation strategies, scalable testing infrastructure, and clear recommendations for technical teams and senior leaders. This role is preferred to be based in San Francisco, CA. In this role, you will: Define and lead fairness and bias-testing strategies for AI-assisted People processes, models, agents, and decision-support systems from development through deployment and ongoing monitoring. Design rigorous algorithmic audits and validation studies, including adverse-impact analysis, subgroup and intersectional evaluation, error-rate analysis, calibration, measurement invariance, reliability, criterion-related validity, and sensitivity testing. Identify the appropriate fairness criteria for each use case, evaluate tradeoffs among competing definitions

Who We Are Notion is the collaborative AI workspace where teams and agents think together . We're building one place where your knowledge, projects, meetings, and AI tools live side by side, so work is faster, clearer, and less fragmented. Millions of individuals, small teams, and large companies run their work on Notion. Notinos (our employees) are customer zero in bringing this future of work to life. We care about craft, building things that last, and the belief that great work is still fundamentally human. Our goal isn’t to ship the next feature. Each and every team of Notinos is working to set the standard for how humans work together in the AI era. From building a business’s system of record to making and managing AI agents to automating away the busy work, we care deeply about giving our customers more time for their life’s work. About the Role: You'll be Recruiting's dedicated people analytics partner — owning the data, shaping the strategy, and in the room where decisions get made. The core of the role is building trusted recruiting analytics infrastructure (funnel + stage conversion, pipeline aging, time-to-fill, offer acceptance, sourcing channel quality, and recruiter capacity) and making it self-serve for recruiters and leaders. You’ll own the stack end-to-end — from raw data in Ashby and Snowflake to clear insights that hold up in a pipeline review. You're the person who turns workforce signals into recruiting strategy — bringing a point of view. This role may be based in either our San Francisco or New York City offices. We work from our offices on Mondays, Tuesdays and Thursdays (our Anchor Days) because we do our best thinking and building together in person. We’re looking for someone who’s excited to work alongside the team during those days. What You'll Achieve: Own recruiting analytics end-to-end: Build and maintain the dashboards and reporting infrastructure Recruiting leadership relies on (funnel health, conversion rates by stage, time-to-fill,

About the Team The Human Data team turns human feedback into reliable signals for training and evaluation. We design and run end-to-end programs that capture the depth of human intent behind everyday and high-stakes uses of our models. Our remit spans bespoke data campaigns, scalable synthetic data generation, and product-embedded signals. We partner closely across all research teams to translate these signals into training datasets, novel evaluations, and feedback loops that push the frontier of our models and advance their applications. About the Role As a Program Manager (PGM) in the Human Data team you will partner with our research teams, operations and engineering to execute complex programs for collecting high-quality data. You will be a key interface between our external vendors and AI trainers, ensuring human data campaigns are successfully completed. Your work will play a key role in enabling OpenAI to train safe models that will land in the real world This role is based in our San Francisco HQ. In this role, you will: Work in a high velocity environment, where the outcome of your work will have a direct impact on the models that OpenAI deploy in the real world Work closely with external vendors, trainers and internal researchers to collect, review, and deliver high-quality data Gather requirements, write instructions, define success criteria, and calibrate the AI trainers Use internal tooling to assess labeled data and provide feedback to AI trainers Think critically and share recommendations on tooling and process improvements, optimizing for quality, throughput, and AI trainer experience You’ll thrive in this role if: You thrive in dynamic environments. You are comfortable navigating ambiguity, managing shifting priorities, and adapting to fast-paced changes without missing a beat. You’re curious about AI, LLMs, Agents. While not required, an interest or background in these areas will help you connect the dots in our broader mission. You have a can-do a

About the Team OpenAI’s mission is to ensure that artificial general intelligence (AGI) benefits all of humanity. A key part of achieving that mission is training models that deeply understand and reflect human preferences — the Human Data team is at the heart of that effort. The Human Data engineering team creates the systems that enable scalable, high-quality human feedback. These systems are essential to how OpenAI trains and improves its most advanced models. Engineers on this team collaborate closely with world-class researchers to bring alignment techniques to life — from experimental ideas to production-ready feedback loops. About the Role We’re looking for software engineers to join the Human Data team and build the platforms, prototypes, tools, and infrastructure that power how our AI models are trained, aligned, and evaluated. You’ll partner with researchers and cross-functional teams to bring alignment ideas to life, influence future model training, and shape how models interact with the real world. We’re looking for people who are excited by technical ownership, enjoy working across the stack, and are eager to solve ambiguous problems in a high-impact, fast-paced environment. This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees. In this role, you will: Build and maintain robust full-stack systems for feedback collection, data labeling, and evaluation pipelines, while maintaining high levels of security. Translate experimental alignment research into scalable production infrastructure, including inference and model training stacks. Design and iterate on user-facing tools and backend services to support high-quality data workflows Partner with researchers, engineers, and program leads to shape feedback loops and model interaction paradigms Drive infrastructure improvements that enable faster iteration and scaling across OpenAI’s frontier models, from internal r

About The Team The Data Understanding team is responsible for creating the high quality datasets and their quantized representation for OpenAI. This includes synthesizing data, building VQ representations, and processing, filtering, deduplication, quality control, and tokenization so it can be used effectively in big model training runs. About The Role We're looking to advance how OpenAI builds and understands pretraining data at scale. You'll treat data quality and curation as core research problems: developing new methods to select, combine, and transform data; creating datasets that improve model capabilities; and designing rigorous experiments to understand how data choices and interventions affect model learning and downstream behavior. You'll work closely with frontier models and web-scale data to build evidence for which approaches work and why, then translate successful research into scalable data processing pipelines We Expect You To Have a strong track record of new or improved ML ideas, through publications, projects, or applied research. Own and drive a research agenda, from choosing the right problems to carrying long-running work through to impact. Be excited by OpenAI’s empirical, collaborative approach to research. Nice To Have Thoughtfulness about AI’s impact, including privacy, provenance, and data quality. Experience building high-performance deep learning or large-scale data processing systems. About OpenAI OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity. We are an equal opportunity employer

About The Team The Data Understanding team is responsible for creating the high quality datasets and their quantized representation for OpenAI. This includes synthesizing multimodal data, building VQ representations, and processing, filtering, deduplication, quality control, and tokenization so it can be used effectively in big model training runs. About The Role We’re looking to advance how OpenAI prepares, curates, synthesizes and understands multimodal data at scale. You’ll work on research and production problems like synthesizing multimodal content (images, audio, and video) and their supervisions, improving noisy data pipelines, building better quality filters, using models to automate data prep, and measuring whether changes in the dataset improve model performance. We Expect You To Have a strong track record of new or improved ML ideas, through publications, projects, or applied research. Own and drive a research agenda, from choosing the right multimodal data problems to carrying long-running work through to impact. Be excited by OpenAI’s empirical, collaborative approach to research. Nice To Have Experience with multimodal learning, audio, vision, video, synthetic data, or data-centric ML. Thoughtfulness about AI’s impact, including privacy, provenance, and data quality. Experience building high-performance deep learning or large-scale data processing systems. About OpenAI OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity. We are an equal opportunity employer, and we do not discriminate on the basis of

About the Team The Strategic Initiatives & Operations team is in need of a Technical Program Manager (TPM) to streamline our processes, including full safety governance and integration of various safety research and mitigations into our ChatGPT, API, and any frontier models. This role is critical for driving safe deployment of our new models, synthesizing inputs from multiple stakeholders, ranging across research, product, engineering, legal and policy, and ensuring all the risks are effectively and properly monitored, mitigated or resolved. About the Role As a TPM, you will be responsible for critical tasks ranging from tracking safety research progress and risk tables to overseeing the quality of human data campaigns – acting as the connective tissue to enhance the deployment of OpenAI’s safety system. Additionally, you will create and execute a compute roadmap for your team to ensure that our top priorities are resourced while taking advantage of new opportunities to make key safety research discoveries. Your primary focus will be to ensure our models are qualified for safe deployment. This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees. In this role, you will: Manage key risk areas and corresponding stakeholders. Keep track of a stack of existing and future mitigations for every major product and model deployment. Standardize the lifecycle of risk assessment, setting safety bars, consolidating inputs from multiple stakeholders across research, product, engineering, legal and policy, pre-launch safety reviews and post-launch followup. Manage pre-launch safety reviews. Share launch calendars and key safety practices and evaluations with our key parter (i.e. Microsoft). Develop comprehensive documentation for all the safety work, including metrics, evaluations, and progress tracking across multiple teams within OpenAI. Help with publishing and open sourcing safety

About the Team The Personal AGI team is responsible for training and improving pre-trained models to be deployed into ChatGPT, the API, and potential future products. In the Model Experience team, we shape the default character and behavior of ChatGPT: how the model communicates, responds to users, uses its capabilities, and behaves across different contexts and languages. Our goal is to make every interaction with ChatGPT thoughtful, helpful, and trustworthy. We take an opinionated view of what good human–AI interaction should look like, then turn that vision into real model behavior through human data, evaluations, reward models, and post-training. Our work sits at the intersection of research, product, and model design. We partner closely with teams across OpenAI to conduct research and ensure our models are thoughtful, safe, reliable to serve millions of users. About the Role As a Research Engineer / Scientist, you will research and develop improvements to our models. Our team works in research areas combining reinforcement learning and products. We're looking for individuals with strong ML engineering skills and research experience, especially with novel and highly capable models. An ideal candidate is passionate about product-driven research and the quality of human-AI interaction. This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees. In this role, you will: Own and pursue a research agenda to improve model capability and performance. Collaborate closely with the other research and product teams, allowing customers to optimize their own models. Build robust evaluations for tracking modeling improvements. Design, implement, test, and debug code across our research stack. You might thrive in this role if you: Have a deep understanding of machine learning and machine learning applications. Have good judgment about model behavior and can communicate this judgment effec

About the Team The Product & Platform teams at OpenAI are responsible for delivering the company’s most impactful offerings—such as ChatGPT, our API platform, and new enterprise capabilities—to a global and diverse customer base. These systems must perform at scale and deliver exceptional experiences to developers, consumers, and businesses alike. The ChatGPT Multimodal team works across voice, image generation, and other multimodal experiences to turn frontier research capabilities into reliable products. The team connects product usage and failure patterns with research, evaluation, data, inference, capacity, and external partnerships so that model and product improvements translate into better experiences for users. About the Role We are seeking a Technical Program Manager to build the flywheel that helps ChatGPT multimodal products learn from real-world usage and improve quickly. You will lead programs spanning production-signal mining, evaluation and data pipelines, research-to-production parity, multimodal capacity planning, and complex cross-functional dependencies for voice and image-generation launches. You will work closely with product engineering, research, Human Data, inference and capacity teams, safety partners, and external vendors or product partners. Success requires technical depth, strong systems thinking, comfort with ambiguity, and the ability to turn fragmented or manual work into durable mechanisms that teams adopt. This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees. In this role, you will: Build a system for mining production conversations and product signals to identify representative multimodal workflows, user needs, and failure modes. Establish and maintain evaluations for the highest-priority multimodal behaviors and use cases, with clear coverage, quality standards, and ownership. Package production signals into decision-ready data and

About the Team The Core Models team shapes how our models interact with people. We view the model as the product itself, aiming for intuitive experiences that exceed user expectations and feel like magic. About the Role As a Model Designer, you’ll have an outsized impact on how our models interact and resonate with users. You’ll strike a delicate balance between maximizing the model’s capabilities, reading in between the lines in user queries to understand how best to help, and upholding user trust. We’re looking for people who are passionate about the intersection of design, technology, and user experience — and are up for the challenge of defining new human-AI interaction paradigms. This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees. In this role, you’ll join a small team in evolving and expanding the model design function. You will: Collaborate closely with researchers to understand, predict, and design model behavior. Partner with product managers and designers across the company to ensure a cohesive voice and approach. Proactively identify ways to improve our models based on product sense, user feedback, quantitative insights, and the research roadmap. Come up with creative strategies for collecting high-quality data. Do whatever needs to be done to make our models better. You might thrive in this role if you: Possess exceptional taste, creativity, and writing skills, allowing you to craft responses that delight users. Don’t mind ambiguity — you’re happy to throw yourself into a new, unfamiliar environment, build relationships, define a problem, and make progress. Love experimentation, and are willing to test and reject new ideas when the results don’t pan out. Exhibit high levels of empathy and self-awareness required to serve everyone in the world. Enjoy tackling profound and often philosophical questions while always driving towards clarity. Demonstrate technic

🚀 About WRITER WRITER is where the world's leading enterprises orchestrate AI-powered work. Our vision is to expand human capacity through superintelligence. And we're proving it's possible – through powerful, trustworthy AI that unites IT and business teams together to unlock enterprise-wide transformation. With WRITER's end-to-end platform, hundreds of companies like Mars, Marriott, Uber, and Vanguard are building and deploying AI agents that are grounded in their company's data and fueled by WRITER's enterprise-grade LLMs. Valued at $1.9B and backed by industry-leading investors including Premji Invest, Radical Ventures, and ICONIQ Growth, WRITER is rapidly cementing its position as the leader in enterprise generative AI. Founded in 2020 with office hubs in San Francisco, New York City, Seattle, Austin, Chicago, and London, our team thinks big and moves fast, and we're looking for smart, hardworking builders and scalers to join us on our journey to create a better future of work with AI. 📐 About the role Are you passionate about ensuring the highest quality for cutting-edge generative AI applications? As a software quality engineer at WRITER, you'll play a critical role in shaping the reliability, performance, and trustworthiness of our AI-powered work orchestration platform. You’ll be at the forefront of defining and implementing rigorous quality strategies for our enterprise-grade LLMs and AI agents, directly impacting how hundreds of global companies unlock transformational value through AI. This is a unique chance to dive deep into the unique challenges of AI quality assurance and make a tangible difference in a rapidly evolving field. This is a hybrid role based out of our London, San Francisco, Seattle, and New York City hubs. You will report directly to the director of engineering. 🦸🏻‍♀️ What you'll do Define and implement comprehensive quality assurance strategies and test plans for our AI agents and LLM-powered applications, ensuring exceptional prod

About the Team GTM Growth Engineering builds AI-native products that help OpenAI's go-to-market and B2B marketing organizations scale with greater speed, intelligence, and operational effectiveness. We apply OpenAI models to real business workflows and build the systems that make those applications useful and dependable: customer context, agent behavior, feedback, evaluation, experimentation, and appropriate human oversight. Our work brings together software engineering, applied AI, product, data, and GTM operations. We measure success through the quality of customer engagement, pipeline, conversion, and the effectiveness of our sales and marketing teams. About the Role We're looking for an Applied AI Engineer to build production systems that help AI-powered go-to-market workflows improve over time. You will connect agent behavior, customer and operator feedback, evaluation, experimentation, and business outcomes to make these systems more effective, reliable, and responsive to evolving customer needs. This is a deeply technical, cross-functional role with end-to-end ownership of the agent improvement loop: understand production behavior, identify failure modes, improve how the system decides or acts, and validate the resulting impact. You will partner with Engineering, Product, Data Science, Sales, and B2B Marketing to turn real-world signals into safer, more effective agent behavior and measurable improvements in customer engagement, conversion, qualified pipeline, and team productivity. In this role, you will: Own the production improvement loop across agent behavior, customer and operator feedback, evaluation, experimentation, and verified business outcomes. Instrument agent workflows so model interactions, tool use, decisions, failures, human edits, and downstream outcomes can be understood in context. Define meaningful quality standards, representative evaluation datasets, regression coverage, and production monitoring for real GTM workflows. Investigate why a

Who We Are Notion is the collaborative AI workspace where teams and agents think together . We're building one place where your knowledge, projects, meetings, and AI tools live side by side, so work is faster, clearer, and less fragmented. Millions of individuals, small teams, and large companies run their work on Notion. Notinos (our employees) are customer zero in bringing this future of work to life. We care about craft, building things that last, and the belief that great work is still fundamentally human. Our goal isn’t to ship the next feature. Each and every team of Notinos is working to set the standard for how humans work together in the AI era. From building a business’s system of record to making and managing AI agents to automating away the busy work, we care deeply about giving our customers more time for their life’s work. About the Role: We’re seeking an experienced UX Researcher to define and scale how we evaluate Notion’s AI-powered experiences—focusing on what “good” looks like not only for model output quality, but for the end-to-end product experience where people discover, set goals, delegate work, review results, and build trust over time with AI. This role sits at the intersection of research craft and evaluation operations: you’ll run studies that uncover user mental models, expectations, and failure/recovery behaviors, then translate those insights into reusable rubrics, workflows, and measurement approaches that product, design, engineering, and data science can apply consistently. This role can be based in either San Francisco or New York City. We work from our offices on Mondays, Tuesdays and Thursdays (our Anchor Days) because we do our best thinking and building together in person. We’re looking for someone who’s excited to work alongside the team during those days. What You'll Achieve: Define what “good” looks like (frameworks & rubrics): Establish clear, reusable evaluation criteria that reflect real user expectations—helpfulness,

About the Team Business Systems / Enterprise Platform Technology builds the internal systems, data foundations, workflow infrastructure, and enterprise platforms that help OpenAI operate at scale. The EPT AI Pod builds AI-native internal apps, MCP connectors, multi-agent workflows, and reusable platform capabilities across Finance, People, and GTM. About the Role As an Enterprise Applied AI Engineer, you will build internal apps for enterprise operations and the shared platform components those apps run on. This includes MCP connectors, multi-agent orchestration, data architecture, evals, monitoring, auditability, and governance. We’re looking for a hands-on engineer who is strong in Python, system design, enterprise integrations, data architecture, and applied AI systems. You should be excited to turn ambiguous business workflows into reliable internal products and shared infrastructure. In this role, you will: • Build internal apps for enterprise operations across Finance, People, and GTM • Build MCP connectors and enterprise integrations with strong auth, permissions, idempotency, retries, and rate-limit handling • Design end-to-end multi-agent workflows with tool routing, human approvals, audit trails, and safe action boundaries • Design data architecture for operational AI systems, including ingestion, schemas, quality checks, lineage, and governance • Build evals, monitoring, metrics, and regression tests for agentic workflows • Create reusable infrastructure, patterns, and components that other enterprise teams can build on • Partner with system owners and business owners to turn messy enterprise workflows into reliable internal products You might thrive in this role if you: • Have strong Python engineering skills for backend services, MCP connectors, agent/tool workflows, eval harnesses, and data ingestion jobs • Have strong system design skills across shared infrastructure, app architecture, reliability, and scaling • Have experience building internal apps,