Human Evaluator Jobs

Zscaler (NASDAQ: ZS) accelerates digital transformation so customers can be more agile, efficient, resilient, and secure. The Zscaler Zero Trust Exchange™️ platform protects thousands of customers from cyberattacks and data loss by securely connecting users, devices, and applications in any location. Distributed across 160+ public exchanges globally and thousands of private exchanges at the edge, the SASE-based Zero Trust Exchange is the world’s largest in-line cloud security platform. We believe the future of work is Human + AI and are building an AI-native enterprise where human potential is amplified by machine intelligence to solve the world’s hardest security challenges. Driven by deep customer obsession, we are committed to the mission, outcome, and to each other. We bring these commitments to life through three core behaviors: ownership and collaboration, trust through outcomes and impact, and a challenge culture with ongoing feedback. Ready to make an impact at the company pioneering security transformation in the AI era? Join us at Zscaler. Role We are looking for a Senior Sales Engineer to join our team. This is a Texas based role, reporting to the Sales Engineering Manager for the Americas Field Sales team in the South Enterprise department. You will be responsible for architecting technical solutions that directly address and resolve critical customer pain points. You will serve as a technical advisor throughout the sales cycle, ensuring that Zscaler solutions are perfectly aligned with customer goals and technical requirements. What you’ll do (Role Expectations) Creating and delivering technical product presentations to prospective customers where you will promote the value proposition of Zscaler products Gathering customer technical requirements to ensure proposed solutions will meet customer goals and needs Leading product evaluations and custom product configurations Running Proof of Value from start to finish, as the technical and industry expert Co

Zscaler (NASDAQ: ZS) accelerates digital transformation so customers can be more agile, efficient, resilient, and secure. The Zscaler Zero Trust Exchange™️ platform protects thousands of customers from cyberattacks and data loss by securely connecting users, devices, and applications in any location. Distributed across 160+ public exchanges globally and thousands of private exchanges at the edge, the SASE-based Zero Trust Exchange is the world’s largest in-line cloud security platform. We believe the future of work is Human + AI and are building an AI-native enterprise where human potential is amplified by machine intelligence to solve the world’s hardest security challenges. Driven by deep customer obsession, we are committed to the mission, outcome, and to each other. We bring these commitments to life through three core behaviors: ownership and collaboration, trust through outcomes and impact, and a challenge culture with ongoing feedback. Ready to make an impact at the company pioneering security transformation in the AI era? Join us at Zscaler. We are looking for a Commercial Sales Engineer to join our East team. This is a remote role, reporting to the Manager of Sales Engineering. The Zscaler Sales and Go-to-Market team is a global group of professionals who are passionate about delighting our customers, nurturing trusted partnerships, and sharing their expertise to drive a secure, cloud-enabled digital future and further cement our position as the world leader in cloud security. Sales, Customer Success, Sales Enablement, Solution Architects, Business Development, Transformation, and Technology Partnerships all work together to demonstrate the power and agility of Zscaler cloud transformation to the world. What you’ll do (Role Expectations) Create and deliver technical product presentations to prospective customers Gather customer technical requirements to ensure proposed solutions will meet customer goals and needs Lead product evaluations and custom pr

Zscaler (NASDAQ: ZS) accelerates digital transformation so customers can be more agile, efficient, resilient, and secure. The Zscaler Zero Trust Exchange™️ platform protects thousands of customers from cyberattacks and data loss by securely connecting users, devices, and applications in any location. Distributed across 160+ public exchanges globally and thousands of private exchanges at the edge, the SASE-based Zero Trust Exchange is the world’s largest in-line cloud security platform. We believe the future of work is Human + AI and are building an AI-native enterprise where human potential is amplified by machine intelligence to solve the world’s hardest security challenges. Driven by deep customer obsession, we are committed to the mission, outcome, and to each other. We bring these commitments to life through three core behaviors: ownership and collaboration, trust through outcomes and impact, and a challenge culture with ongoing feedback. Ready to make an impact at the company pioneering security transformation in the AI era? Join us at Zscaler. We are looking for a Transformation Architect to join our Emerging markets team in a remote capacity in Mexico, reporting to the Senior Director, Sales Engineering within the Commercial Americas department. You will drive customer engagement and solution design, providing the technical leadership and expertise necessary for deployment design and optimization. What you’ll do (Role Expectations) Design and implement our Zero Trust architectures Gather requirements and understand specifications while evaluating existing systems Influence and mentor activities within the team and across the organization Who You Are (Success Profile) You thrive in ambiguity. You are comfortable building the path as you walk it and see ambiguity as raw material to build something meaningful. You act like an owner. You navigate seamlessly between high-level strategy and hands-on execution, fueled by a bias for action. You are a problem-sol

Zscaler (NASDAQ: ZS) accelerates digital transformation so customers can be more agile, efficient, resilient, and secure. The Zscaler Zero Trust Exchange™️ platform protects thousands of customers from cyberattacks and data loss by securely connecting users, devices, and applications in any location. Distributed across 160+ public exchanges globally and thousands of private exchanges at the edge, the SASE-based Zero Trust Exchange is the world’s largest in-line cloud security platform. We believe the future of work is Human + AI and are building an AI-native enterprise where human potential is amplified by machine intelligence to solve the world’s hardest security challenges. Driven by deep customer obsession, we are committed to the mission, outcome, and to each other. We bring these commitments to life through three core behaviors: ownership and collaboration, trust through outcomes and impact, and a challenge culture with ongoing feedback. Ready to make an impact at the company pioneering security transformation in the AI era? Join us at Zscaler. We are looking for a Senior Sales Engineer to join our Canada team. This is a remote role, reporting to the Manager of Sales Engineering. The Zscaler Sales and Go-to-Market team is a global group of professionals who are passionate about delighting our customers, nurturing trusted partnerships, and sharing their expertise to drive a secure, cloud-enabled digital future and further cement our position as the world leader in cloud security. Sales, Customer Success, Sales Enablement, Solution Architects, Business Development, Transformation, and Technology Partnerships all work together to demonstrate the power and agility of Zscaler cloud transformation to the world. What you’ll do (Role Expectations) Create and deliver technical product presentations to prospective customers Gather customer technical requirements to ensure proposed solutions will meet customer goals and needs Lead product evaluations and custom prod

Zscaler (NASDAQ: ZS) accelerates digital transformation so customers can be more agile, efficient, resilient, and secure. The Zscaler Zero Trust Exchange™️ platform protects thousands of customers from cyberattacks and data loss by securely connecting users, devices, and applications in any location. Distributed across 160+ public exchanges globally and thousands of private exchanges at the edge, the SASE-based Zero Trust Exchange is the world’s largest in-line cloud security platform. We believe the future of work is Human + AI and are building an AI-native enterprise where human potential is amplified by machine intelligence to solve the world’s hardest security challenges. Driven by deep customer obsession, we are committed to the mission, outcome, and to each other. We bring these commitments to life through three core behaviors: ownership and collaboration, trust through outcomes and impact, and a challenge culture with ongoing feedback. Ready to make an impact at the company pioneering security transformation in the AI era? Join us at Zscaler. We are looking for a Senior Sales Engineer to join our Canada team. This is a remote role, reporting to the Manager of Sales Engineering. The Zscaler Sales and Go-to-Market team is a global group of professionals who are passionate about delighting our customers, nurturing trusted partnerships, and sharing their expertise to drive a secure, cloud-enabled digital future and further cement our position as the world leader in cloud security. Sales, Customer Success, Sales Enablement, Solution Architects, Business Development, Transformation, and Technology Partnerships all work together to demonstrate the power and agility of Zscaler cloud transformation to the world. What you’ll do (Role Expectations) Create and deliver technical product presentations to prospective customers Gather customer technical requirements to ensure proposed solutions will meet customer goals and needs Lead product evaluations and custom prod

About the Team The Product & Platform teams at OpenAI are responsible for delivering the company’s most impactful offerings—such as ChatGPT, our API platform, and new enterprise capabilities—to a global and diverse customer base. These systems must perform at scale and deliver exceptional experiences to developers, consumers, and businesses alike. The ChatGPT Multimodal team works across voice, image generation, and other multimodal experiences to turn frontier research capabilities into reliable products. The team connects product usage and failure patterns with research, evaluation, data, inference, capacity, and external partnerships so that model and product improvements translate into better experiences for users. About the Role We are seeking a Technical Program Manager to build the flywheel that helps ChatGPT multimodal products learn from real-world usage and improve quickly. You will lead programs spanning production-signal mining, evaluation and data pipelines, research-to-production parity, multimodal capacity planning, and complex cross-functional dependencies for voice and image-generation launches. You will work closely with product engineering, research, Human Data, inference and capacity teams, safety partners, and external vendors or product partners. Success requires technical depth, strong systems thinking, comfort with ambiguity, and the ability to turn fragmented or manual work into durable mechanisms that teams adopt. This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees. In this role, you will: Build a system for mining production conversations and product signals to identify representative multimodal workflows, user needs, and failure modes. Establish and maintain evaluations for the highest-priority multimodal behaviors and use cases, with clear coverage, quality standards, and ownership. Package production signals into decision-ready data and

About the Team Our Safety Systems team is at the forefront of OpenAI's mission to build and deploy safe AGI, driving our commitment to AI safety and fostering a culture of trust and transparency. Within Safety Systems, the Model Policy team works to ensure that increasingly capable models behave safely and reliably in real-world environments. We investigate emerging model failures, define the behavior models should exhibit instead, and develop the data, evaluations, monitoring, and safeguards needed to improve and validate that behavior. Our work connects alignment research with the practical challenges of training and deploying frontier models. About the Role In this role, you will shape how OpenAI understands and addresses real-world risks that emerge from model misalignment as models become more autonomous and operate over longer horizons. You will investigate how misaligned behavior emerges across extended trajectories - including when models persist toward the wrong objective, take unsafe shortcuts, lose track of instructions, exploit weaknesses in their environment, or circumvent constraints - and translate these insights into behavioral policies, evaluations, monitoring, and safeguards. This role is ideal for someone who wants to turn alignment and safety concerns into concrete, empirically grounded improvements to frontier AI systems. Your Responsibilities: Identify vulnerabilities that emerge as models interact with tools, data, and external systems, and translate them into model- and system-level safeguards. Develop threat models and empirical frameworks for understanding harmful outcomes from misaligned behavior. Build frameworks for understanding harmful outcomes arising from model misalignment. Identify the underlying behaviors and system conditions that drive those outcomes. Turn findings into policy frameworks, evaluation criteria, online measurement and safeguards. Develop human data campaigns and gold sets to ground measurement and evaluation of eme

About the Team OpenAI’s People team hires, engages, and retains world-class talent to safely build and deploy AGI that benefits all of humanity. The People Analytics team helps leaders make rigorous, evidence-based talent decisions and ensures that the systems supporting those decisions are valid, reliable, fair, and accountable. About the Role As a People Data Scientist focused on AI fairness and bias testing, you will help establish how OpenAI evaluates AI-assisted People systems and high-impact talent processes. You will design and conduct rigorous assessments to identify, measure, and mitigate potential bias across the lifecycle of models, agents, decision-support tools, and automated workflows. Your work will span the entire employee life-cycle, such as hiring, performance, promotion, employee development, workforce planning, etc. You will evaluate both technical systems and the broader human-AI decision processes in which they operate, examining not only model performance but also data quality, measurement validity, differential outcomes, human oversight, and unintended consequences. We’re looking for an experienced data scientist or applied researcher who can translate complex fairness questions into defensible evaluation strategies, scalable testing infrastructure, and clear recommendations for technical teams and senior leaders. This role is preferred to be based in San Francisco, CA. In this role, you will: Define and lead fairness and bias-testing strategies for AI-assisted People processes, models, agents, and decision-support systems from development through deployment and ongoing monitoring. Design rigorous algorithmic audits and validation studies, including adverse-impact analysis, subgroup and intersectional evaluation, error-rate analysis, calibration, measurement invariance, reliability, criterion-related validity, and sensitivity testing. Identify the appropriate fairness criteria for each use case, evaluate tradeoffs among competing definitions

What if the work you did every day could impact the lives of people you know? Or all of humanity? At Illumina, we are expanding access to genomic technology to realize health equity for billions of people around the world. Our efforts enable life-changing discoveries that are transforming human health through the early detection and diagnosis of diseases and new treatment options for patients. Working at Illumina means being part of something bigger than yourself. Every person, in every role, has the opportunity to make a difference. Surrounded by extraordinary people, inspiring leaders, and world changing projects, you will do more and become more than you ever thought possible. Position Summary: Designs, models, and develops data management systems that integrate laboratory, operation and research data. Designs, builds, improves and maintains high performance and highly scalable data pipelines. Oversees transition of data pipelines to production and commercial settings; develops technology choices and practices that support pipeline maturation. Researches and assesses the viability of new data management and processing technologies. Scope of Responsibilities: Applies subject matter expertise and extensive knowledge of theories, practices, and design matters to lead the application of existing principles and guide the development of new ideas. Provides, guides, and uses professional concepts to resolve critical issues. Works on highly complex assignments that address future concepts where problem solving requires conceptual thinking and an evaluation of enigmatic components. Accountable for results which may impact multiple functions. Enhances formal networks with key decision makers and recognized as a thought le

Become a part of our caring community Most AI engineering jobs are a thin wrapper around a model API. This role is different. We build the platform that transforms millions of clinical documents into trusted, actionable data. Our systems use large language models (LLMs) to read medical records, extract structured facts, answer complex questions with citations back to the source document, and route ambiguous cases to human experts for review. Our users make decisions that impact real healthcare outcomes, so “good enough” is not good enough. Building AI systems that are accurate, reliable, auditable, and scalable is at the core of this role. As a Senior AI Applied Engineer, you will design, build, deploy, and operate production AI systems used at scale within one of the largest health insurers in the United States. You will own solutions end-to-end, from user experience and APIs to model orchestration, evaluation frameworks, infrastructure, and production operations. Why Join Us Build production AI systems where LLMs are in the critical path, not just demos or proofs of concept. Work on extraction, retrieval, agentic workflows, and human-review systems that process real healthcare data at scale. Own projects end-to-end across frontend, backend, AI orchestration, infrastructure, deployment, and operations. Solve challenging problems around accuracy, explainability, traceability, and reliability in regulated environments. Ship quickly in a small, high-impact team that embraces AI-assisted development and rigorous quality standards. Build systems that continuously improve through expert feedback, evaluations, and human-in-the-loop workflows. Key Responsibilities Design, develop, and deploy full-stack AI-powered application

Become a part of our caring community Every large organization is making critical decisions today about how it will leverage AI over the next decade. Few have leaders who can both define that vision and demonstrate its viability through hands-on engineering. This role requires both. We build the platform that transforms millions of clinical documents into trusted, actionable data. Our systems use large language models (LLMs) to read medical records, extract structured facts, answer complex questions with citations to source documents, and route difficult cases to human experts. These capabilities support decisions that impact real healthcare outcomes for members. As a Principal AI Applied Engineer, you will define the technical strategy, architectural standards, and long-term vision for AI-enabled products across the organization. You will influence enterprise-wide decisions regarding AI platforms, model strategies, engineering standards, and technology investments while remaining deeply hands-on in prototyping, experimentation, architecture, and software development. This is the highest-level individual contributor role within the AI Applied Engineering organization. Success requires exceptional technical depth, organizational influence, strategic thinking, and the ability to translate emerging AI capabilities into scalable, reliable, and responsible production systems. Why Join Us Shape the long-term AI architecture and engineering direction for a large enterprise healthcare organization. Influence how AI-enabled products are designed, built, evaluated, deployed, and governed across multiple teams. Drive strategic decisions involving models, vendors, platforms, infrastructure, and shared capabilities. Prototype and validate emerging technologies before the organization invests at scale.</

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. As a Senior Software Engineer within the Creator Organization, you will develop innovative full-stack solutions that define the future of Roblox’s Content Understanding Platform. This platform processes billions of pieces of content—spanning 3D models, audio files, text, video, and entire experiences—extracting structured information about their meaning, context, and relationships. The Content Understanding Team develops cutting-edge AI models, advanced computer vision systems, and highly scalable backend platforms to power search, discovery, and moderation across Roblox. Your contributions will enable seamless asset discovery, automate moderation at scale, and drive transformative generative AI tools that reshape how millions of creators and users engage with Roblox. You Will: Solve full-stack challenges to improve how AI, creators, and users describe and get along with content, including images, 3D models, audio, text, and video. Craft and build scalable pipelines for training, evaluating, and deploying machine learning models to support content annotation and discovery. Develop robust backend systems to power real-time search, discovery, and powerful generative AI features. Blend innovat

We're Hiring Senior AI Engineer (Generative AI Azure) Remote Immediate Joiners Preferred Salary: Up to 12 LPA We are seeking an experienced Senior AI Engineer to design, develop, and deploy enterprise-grade Generative AI solutions on Microsoft Azure. The ideal candidate will have strong expertise in Large Language Models (LLMs), Agentic AI, Retrieval-Augmented Generation (RAG), Model Context Protocol (MCP), Azure AI Services, and modern AI engineering practices. Technical Summary • AI & LLMs: Azure OpenAI, GPT-4.x, OpenAI, Prompt Engineering, Prompt Chaining, Function Calling, Structured Outputs, Tool Calling, JSON Schema, Model Evaluation, Guardrails, Fine-tuning Concepts • Agentic AI: Multi-step Reasoning, Planning, Memory Management, Tool Orchestration, Multi-Agent Systems, Human-in-the-Loop Workflows, Reflection, Context Management, AI Observability • Frameworks: Semantic Kernel, LangChain, LangGraph, AutoGen, Azure AI Agent Service • RAG & MCP: Retrieval-Augmented Generation, Vector Search, Semantic Search, Hybrid Search, Embeddings, Knowledge Grounding, Citation Generation, Document Ingestion, Chunking, MCP Architecture, MCP Servers & Clients • Azure Technologies: Azure AI Foundry, Azure AI Search, Azure AI Document Intelligence, Azure Machine Learning, Azure Functions, API Management, Logic Apps, App Service, Container Apps, AKS, Azure Storage, Data Lake Gen2, Azure Key Vault, Azure Entra ID, Azure Monitor, Application Insights, Event Grid, Service Bus • Programming & DevOps: Python, C#, REST APIs, FastAPI, ASP.NET Core, JSON, YAML, Git, Azure DevOps, Docker, Kubernetes • Data Platforms: SQL Server, Azure SQL, PostgreSQL, Cosmos DB, Snowflake, Microsoft Fabric, Azure Databricks, Delta Lake, Vector Databases (Azure AI Search, Pinecone, Weaviate, Milvus, Qdrant) • Security & Governance: Responsible AI, Prompt Injection Prevention, RBAC, Content Filtering, Data Privacy, GDPR, ISO 27001, Azure Key Vault, Audit Logging & Monitoring • Nice to Have: Microsoft Copilo

At ClickUp, we're building the future of work: the first truly converged AI workspace unifying tasks, docs, chat, calendar, and enterprise search, all supercharged by context-driven AI. We are an AI-native company. Every team member is expected to leverage AI daily, and we evaluate AI fluency as part of our hiring process. Join us and help redefine what's possible. 🚀 We’re hiring a highly influential product analytics leader who can turn ambiguous questions into sharp insight, scalable measurement, and recommendations that directly shape what we build. This person will partner closely with Product, Engineering, Design, and Growth to raise the bar on decision quality and establish a more AI-native analytics operating model. Mission Drive the product insight agenda by helping ClickUp make faster, smarter product decisions through rigorous analysis, strong product judgment, and AI-enabled analytics workflows. What You'll Do Own the product analytics agenda across product usage, activation, feature adoption, retention, and expansion, and translate open-ended business questions into structured analyses and clear recommendations Partner with Product, Engineering, and Design to define success metrics early, improve instrumentation quality, and ensure important product surfaces are measurable from launch Build reusable analysis frameworks, semantic layers, metric definitions, and self-serve resources that help product teams answer routine questions faster and more consistently Apply AI-first methods across the analytics workflow, using large language models, coding agents, and automation for tasks like query drafting, QA, validation, documentation, and first-pass synthesis while keeping human judgment at the center of final recommendations Design and interpret experiments, observational analyses, and trend investigations, including situations where data is incomplete or traditional experimentation is not feasible Surface meaningful patterns in behavioral, subscription, and

For over 20 years, Smartsheet has empowered teams to manage work seamlessly and scale solutions smarter. Now, in our most ambitious chapter yet, we are uniting human teams with AI agents. By orchestrating the work agents do best, automating manual tasks and uncovering insights at scale, we create the space for people to focus on what truly matters: judgment, creativity, and big thinking. That is magic at work, and it’s what we show up for every day. Smartsheet is looking for a Senior Enterprise Solutions Engineer who has a passion for solving complex challenges. In this role, you will partner directly with Sales to provide technical leadership throughout the sales process, navigating intricate deal cycles, providing value-based demonstrations, and serving as a trusted advisor to prospects and clients throughout their evaluations. You will be a force multiplier within the Solutions Engineering organisation that is on a mission to grow revenue and to empower anyone to drive meaningful change — for themselves, their businesses, and the world. This role is part of the Go-To-Market team reporting into the Director, Solutions Engineering APJ based in Sydney, Australia. You Will: Identify, build, and demonstrate innovative solutions to solve complex business challenges for high visibility Smartsheet customers Lead technical sales engagements, collaborate closely with sales teams to identify customer requirements, summarize challenges, design solutions, and provide technical guidance that demonstrates the value of Smartsheet's offerings Conduct discovery sessions to uncover business processes and challenges to inform the sales and solution delivery strategy Establish yourself as a st