Human Evaluator Jobs

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. About Okta: Okta is the leading independent provider of identity solutions for enterprises. With a deep commitment to innovation and customer success, we power and secure digital transformation for the world's biggest brands. We're scaling rapidly, and we’re seeking talented individuals to join us in enabling secure and seamless digital experiences. One of the most important components of our operations is an efficient financial system, including seamless processes for the Record-to-Report (R2R) cycle. As a NetSuite R2R Analyst, your role will support and strengthen Okta's financial systems and processes to ensure accuracy, efficiency, and scalability. This is a potential Contract to Hire position. Roles & Responsibilities: Act as the subject matter expert (SME) for NetSuite functionality in Record-to-Report (R2R) processes, maintaining workflows, configurations, and data integrity to enhance efficiency and compliance (e.g., SOX, GAAP). Lead or participate in R2R-related projects, including implementations, migrations, process improvements, customizations, upgrades, and integrations with NetSuite’s financial modules. Support month-end, quarter-end, and year-end close processes by ensuring timely reconciliations, reporting, general ledger accuracy, consolidations, and inter company accounting. Partner with finance, IT, and external vendors to provide NetSuite support, resolve issues, streamline processes, evaluate automation opportunities, and deliver us

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. The Infrastructure Platform and Shared Services Team Okta authenticates, authorizes and provisions millions of users a day. The service is hosted on Amazon Web Services (AWS) across multiple availability zones and geographically separated regions. The service is designed for high throughput and 99.999 availability. We're looking for a technical leader to help us continue to scale the service with great people and reliable, cost-effective, and efficient infrastructure, processes, and tooling. As the Sr. Manager of Infrastructure Platform and Shared Services, you will oversee multiple teams focused on Edge networking, K8s platform, Observability, automation platform & tooling. What you’ll be doing Lead the Infra platform and shared services org and various initiatives across SRE & Infrastructure organization. Build a world-class observability platform and monitoring capabilities enabled with self-service Accelerate the velocity of SRE and product engineering by developing robust platforms, powerful tooling, and intuitive self-service capabilities. Own the design and operation of scalable, self-service Cloud infrastructure platforms (e.g. Observability Platform, SRE Productivity, deployments, and Edge Infrastructure) Lead, mentor, and grow a high-performing team of engineers and managers across SRE and infrastructure shared services domains. Perform engineering design evaluations and ensure the completion of projects within resource,

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. The Okta Privileged Access Management (PAM) is an identity-centric approach to a common and critical privileged access use case. Our elegant Zero Trust architecture is purpose-built for the modern cloud and helps customers solve challenging security and operations pain points at scale. We're looking for a Staff level Platform Engineer to join a team of highly skilled and talented team players who are proud of what they own and deliver. We are a fast, collaborative, and adaptable team. Operating with a weekly release cycle and a high degree of individual ownership, we empower our engineers to take on challenging projects, work with cutting-edge technologies, and directly share in Okta's success through meaningful equity. What you’ll do Leverage cutting-edge AI pair-programmers and LLMs (such as Copilot and Claude) to accelerate the development of secure, enterprise-grade Privileged Access Management (PAM) products Act as a core contributor to Okta’s FedRAMP initiative Work with engineering teams to design, develop and deliver cloud-based infrastructure projects on a modern tech stack (Kubernetes/EKS, RDS, DynamoDB, Kinesis, MKS, Redis, OpenSearch, Docker, Terraform on AWS) Drive evaluation, development, and rollout of microservices Operate, support, and upgrade shared services and frameworks. Scale these as their usage invariably grows along with Okta's business Evaluate and scale existing systems to meet specialized requirements and suppo

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. Here is the updated summary for the Okta Device Access (ODA) and Okta Verify (OV) space, adapted to match the collaborative, ownership-focused, and candidate-centric tone of your PAM example: In this role, you will work as a trusted cross-functional partner on our Okta Device Access (ODA) and Okta Verify (OV) products, and have the opportunity to influence product and design decisions through your work. You will own the UX roadmap for ODA and OV, working closely with your design, product, and engineering partners to ensure that your work is aligned to the highest priority needs of the endpoint identity area, informing both long and near-term decisions. You’ll provide mentorship and coaching to more junior team members, helping raise the quality bar and impact of the team. ODA and OV’s goal is to develop products that enable Okta customers to secure their workforce right from physical device bootup, providing frictionless, phishing-resistant, and passwordless authentication. In this role, you’ll get to: Grow and develop a research practice aimed at understanding the needs of the OPA and OV teams, collaborate with stakeholders to craft research questions and execute research Scope, prioritize, organize, and manage research initiatives across these teams. Exercise flexibility in research methodology (as appropriate for the product space and timeline) by designing and utilizing generative studies or evaluative research, appropriately. Facilitate cross-dis

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. The Identity Threat Protection Team Identity Threat Protection (ITP), powered by cutting-edge Okta AI, shatters the mold of traditional login security. We deliver real-time, surgical defense against the most sophisticated and rapidly evolving threats. By deeply integrating with existing security infrastructure, ITP relentlessly analyzes user behavior and system data to proactively hunt down and neutralize risks before they impact our customers. Join an elite team of full-stack engineers, visionary data scientists, and pioneering ML engineers. We are not just redefining cloud authentication—we are engineering the next-gen security platform that continuously evaluates global threats and constructs a comprehensive, dynamic risk . ITP stands as the uncompromising guardian of trust and security, placing decisive risk prioritization at the core of every authentication and authorization decision. Identity Threat Protection with Okta AI The Staff Software Engineer Opportunity Okta is seeking a Staff Software Engineer to be a driving force on the ITP engineering team. This is your chance to dive into the deep end and solve the most critical and complex security challenges facing enterprises today. We seek a candidate who has not just experience, but a proven record of designing, building, launching, and scaling world-class security products —from concept to global deployment. You will be a technical cornerstone of a team that lives and breathes elegant solutions and

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. The Digital Platform Team The Digital Platform team is a global organization distributed across the US and India, responsible for the integration and infrastructure layer that connects Adobe Experience Cloud to the rest of Okta's marketing technology, and analytics stack. The team owns Adobe Experience Platform (AEP) architecture, data collection infrastructure, AEM platform services, and the integration portfolio — including the MCP server architecture, LLM provider integrations, and AI agent infrastructure that power AI-native marketing experiences across the company. The team partners closely with Marketing, IT, DevSecOps, and Adobe to extend platform capabilities, govern data, and deliver scalable AI-driven experiences to the GTM organization. The work is strategic and product-driven: anticipating capability gaps, evaluating new tools and integrations, and building the connective tissue that keeps the broader ecosystem unified and well-governed

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. Okta is searching for a Global Tax Manager to join its Tax Compliance and Reporting team. The ideal candidate is a self-starter with strong organizational, project management, and process improvement skills. The ideal candidate should be proficient in most areas of income tax compliance and reporting encountered by multinational technology corporations, and is excited to leverage AI and tax technology as a core part of how they work. The candidate should be comfortable identifying where AI and automation can improve accuracy, speed, and quality across the tax function. This role is hybrid with an onsite presence at our San Francisco office two days per week. What you will do: Lead the quarterly and annual income tax provision process in accordance with ASC 740, ensuring accuracy, completeness, and audit readiness Manage global income tax compliance, including preparation, review, and coordination with external advisors for U.S. federal, state, and international filings Actively use AI tools (e.g., generative AI for research, drafting, and analysis) and tax technology platforms as part of day-to-day compliance and provision work, and help evaluate new tools to bring into the team's workflow Identify and help implement opportunities to automate manual, repetitive tax processes, with a focus on improving efficiency and reducing reliance on outside providers Design, maintain, and improve SOX controls and documentation Mentor and develop Okta senior tax an

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. Auth0 is an easy-to-implement authentication and authorization platform designed by developers for developers. We make applications’ login boxes safe, secure, and seamless for anyone logging in. As a Developer Platform Partner (DPP) Marketing Manager , you will be responsible for sourcing, building and maintaining strong relationships, taking those partnerships from 0-1 with product, and co-marketing campaigns with leading developer platforms like Vercel, Netlify, Repplit, Lovable, Cloudflare and others. You will help ensure Auth0 shows up in the ecosystems where developers already work and learn, by driving joint campaigns, integrations, and events. You’ll work cross-functionally with Product, Engineering, Developer Relations, Startups and Marketing teams, as well as with external partner teams. This role is ideal for someone who deeply understands the AI dev tools landscape, thrives on building successful partnerships, and is passionate about creating technical marketing that resonates with developers and the community. Key Responsibilities Proactively identify, source, and cold-outreach high-potential developer platform partners (Replit, Lovable, Vercel, Bolt, Builder.io ) and serve as their primary point of contact for all Developer Marketing activities at Auth0. Evaluate how prospective partners fit our integration ecosystem and strategize how the partnership drives developer signups and activation. Influence internally and externally without formal au

Become a part of our caring community The Senior Process Improvement Professional analyzes and measures the effectiveness of existing business processes and develops sustainable, repeatable and quantifiable business process improvements. The Senior Process Improvement Professional work assignments involve moderately complex to complex issues where the analysis of situations or data requires an in-depth evaluation of variable factors. The Senior Process Improvement Professional researches best business practices within and outside the organization to establish benchmark data. Collects and analyzes process data to initiate, develop and recommend business practices and procedures that focus on enhanced safety, increased productivity and reduced cost. Determines how new information technologies can support re-engineering business May specialize in one or more of the following areas: benchmarking, business process analysis and re-engineering, change management and measurement, and/or process-driven systems requirements. Begins to influence department’s strategy. Makes decisions on moderately complex to complex issues regarding technical approach for project components, and work is performed without direction. Exercises considerable latitude in determining objectives and approaches to assignments. Use your skills to make an impact Required Qualifications The Sr. Process Improvement Professional must meet one, (1) of the following requirements: Licensed Professional (LSW) in the state of Kentucky OR ability to obtain, Master’s degree from an accredited university or college in social work, psychology or related health and human services area with a minimum of one, (1) year professional experience in case management.

Asana is seeking an experienced and empathetic Software Engineer to join our AI Teammates Experience team. Asana’s AI Teammates are fundamentally changing the way teams collaborate in and outside of Asana by giving teams access to AI agents that can take on tasks and collaboratively work with human colleagues to complete them. You’ll be building engaging experiences that enable our customers to work effectively with their AI Teammates. You’ll build the interaction and product surfaces for teams, users, and admins, working closely with foundation teams on prompting systems, evaluation/testing infrastructure, and agent capabilities. We’re looking for an engineer who is passionate about AI, curious to learn about everything from how people interact with agents to prompt engineering. This role can be based in our San Francisco offices with an office-centric hybrid schedule. The standard in-office days are Monday, Tuesday, and Thursday. Most Asanas have the option to work from home on Wednesdays. Working from home on Fridays depends on the type of work you do and the teams with which you partner. If you're interviewing for this role, your recruiter will share more about the in-office requirements. About the team AI Teammates Experience owns the end-to-end user-facing experience of AI Teammates, from first discovery and adoption through ongoing collaboration. We build for three core audiences: teams, admins, and individual users. We help teams create, discover, interact with, and manage their AI Teammates; help admins deploy AI Teammates successfully in their organization; and help users discover and work with their Personal Teammates. About Asana AI Teammates AI teammates are autonomous, specialized AI agents designed to work alongside human teams, handling complex, multi-step workflows like product management, customer service, and sales. They can help teams make progress across workflows around the clock, share context to boost productivity, and act as collaborato

About the Team Safety Systems manages the complete lifecycle of safety efforts for OpenAI’s frontier models, ensuring our models are deployed responsibly and have a positive impact on society. Our work spans diverse research and engineering initiatives—from system-level safeguards and model training to evaluation and red-teaming—all aimed at mitigating misuse, misalignment, and maintaining our high bar for safety. We lead OpenAI's commitment to developing and deploying safe Artificial General Intelligence (AGI), fostering a culture of trust, responsibility, and transparency. Our goal is to continuously learn from deployments, distribute AI’s benefits widely, and ensure that powerful tools remain aligned with human values and safety considerations. Within Safety Systems, the Model Policy team works to ensure that frontier models behave safely and reliably in real-world environments by designing policies that define safe model behavior. Some of our publications include: Safety at every step OpenAI GPT6 System Card OpenAI Model Spec About the Role We’re hiring a Model Policy Manager to shape model behavior for U.S. government use, with a focus on national security applications. You’ll define nuanced policies and translate them into training and evaluation criteria, helping models navigate high-stakes scenarios while preserving their usefulness and capabilities. In this role, you will: Develop model policies that guide safe and useful behavior. Build evaluations, identify policy gaps and model failures, and use findings to improve policies and training. Work with research, engineering, and domain experts to support safe, reliable deployment. You might thrive in this role if you: Bring relevant experience in AI safety, policy, or risk assessment. Have strong judgment and can turn complex safety questions into clear, practical policies. Have the technical fluency to work hands-on with model data and evaluations. Are motivated by OpenAI’s mission and the responsible use of

About the Team Safety Systems manages the complete lifecycle of safety efforts for OpenAI’s frontier models, ensuring our models are deployed responsibly and have a positive impact on society. Our work spans diverse research and engineering initiatives—from system-level safeguards and model training to evaluation and red-teaming—all aimed at mitigating misuse, misalignment, and maintaining our high bar for safety. We lead OpenAI's commitment to developing and deploying safe Artificial General Intelligence (AGI), fostering a culture of trust, responsibility, and transparency. Our goal is to continuously learn from deployments, distribute AI’s benefits widely, and ensure that powerful tools remain aligned with human values and safety considerations. Within Safety Systems, the Model Policy team works to ensure that frontier models behave safely and reliably in real-world environments by designing policies that define safe model behavior. Our relevant publications include: Safety at every step OpenAI GPT6 System Card OpenAI Model Spec GPT-Live ChatGPT Images 2.5 About the Role We are hiring a Model Policy Manager to focus on the safety of multimodal models. In this role, you will shape how OpenAI identifies, evaluates, and addresses risks in multimodal AI models - such as GPT-Live and ChatGPT Images - as well as multimodal capabilities in frontier AI models. This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees. In this role, you will: Design and maintain model policies for audio, image, video, and omni-modal behavior. Translate theories of harm and threat models into behavioral safety policies, evaluation criteria, grading guidance, and safeguards. Identify and analyze safety regressions and failure patterns to identify gaps in existing policies and inform policy iteration. Develop policy artifacts that support model training, evaluation, and deployment, including behavior i

About Us Pearl is AI for professional services at global scale, combining advanced AI with verified human expertise to deliver help that is accurate, accountable, and fast. Since 2003, our network has connected millions of customers with licensed professionals across 196 countries, making real expertise available anytime, anywhere. Our Values Data driven: Start with truth, measure what matters. Courageous: Bias to action; run toward hard problems. Innovative: Seek novel, elegant solutions. Lean: Do more with less. Build, ship, learn fast. Humble: Strong opinions, lightly held. About the Role We are looking for a Senior Analyst – Digital Marketing to join our International Team. In this role, you will serve as the analytical backbone of our marketing efforts, delivering data-driven insights that optimize campaign performance across Google Search and Display. Your expertise will help shape bidding strategies, targeting, creative testing, and overall ROI improvement. This position offers a clear growth path into a PPC Account Manager role, where you will take ownership of campaigns based on your performance. What You’ll Do Own the execution and performance of PPC campaigns, driving optimization strategies and ensuring measurable business impact. Analyze campaign performance data using SQL, Tableau, Google Analytics, and Looker Studio to identify opportunities, generate insights, and influence strategic decisions. Lead the design, execution, and evaluation of A/B and multivariate testing programs, translating findings into scalable improvements and measurable outcomes. Develop and apply advanced segmentation and clustering methodologies to improve audience targeting, campaign efficiency, and growth opportunities. Leverage AI-driven tools and predictive models to optimize creative assets, keyword strategies, and prioritization frameworks that improve performance at scale. Drive cross-functional alignment with marketing, product, analytics, and business stakeholders

About Us Pearl is AI for professional services at global scale, combining advanced AI with verified human expertise to deliver help that is accurate, accountable, and fast. Since 2003, our network has connected millions of customers with licensed professionals across 196 countries, making real expertise available anytime, anywhere. Our Values Data driven: Start with truth, measure what matters. Courageous: Bias to action; run toward hard problems. Innovative: Seek novel, elegant solutions. Lean: Do more with less. Build, ship, learn fast. Humble: Strong opinions, lightly held. About the Role As a Senior Business Systems Analyst at Pearl, you'll play a central role in enabling workflow and workforce AI transformation, with a strong emphasis on Agent Hub requirements analysis, platform support, and project coordination. This role blends classic BSA responsibilities with project management and practical AI fluency, supporting both Agent Hub product evolution and broader AI transformation initiatives across the organization. You'll help define and document requirements for complex AI opportunities, translate high-level ideas into actionable work, and coordinate across engineering, prompt engineering, and internal stakeholders. This is a hybrid role designed to support both requirements analysis and execution oversight for high-impact AI operations and transformation projects. You'll be a heavy, daily user of AI tools, not just experimenting on the side, while also contributing directly to the development and improvement of AI tools and features, especially within Agent Hub and related AI transformation initiatives. This is a great opportunity for someone who wants to work at the center of a company's AI transformation, with meaningful ownership across both platform evolution and broader AI operations, and the chance to help shape how AI-enabled work gets defined, prioritized, and operationalized. What You’ll Do Evaluate and document Agent Hub product requirements,

Software is eating the world, but AI is eating software. We live in unprecedented times – AI has the potential to exponentially augment human intelligence. Every person will have a personal tutor, coach, assistant, personal shopper, travel guide, and therapist throughout life. As the world adjusts to this new reality, leading platform companies are scrambling to build LLMs at billion scale, while large enterprises figure out how to add it to their products. To make them safe, aligned and actually useful, these models need human eval and reinforcement learning through human feedback (RLHF) during pre-training, fine-tuning, and production evaluations. This is the main innovation that’s enabled ChatGPT to get such a large headstart among competition. At Scale, our products include the Generative AI Data Engine, SGP, Donovan, and others that power the most advanced LLMs and generative models in the world through world-class RLHF, human data generation, model evaluation, safety, and alignment. The data we are producing is some of the most important work for how humanity will interact with AI. At the foundation of these products is the Platform Engineering team. In this role, you will support the design and development of shared platforms used across Scale. This includes designing our foundational data platforms and lifecycle, architecting Scale’s core cloud infrastructure and orchestration stack, and redefining how engineers develop, build, test, and deploy software at Scale. You’ll also get widespread exposure to the forefront of the AI race as Scale sees it in enterprises, startups, governments, and large tech companies. You will: Drive the design, and implementation of our foundational platforms and systems, working closely with stakeholders and internal customers to understand and refine requirements. Collaborating with cross-functional teams to define, design, and deliver new features. Proactively identifying opportunities for, and driving improvements to, current p