Human Evaluator in Toronto

At Lyft, our purpose is to serve and connect. We aim to achieve this by cultivating a work environment where all team members belong and have the opportunity to thrive. The Safety and Customer Care (SCC) team at Lyft manages over 1.7 million monthly human and AI interactions and serves as Lyft's primary direct touchpoint with riders and drivers. We handle critical infrastructure that powers both human associates and AI agents to make riders and drivers feel safe and comfortable while riding or driving with Lyft, transforming every support interaction into a moment of genuine connection. Agentic AI is at the center of how we scale that mission. We fine-tune and align open-source models, build AI-powered support agents, and develop end-to-end AI agents for safety case management, systems that reason over complex, high-stakes cases and drive them to resolution. SCC brings together ML, data, backend, and product engineers alongside data scientists and operations partners to transform these systems. As a Machine Learning Engineer on the SCC team, you will fine-tune and align models and build AI Agents that power how riders and drivers get help. Your work spans the full loop: post-training open-source models for our domain, composing them into multi-step agents, and building the evaluation that proves they are safe to ship in a customer-facing, safety-critical setting. Post-train and adapt open-source LLMs for SCC use cases using SFT, LoRA, and preference-tuning methods (RLHF, RLAIF, RLVR). Design and build AI-powered support agents and end-to-end agents for safety case management using LangGraph or equivalent agentic frameworks. Own the evaluation data flywheel, offline and online, that defines what "good" looks like and build benchmarks for the team to hill-climb. Turn interaction feedback into training data and learning signals, closing the data flywheel that continuously improves the models. Responsibilities: Conduct literature review and build post-training fra

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. The Agent Gateway Team The Agent Gateway team owns the identity-aware infrastructure that connects enterprise AI agents to the tools, data, and services their organizations authorize. Every call from Claude, Agentforce, Codex, and internal/homegrown agents to a resource flows through us. We enforce authorization, isolate credentials, mint the right token per target, and produce the audit trails that security teams rely on. We are early in a rapidly evolving space. The standards for agent identity (MCP, OAuth token exchange, DCR) are being built under our feet. Our roadmap includes hardening the data plane for on-premises customer deployments, extending policy semantics beyond tool-level allowlists, adding native support for Agent-to-Agent brokered delegation, and scaling to tenants with thousands of virtual MCP servers. The Senior Software Engineer Opportunity Okta is looking for a Senior Software Engineer to help build the Agent Gateway. You will own features and components across the data and control planes, turning technical designs and product requirements into reliable production systems. Working alongside staff engineers, you will implement token exchange, request routing, credential resolution, and policy evaluation as agent identity specifications evolve. This is a hands-on software development role at the intersection of product, security, and infrastructure. You will ship services that handle agentic traffic reliably and performantly at scale. Wha

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. Position Description: You will proactively construct and own an AI-forward research roadmap for key product pillars at Okta, operating at the intersection of Identity, Security, and Artificial Intelligence. As a Staff-level individual contributor, you will drive both qualitative and quantitative methodologies to define user experiences for non-deterministic AI agents, automated workflows, and enterprise platforms. Additionally, you leverage agentic tools and AI workflows internally to scale qualitative coding, synthesis, and research operations across the organization. In this role, you’ll get to: Drive Product-shaping Research: Lead foundational and evaluative research to inform strategic and tactical product decisions. Execute Balanced Mixed-Methods: Drive qualitative depth (contextual inquiry, cognitive walkthroughs) and quantitative rigor (large-scale surveys, behavioral telemetry, statistical modeling) to evaluate AI products. Pioneer AI Research Operations: Integrate AI tools, prompt engineering, and agentic research workflows into your daily practice to accelerate quality research outputs. Lead Cross-Discipline Collaboration: Facilitate research to drive cross-discipline clarity, executive alignment, and decision-making across complex security and admin workflows. Solve Complex Problems: Bring stakeholders together across multiple teams to scope problems and collaborate on research in ways that nurture a culture of curiosity and learning. Cultivate E

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. The Okta Privileged Access Management (PAM) is an identity-centric approach to a common and critical privileged access use case. Our elegant Zero Trust architecture is purpose-built for the modern cloud and helps customers solve challenging security and operations pain points at scale. We're looking for a Senior level Platform Engineer to join a team of highly skilled and talented team players who are proud of what they own and deliver. Our elite team is fast, creative, and flexible; with a weekly release cycle and individual ownership, we expect great things from our engineers and reward them with stimulating new projects, new technologies, and the chance to have significant equity in a company that is changing the cloud computing landscape forever. What you’ll do Leverage cutting-edge AI pair-programmers and LLMs (such as Copilot and Claude) to accelerate the development of secure, enterprise-grade Privileged Access Management (PAM) products. Work with engineering teams to design, develop and deliver cloud-based infrastructure projects on a modern tech stack (Kubernetes/EKS, RDS, DynamoDB, Kinesis, MKS, Redis, OpenSearch, Docker, Terraform on AWS) Drive evaluation, development, and rollout of microservices Operate, support, and upgrade shared services and frameworks. Scale these as their usage invariably grows along with Okta's business. Evaluate and scale existing systems to meet specialized requirements and support Okta’s future business ne

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. The role At Okta, we believe AI will fundamentally transform how design, research, and product development happen. We are looking for a Senior Design Operations Program Manager, Agentic Design to lead the operationalization and enablement of internal AI agents, plug-ins, and automated workflows across our Design team — reimaging how designers create in the AI era through internal AI agents. In this role, you will bridge the gap between AI tooling capabilities and practical design ops execution. You will partner closely with design, product management and design systems to govern, evaluate, deploy, and scale AI-driven design capabilities — making our 60+ person design org dramatically faster, more consistent, and highly innovative. This is a tactical, execution-focused builder role for an operations leader who is deeply curious about AI tools, understands modern UX workflows, and excels at change management, team enablement, and operationalizing new frameworks. What you'll do AI & Agentic Design Enablement Serve as the operational connective tissue driving AI adoption, evaluation, and workflow integration across Product Design, UX Research, and Content Design. Manage the intake, evaluation, and approval process for design plug-ins, Model Context Protocol (MCP) integrations, and generative AI tools (e.g., custom content design assistants, design-to-code agents). Partner with internal engineering teams to operationalize, ship, and drive adopt

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. Here is the updated summary for the Okta Device Access (ODA) and Okta Verify (OV) space, adapted to match the collaborative, ownership-focused, and candidate-centric tone of your PAM example: In this role, you will work as a trusted cross-functional partner on our Okta Device Access (ODA) and Okta Verify (OV) products, and have the opportunity to influence product and design decisions through your work. You will own the UX roadmap for ODA and OV, working closely with your design, product, and engineering partners to ensure that your work is aligned to the highest priority needs of the endpoint identity area, informing both long and near-term decisions. You’ll provide mentorship and coaching to more junior team members, helping raise the quality bar and impact of the team. ODA and OV’s goal is to develop products that enable Okta customers to secure their workforce right from physical device bootup, providing frictionless, phishing-resistant, and passwordless authentication. In this role, you’ll get to: Grow and develop a research practice aimed at understanding the needs of the OPA and OV teams, collaborate with stakeholders to craft research questions and execute research Scope, prioritize, organize, and manage research initiatives across these teams. Exercise flexibility in research methodology (as appropriate for the product space and timeline) by designing and utilizing generative studies or evaluative research, appropriately. Facilitate cross-dis

About the Team: Tubi's Internal Tools team is at the forefront of AI integration, developing everything from developer resources to production-grade AI for business operations. We are the group responsible for turning AI from an experiment into an operating capability: training, infrastructure, developer agents, and AI-powered business systems. Engineers operate with high ownership and autonomy, collaborating on shared architectural decisions and AI infrastructure. What You'll Do: Own systems end to end — design them, build them, and support them in production. Lead the projects you own: sequence the work, decide what lands first, and set technical direction for the engineers working with you. Sit with the people who use what you build, and turn what you learn there into a system. Design the service boundaries, contracts and schema evolution that let our platforms grow without breaking the teams depending on them. Make our AI systems dependable in production: evaluation harnesses, human approval steps before an agent acts, retries that handle a model returning something unexpected, and cost tracking that tells you what a task costs before you run it. Build what other engineers build on — agent skills, tool and MCP integrations, shared libraries — and raise the bar through code review, design discussion and mentoring. Spot the platform work nobody has asked for yet, make the case for it, and build it. Your Background: 5+ years of professional experience building and operating production systems, from design through production ownership. A system you designed and can walk us through end to end — where its boundaries sit, what constrained it, and what you chose against. Strong programming proficiency in a statically typed language such as Rust, Go, C++, Java, Kotlin, C#, or TypeScript. Production Rust is a plus rather than a requirement. You have owned a service in production: you wrote the runbooks, you knew what it cost, and you were the one paged when it broke. Expe

Join us in building the future of finance. Our mission is to democratize finance for all. An estimated $124 trillion of assets will be inherited by younger generations in the next two decades. The largest transfer of wealth in human history. If you’re ready to be at the epicenter of this historic cultural and financial shift, keep reading. About the team + role We are building an elite team, applying frontier technologies to the world’s biggest financial problems. We’re looking for bold thinkers. Sharp problem-solvers. Builders who are wired to make an impact. Robinhood isn’t a place for complacency, it’s where ambitious people do the best work of their careers. We’re a high-performing, fast-moving team with ethics at the center of everything we do. Expectations are high, and so are the rewards. The Red Team team’s mission is to proactively identify and simulate real-world threats against Robinhood’s platforms, properties, and people. Through red teaming and adversarial simulations, the team evaluates security controls, uncovers vulnerabilities, and helps continuously strengthen Robinhood’s overall security posture in close partnership with Detection & Response, Physical Security, and Engineering. As a Staff Offensive Security Engineer, you will take a hands-on role in designing and executing stealthy adversarial simulations to validate assumptions and uncover gaps in detection and response. You’ll leverage threat modeling, penetration testing, and research-driven techniques to emulate sophisticated attackers, while collaborating cross-functionally to improve defenses and shape more secure systems. This role is based in our Toronto, Canada office(s), with in-person attendance expected at least 3 days per week. At Robinhood, we believe in the power of in-person work to accelerate progress, spark innovation, and strengthen community. Our office experience is intentional, energizing, and designed to fully support high-performing teams. What you’ll

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. Senior Engineering Manager, SDET — Okta Identity Governance (OIG) Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. The Okta Identity Governance Team Okta Identity Governance (OIG) is Okta's Identity Governance and Administration solution, responsible for some of the most critical and visible workflows in enterprise identity: how people request access, how approvers grant it, how entitlements are enforced, and how organizations prove to auditors that only the right people have the right access. OIG is the leading contributor to Okta's new product bookings, and it runs inside some of the largest enterprises in the world — the kind of accounts where a single tenant carries hundreds of thousands of identities across thousands of applications. Quality here is a compliance bar, not just a user experience one. When a governance product has a defect, a company can stay over-provisioned on access it believes it revoked, or hand an auditor evidence that isn't true. The Senior Engineering Manager, SDET Oppo

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. Okta Workflows is the secure, no-code automation platform that empowers organizations to build identity-centric workflows across cloud applications — all without writing code. Our intuitive drag-and-drop interface allows enterprises to automate complex business processes at scale, enhancing productivity, enforcing security, and simplifying IT operations. Customers like Netflix, MGM, and NTT rely on Workflows to automate high-impact identity scenarios with speed and confidence. As we continue to scale, we’re investing in extensibility, developer experience, and performance. Join us to help shape the future of cloud automation and low-code development. Position Description We’re hiring a Staff Full-Stack Engineer to join the Integration Builder team within Okta Workflows. This team owns the core no-code surface that enables both internal teams and third-party developers (ISVs) to build powerful integrations and automation experiences with ease. As a Staff Engineer, you’ll lead initiatives that span front-end and back-end services — delivering performant, secure, and scalable features. You’ll help define architecture, drive implementation, and collaborate closely with Design, PM, and Platform teams. You’ll also work directly with our technical architects to help shape what we build — and how we build it. This is a high-impact role in a growing, strategic product area with strong executive visibility. Role Details: Design, build, and maintain end-to-end feature

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. The Opportunity: Okta Access Gateway (OAG) Enterprises run on a mix of modern cloud services and mission-critical on-premises systems (such as Oracle E-Business Suite, SAP, PeopleSoft, and custom legacy web apps). Okta Access Gateway (OAG) solves the enterprise hybrid cloud challenge by extending Okta’s cloud identity, Adaptive MFA, and Zero Trust security policies to on-premises and legacy applications without requiring custom code changes or traditional VPNs. As the Engineering Manager for Okta Access Gateway in Toronto, you will lead and grow a team of software engineers building the next generation of our hybrid access and gateway infrastructure. You will partner closely with Product Management, Architecture, Security, and Quality teams to deliver high-throughput, mission-critical security software deployed across multi-cloud and enterprise datacenters globally. What You’ll Do People Leadership & Team Growth Lead, mentor, and empower an engineering team, fostering an inclusive, high-performance, and psychologically safe engineering culture. Drive career progression, goal setting, regular 1:1s, and continuous feedback to help engineers grow their technical and leadership skills. Attract, interview, and hire diverse engineering talent to scale Okta’s engineering presence in Toronto. Delivery & Operational Excellence Own the end-to-end execution and delivery of key product roadmap initiatives, balancing feature velocity, technical debt, and softwar

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. The Data Enablement Team Auth0 is an easy-to-implement authentication and authorization platform designed by developers for developers. We make access to applications safe, secure, and seamless for the more than 100 million daily logins around the world. Our modern approach to identity enables this Tier 0 global service to deliver convenience, privacy, and security so customers can focus on innovation. Within the Data organization, the Data Enablement team is chartered with creating scalable semantic models that make it safe for anyone to see and understand data. By partnering with product and engineering leaders, this team owns the data lifecycle from strategy to prototyping to implementation, executive dashboard and product level self-service support. The Senior Data Analyst Opportunity Reporting to the Manager, Data Enablement - this role will be responsible for unlocking analytics across the company. You will do this through a combination of data lifecycle ownership, data tooling enablement and outreach programs designed to foster the safe usage of scalable data models for reporting and analytic consumption. You'll serve as a critical data and technical partner that empowers our Sales, TAM, and Customer Success teams with the deep product insights they need to succeed. An average week will find you partnering with these teams to understand their measurement needs, reviewing engineering plans to ensure we capture critical data signals from the Auth

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. The Okta Privileged Access Management (PAM) is an identity-centric approach to a common and critical privileged access use case. Our elegant Zero Trust architecture is purpose-built for the modern cloud and helps customers solve challenging security and operations pain points at scale. We are looking for a software engineer to join our fast-growing team with a focus on scalability, reliability, and enhancing the core building blocks of the product. In this role you will: Be deeply involved in evolving the core architecture of PAM. Work in our product development teams to build scalable, composable components of our platform. Be responsible for designing and implementing scalable architecture patterns. Delight our customers by providing world class UX using our React-based design system Design and build APIs that customers rely on for access to production infrastructure. Work on backend components written in Go and frontend components written in React. You might be a good fit if you: Have 3-5 years of software development experience with a background in Golang or similar programming languages. Proficient in React or similar front-end UI stacks. Experienced working with relational databases like PostgreSQL or similar RDBMS technologies. have the ability to complete a feature end to end from designing database models to backend APIs and frontend UI components. Experienced working with any cloud provider such as AWS, GCP or Azure. Thrive in a collaborativ

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. The Role We are looking for a Principal Product Manager to lead the strategic evolution of the Auth0 Platform Infrastructure. In this highly visible, high-impact leadership role, you will be the visionary force behind the core infrastructure, consumption models, and platform capabilities that power Auth0’s immense global scale. You will own the multi-year strategy for critical platform pillars, with a heavy emphasis on redefining our rate-limiting and platform consumption frameworks. As a Principal product leader, you will navigate multi-dimensional scaling challenges (including the paradigm shift of autonomous AI agents), influence executive stakeholders, and ensure our platform remains the most resilient, secure, and performant identity solution for the world’s largest enterprises. What You’ll Do Drive the Long-Term Platform & Consumption Strategy: Define and own the multi-year roadmap for Auth0’s core infrastructure. Align deeply technical platform architecture with broader business objectives, Go-To-Market strategies, and overarching revenue goals. Spearhead the Strategic Evolution of Rate Limits & Monetization: Transform rate limits from a protective mechanism into a core business strategy. Pioneer dynamic consumption models, proactive policy-aware observability, and intelligent throttling strategies that align platform costs with customer value, reduce escalations, and unlock new SKUs. Architect Global Cloud Resilience & Scale: Champion ov

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. We are looking for an experienced Staff Software Engineer in Test to join our Identity Management Engineering (IDM) team serving the Privileged Access Team (PAM). This team is passionate about delivering large-scale, mission-critical software in a fast-paced Agile environment. In this role you'll be working with a team of highly-skilled and talented engineers, responsible for delivering sophisticated backend solutions that help Okta reliably operate at large scale and be highly available. As part of the team, you’ll be ensuring projects are completed with the highest quality and reliability using automation at every level for fast, robust and secure releases. Job Duties and Responsibilities: Review requirements and design specs to develop relative test plans and test cases Automate API tests, end-to-end tests, reliability/scale tests Work with engineering management to scope and plan engineering efforts Communicate and document QE plans for scrum teams to review Review application code, identify bug and other areas of weakness, architect tools for future coverage Automate all critical features to maintain zero-debt cadence Release features with solid quality Respond to production issues/alerts and customer issues during on-call rotation Be a strong customer advocate with a strong quality DNA. Requirements: 5+ years of QE experience preferably in an enterprise SaaS company 3+ years experience in quality engineering for enterprise level software. 5+ yea