Human Evaluator in Canada

Who are we? Cohere is the leading security-first enterprise AI company. We build cutting-edge foundation AI models and end-to-end products that are designed to solve real-world business problems. We’re training and deploying frontier models for enterprises who are building AI systems. We believe that our work is instrumental to the widespread adoption of AI and we are looking for folks that want to be part of that. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. Cohere is a team of researchers, engineers, designers, and more, who are all passionate about their craft. We are a global technology company headquartered in Toronto with key offices in London, New York City, San Francisco, Montreal, Paris, Berlin and Seoul. Join us! Why this role? This role will focus on evaluating coding tasks, requiring you to review and debug code, navigate repository architecture, and analyze model trajectories. Your work will contribute to our model development efforts and the logic our models apply when completing task requests. Please note : This is a part-time independent contractor position available within Canada. We seek candidates who are able to commit to 16 hours per week minimum at a 40 CAD/hour contract rate. This role is BYOD 💻 - Bring Your Own Device (laptop). Remote work within Canada. 12 month contract. Performance incentives included! As a Data Annotation Specialist, you will: Evaluate the model's ability to respond to coding requests, workflows, and code base-related questions using available tools. Assess agent trajectories and model capabilities for code generation and debugging requests. Prompt models to complete complex coding tasks and review the accuracy of generated responses. Label, proofread, and improve machine-written and human-written software engineering-related outputs. Report quality and performance trends related to model/agent behavio

About Scale AI Scale AI is the data foundation for AI, helping organizations build and deploy reliable production AI applications. We partner with the world's leading enterprises and government organizations to accelerate their AI transformation through frontier AI systems that solve real business problems. Every day, we work with organizations across finance, healthcare, manufacturing, media and telecommunications to build production AI agents that automate complex workflows, help humans, reason over enterprise knowledge, and operate safely at scale. The Opportunity Applied AI is moving faster than ever. New foundation models, reasoning techniques, agent architectures, and research papers emerge every week. Yet building AI systems that reliably solve real-world problems remains one of the hardest engineering challenges. As a Frontier Agent Engineer (Applied AI) , you'll bridge the gap between cutting-edge AI research and production deployment. You'll work directly with enterprise customers to design, evaluate, and deploy intelligent systems that combine frontier models with structured knowledge, retrieval, traditional machine learning, and enterprise software. Unlike traditional ML roles that focus on a single model or product, you'll work across a diverse portfolio of AI challenges spanning multiple industries and use cases. You may build a multi-agent research system and then participate in designing a customer intelligence platform, a healthcare copilot, or an autonomous workflow for a Fortune 100 company. If you enjoy reading new AI papers, experimenting with the latest models, and shipping production systems that create measurable business impact, you'll fit right in. What You'll Build Frontier AI Systems Design and deploy production AI agents that leverage the latest advances in large language models, reasoning, retrieval, memory, and tool use. Architect intelligent systems that combine LLMs, traditional machine learning, structured knowledge, enterprise data

About Scale AI Scale AI is the data foundation for AI, helping organizations build and deploy reliable production AI applications. We partner with the world's leading enterprises and government organizations to accelerate their AI transformation through frontier AI systems that solve real business problems. Every day, we work with organizations across finance, healthcare, manufacturing, media and telecommunications to build production AI agents that automate complex workflows, help humans, reason over enterprise knowledge, and operate safely at scale. The Opportunity Applied AI is moving faster than ever. New foundation models, reasoning techniques, agent architectures, and research papers emerge every week. Yet building AI systems that reliably solve real-world problems remains one of the hardest engineering challenges. As a Senior Frontier Agent Engineer (Applied AI) , you'll bridge the gap between cutting-edge AI research and production deployment. You'll work directly with enterprise customers to design, evaluate, and deploy intelligent systems that combine frontier models with structured knowledge, retrieval, traditional machine learning, and enterprise software. Unlike traditional ML roles that focus on a single model or product, you'll work across a diverse portfolio of AI challenges spanning multiple industries and use cases. You may build a multi-agent research system and then participate in designing a customer intelligence platform, a healthcare copilot, or an autonomous workflow for a Fortune 100 company. If you enjoy reading new AI papers, experimenting with the latest models, and shipping production systems that create measurable business impact, you'll fit right in. What You'll Build Frontier AI Systems Design and deploy production AI agents that leverage the latest advances in large language models, reasoning, retrieval, memory, and tool use. Architect intelligent systems that combine LLMs, traditional machine learning, structured knowledge, enterpri

Scale Labs, Research Scientist — Agent Robustness As the leading data and evaluation partner for frontier AI companies, Scale plays an integral role in understanding the capabilities and safeguarding AI models and systems. Building on this expertise, Scale Labs has launched a new team focused on policy research, to bridge the gap between AI research and global policymakers to make informed, scientific decisions about AI risks and capabilities. Our research tackles the hardest problems in agent robustness, AI control protocols, and AI risk evaluations to help governments, industry, and the public understand and mitigate AI risk while maximizing AI adoption. This team collaborates broadly across industry, the public sector, and academia and regularly publishes our findings. We are actively seeking talented researchers to join us in shaping this vision. As a Research Scientist working on Agent Robustness you will work on the fundamental challenges of building AI agents that are safe and aligned with humans. For example, you might: Research the science of AI agent capabilities with a focus on how they relate to safety, risk factors, and methodologies for benchmarking them; Design and build harnesses to test AI agents’ tendency to take harmful actions when pressured to do so by users or tricked into doing so by elements of their environment; Design and build exploits and mitigations for new and unique failure modes that arise as AI agents gain affordances like coding, web browsing, and computer use; Characterize and design mitigations for potential failure modes or broader risks of systems involving multiple interacting AI agents. Ideally you’d have: Commitment to our mission of promoting safe, secure, and trustworthy AI deployments in the industry as frontier AI capabilities continue to advance. Practical experience conducting technical research collaboratively. You should be comfortable building and leveraging agent scaffolding, designing evaluation harnesses, an

Who are we? Cohere is the leading security-first enterprise AI company. We build cutting-edge foundation AI models and end-to-end products that are designed to solve real-world business problems. We’re training and deploying frontier models for enterprises who are building AI systems. We believe that our work is instrumental to the widespread adoption of AI and we are looking for folks that want to be part of that. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. Cohere is a team of researchers, engineers, designers, and more, who are all passionate about their craft. We are a global technology company headquartered in Toronto with key offices in London, New York City, San Francisco, Montreal, Paris, Berlin and Seoul. Join us! Why this role? We are on a mission to build machines that understand the world and make them safely accessible to all. Data quality is foundational to this process. Machines (or Large Language Models, to be exact) learn in similar ways to humans, by way of feedback. By labelling, ranking, auditing, and correcting model output, you will improve Large Language Models' performance for iterations to come, thus having a lasting impact on Cohere's technology. We are hiring Generalist professionals with broad backgrounds that span multiple consumer-facing or personal domains. This is a judgment-driven role, not passive data entry. You will review, assess, and provide structured feedback across a broad and evolving range of tasks, evaluating, stress-testing, and improving our models on English-language data spanning multiple modalities (text, image, and structured formats such as JSON, CSV/TSV, and Markdown). This is a great opportunity for professionals with strong analytical skills to contribute to high-impact annotation projects. Please Note: This is a part-time independent contractor position available within Canada only. We seek ca

About Snorkel At Snorkel, we believe meaningful AI doesn’t start with the model, it starts with the data. We’re on a mission to help enterprises transform expert knowledge into specialized AI at scale. The AI landscape has gone through incredible changes since 2015, when Snorkel started as a research project in the Stanford AI Lab, to the generative AI breakthroughs of today. But one thing has remained constant: the data you use to build AI is the key to achieving differentiation, high performance, and production-ready systems. We work with some of the world’s largest organizations to empower scientists, engineers, financial experts, product creators, journalists, and more to build custom AI with their data faster than ever before. Excited to help us redefine how AI is built? Apply to be the newest Snorkeler! We’re looking for a Research Scientist to advance how high-quality data and environments for AI agents are created. You’ll build and optimize pipelines that combine real-world data, automated generation, and human expert input. Working with domain experts, academic partners, customers, and our product and engineering teams, you’ll scale these pipelines to target frontier model performance gaps and expand data and environment diversity. Your work will amplify human knowledge and judgement, enabling experts to create and refine data and agentic environments that strengthens Snorkel’s position as the frontier data lab. This role is ideal for someone who wants to advance frontier AI through data and environment creation and enjoys turning research into reusable, scalable systems. Location: San Francisco, New York, OR REMOTE Main Responsibilities Design, implement, and optimize reusable pipelines that combine AI capabilities with expert judgment to accelerate data and agentic environment creation. Design and run rigorous experiments to validate proof-of-concept approaches, measure their impact on data quality, pipeline efficiency, and model performance, and communic

About Us At BGBx , we're driven by a simple idea: breakthrough thinking creates breakthrough impact. As an independent commercial solutions partner to pharmaceutical and life science companies, we bring together strategists, scientists, communicators, creatives, technologists, and data experts to tackle some of healthcare's most important challenges. Our teams work across consulting and communications to help clients shape strategy, launch innovations, engage stakeholders, and drive meaningful results throughout the product lifecycle. The work is complex, fast-moving, and deeply connected to improving lives, creating opportunities for curious minds to make a real difference every day. If you're energized by collaboration, inspired by innovation, and motivated by work that matters, you'll find a place to grow, contribute, and make a meaningful impact at BGBx. Talent Acquisition Partner Position Overview The Talent Acquisition Partner at BGBx plays a pivotal role in shaping the organization's future by identifying, attracting, and hiring top talent. This individual will partner closely with hiring managers and departmental leaders across the agency to understand business objectives and develop recruitment strategies that align with the company's mission of delivering innovative healthcare marketing solutions. The ideal candidate is a proactive and resourceful professional with a strong understanding of the healthcare marketing landscape, enabling them to source candidates who can thrive in a dynamic, fast-paced environment. As a trusted advisor, the Talent Acquisition Partner will manage the full-cycle recruitment process, from sourcing and screening candidates to extending offers and ensuring a best-in-class candidate experience. The Talent Acquisition Partner will maintain robust talent pipelines and leverage data-driven insights to optimize hiring outcomes. This is an opportunity to make a significant impact by c

Position Overview We are seeking a transformative Senior Director, Talent Acquisition to lead the overall recruiting function and deliver a high-impact, modern, and scalable hiring experience across the organization. In this role, the successful candidate will partner closely with senior business leaders and HR stakeholders to align hiring priorities, shape workforce plans, and support business growth through innovative, data-driven talent strategies. This leader will oversee end-to-end recruitment strategy and operations, guide team performance, and drive process improvements that strengthen hiring outcomes, scalability, speed, and quality of hire. A key focus of the role will be leading the successful implementation and optimization of Workday Recruiting and related processes, while advancing the use of AI-enabled recruiting tools, automation, talent intelligence, and analytics to improve sourcing effectiveness, decision quality, operational efficiency, and candidate experience. The ideal candidate is an AI-forward talent acquisition leader who brings strong leadership, stakeholder management, strategic thinking, and communication skills, along with deep expertise in recruiting operations, team development, talent analytics, recruiting technology, and enterprise process transformation. They will bring experience leading large-scale systems and process change, including Workday implementation or optimization, as well as a thoughtful approach to the responsible use of AI and technology across the hiring lifecycle. They will also bring a fresh and creative approach to attracting top talent and play a key role in advancing and strengthening our employer brand in the market. Key Responsibilities Lead the global Talent Acquisition function across a broad team or portfolio, with accountability for recruiting strategy, delivery, team capability, technology adoption, and performance against hiring goals. Partner closely with senior business leaders, HR

Product Owner OUR MISSION At Redwood Software we unleash human potential. We empower our customers with lights-out automation for their mission-critical business processes. Redwood Software is the leader in full stack automation for mission-critical business processes. With the first SaaS-based composable automation platform specifically built for ERP, we believe in the transformative power of automation. Our unparalleled solutions empower organizations to orchestrate, manage and monitor their workflows across any application, service or server – in the cloud or on premise – with confidence and control. CORE VALUES One Team. One Redwood Make Your Own Weather Obsess over Customer Success Work the Problem Be Curious Own the Outcome Respect Each Other YOUR IMPACT In the role of Product Owner for the Internal Product line, you will be responsible for driving development of internal tools, working closely with Product Leader, Consultants, Field Teams, Engineering, and QA. We subscribe to the philosophy that Product Managers and Product Owners are effectively CEOs of their product and drive growth and success of the product with the help of engineering, design, marketing, sales etc. The following attributes are key to excel in this role: Thrive in Ambiguity : POs carve the path and change it early and often as they collect more signals, while separating the noise. Strong Leadership: POs drive the day to day processes in managing the roadmap, driving the daily scrum rituals, and driving features/capabilities to on-time delivery. Strong Execution: This entails leading teams that work with you with fast-paced execution, using user research and business objectives to inform decisions, learning from new information and adjusting accordingly, communicating and managing relationships with stakeholders – both internal (your direct team) and external (senior leaders across the company, for example). Key Responsibilities Help PM leaders define the long term

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. Senior Engineering Manager, SDET — Okta Identity Governance (OIG) Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. The Okta Identity Governance Team Okta Identity Governance (OIG) is Okta's Identity Governance and Administration solution, responsible for some of the most critical and visible workflows in enterprise identity: how people request access, how approvers grant it, how entitlements are enforced, and how organizations prove to auditors that only the right people have the right access. OIG is the leading contributor to Okta's new product bookings, and it runs inside some of the largest enterprises in the world — the kind of accounts where a single tenant carries hundreds of thousands of identities across thousands of applications. Quality here is a compliance bar, not just a user experience one. When a governance product has a defect, a company can stay over-provisioned on access it believes it revoked, or hand an auditor evidence that isn't true. The Senior Engineering Manager, SDET Oppo

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. Okta Workflows is the secure, no-code automation platform that empowers organizations to build identity-centric workflows across cloud applications — all without writing code. Our intuitive drag-and-drop interface allows enterprises to automate complex business processes at scale, enhancing productivity, enforcing security, and simplifying IT operations. Customers like Netflix, MGM, and NTT rely on Workflows to automate high-impact identity scenarios with speed and confidence. As we continue to scale, we’re investing in extensibility, developer experience, and performance. Join us to help shape the future of cloud automation and low-code development. Position Description We’re hiring a Staff Full-Stack Engineer to join the Integration Builder team within Okta Workflows. This team owns the core no-code surface that enables both internal teams and third-party developers (ISVs) to build powerful integrations and automation experiences with ease. As a Staff Engineer, you’ll lead initiatives that span front-end and back-end services — delivering performant, secure, and scalable features. You’ll help define architecture, drive implementation, and collaborate closely with Design, PM, and Platform teams. You’ll also work directly with our technical architects to help shape what we build — and how we build it. This is a high-impact role in a growing, strategic product area with strong executive visibility. Role Details: Design, build, and maintain end-to-end feature

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. The Opportunity: Okta Access Gateway (OAG) Enterprises run on a mix of modern cloud services and mission-critical on-premises systems (such as Oracle E-Business Suite, SAP, PeopleSoft, and custom legacy web apps). Okta Access Gateway (OAG) solves the enterprise hybrid cloud challenge by extending Okta’s cloud identity, Adaptive MFA, and Zero Trust security policies to on-premises and legacy applications without requiring custom code changes or traditional VPNs. As the Engineering Manager for Okta Access Gateway in Toronto, you will lead and grow a team of software engineers building the next generation of our hybrid access and gateway infrastructure. You will partner closely with Product Management, Architecture, Security, and Quality teams to deliver high-throughput, mission-critical security software deployed across multi-cloud and enterprise datacenters globally. What You’ll Do People Leadership & Team Growth Lead, mentor, and empower an engineering team, fostering an inclusive, high-performance, and psychologically safe engineering culture. Drive career progression, goal setting, regular 1:1s, and continuous feedback to help engineers grow their technical and leadership skills. Attract, interview, and hire diverse engineering talent to scale Okta’s engineering presence in Toronto. Delivery & Operational Excellence Own the end-to-end execution and delivery of key product roadmap initiatives, balancing feature velocity, technical debt, and softwar

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. The Okta Privileged Access Management (PAM) is an identity-centric approach to a common and critical privileged access use case. Our elegant Zero Trust architecture is purpose-built for the modern cloud and helps customers solve challenging security and operations pain points at scale. We are looking for a software engineer to join our fast-growing team with a focus on scalability, reliability, and enhancing the core building blocks of the product. In this role you will: Be deeply involved in evolving the core architecture of PAM. Work in our product development teams to build scalable, composable components of our platform. Be responsible for designing and implementing scalable architecture patterns. Delight our customers by providing world class UX using our React-based design system Design and build APIs that customers rely on for access to production infrastructure. Work on backend components written in Go and frontend components written in React. You might be a good fit if you: Have 3-5 years of software development experience with a background in Golang or similar programming languages. Proficient in React or similar front-end UI stacks. Experienced working with relational databases like PostgreSQL or similar RDBMS technologies. have the ability to complete a feature end to end from designing database models to backend APIs and frontend UI components. Experienced working with any cloud provider such as AWS, GCP or Azure. Thrive in a collaborativ

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. The Data Enablement Team Auth0 is an easy-to-implement authentication and authorization platform designed by developers for developers. We make access to applications safe, secure, and seamless for the more than 100 million daily logins around the world. Our modern approach to identity enables this Tier 0 global service to deliver convenience, privacy, and security so customers can focus on innovation. Within the Data organization, the Data Enablement team is chartered with creating scalable semantic models that make it safe for anyone to see and understand data. By partnering with product and engineering leaders, this team owns the data lifecycle from strategy to prototyping to implementation, executive dashboard and product level self-service support. The Senior Data Analyst Opportunity Reporting to the Manager, Data Enablement - this role will be responsible for unlocking analytics across the company. You will do this through a combination of data lifecycle ownership, data tooling enablement and outreach programs designed to foster the safe usage of scalable data models for reporting and analytic consumption. You'll serve as a critical data and technical partner that empowers our Sales, TAM, and Customer Success teams with the deep product insights they need to succeed. An average week will find you partnering with these teams to understand their measurement needs, reviewing engineering plans to ensure we capture critical data signals from the Auth

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. The Role We are looking for a Principal Product Manager to lead the strategic evolution of the Auth0 Platform Infrastructure. In this highly visible, high-impact leadership role, you will be the visionary force behind the core infrastructure, consumption models, and platform capabilities that power Auth0’s immense global scale. You will own the multi-year strategy for critical platform pillars, with a heavy emphasis on redefining our rate-limiting and platform consumption frameworks. As a Principal product leader, you will navigate multi-dimensional scaling challenges (including the paradigm shift of autonomous AI agents), influence executive stakeholders, and ensure our platform remains the most resilient, secure, and performant identity solution for the world’s largest enterprises. What You’ll Do Drive the Long-Term Platform & Consumption Strategy: Define and own the multi-year roadmap for Auth0’s core infrastructure. Align deeply technical platform architecture with broader business objectives, Go-To-Market strategies, and overarching revenue goals. Spearhead the Strategic Evolution of Rate Limits & Monetization: Transform rate limits from a protective mechanism into a core business strategy. Pioneer dynamic consumption models, proactive policy-aware observability, and intelligent throttling strategies that align platform costs with customer value, reduce escalations, and unlock new SKUs. Architect Global Cloud Resilience & Scale: Champion ov