Human Evaluator Jobs

Become a part of our caring community The Legal Operations Professional collects, organizes, and analyzes law department data, including matter management metrics, outside counsel spend, and operational benchmarks. Your work assignments are often straightforward and of moderate complexity. Job Responsibilities: Help with the administration and maintenance of legal technology platforms such as e-billing, matter management, and knowledge management systems Assist with vendor management coordination, including tracking outside counsel performance, processing invoices, and maintaining vendor records Help prepare reports, dashboards, and presentations on law department metrics, spend trends, and benchmarking data to inform leadership decision-making Conduct financial analysis of law department spend data, including reviewing and evaluating law firm and legal service provider pricing structures Helps create financial models to analyze law firm alternative fee arrangements (AFAs), prompt pay discounts, volume discounts, and other fee structures to identify cost savings and optimize outside counsel spend Support the preparation of law firm and legal service provider pricing reviews, including rate benchmarking, fee arrangement comparisons, and value assessments to inform negotiation strategies Work is managed and often guided by precedent and/or documented procedures/regulations/professional standards with some interpretation. Use your skills to make an impact Required Qualifications Bachelor's degree 1+ years experience working in legal operations, legal support, knowledge management, or a related professional services environment Experience collecting, organizing, and analyzing data to support operational reporting and decision-making

Become a part of our caring community The Care Coaching Assistant 2 (Transition Coordination Support Staff) employs a variety of strategies and techniques to support a member's wellness state by coordinating services and resources. The Care Coaching Assistant may support multiple responsibilities including Transitions. You/applicant/employee typically focuses decisions on interpretation of area/department policy and methods for completing assignments. Work within defined parameters to identify work expectations and quality standards, but has some latitude over prioritization/timing, and works. Follow standard practices that allow for some opportunity for interpretation/deviation or independent. Position Responsibilities: Contribute to administration of care coordination, for transitions and custodial prevention teams. Provide non-clinical support to the assessment and evaluation of members' needs and requirements. This support helps achieve and/or maintain an optimal wellness state by guiding members/families toward resources appropriate for their care and wellbeing. Additionally, it facilitates interaction with these resources. Perform varied activities and moderately complex administrative/operational/customer support assignments. Perform computations. Typically work on semi-routine assignments. Use your skills to make an impact Required Qualifications Less than 3 years of technical experience Must reside in the State of Indiana Proficiency in Microsoft Word, Excel and Outlook Excellent verbal and written communication skills Must Reside in the State of Indiana Preferred Qualifications Clerical support background in a healthcare environment Associate or Bachelor's Degree </

Become a part of our caring community The Senior Full Stack Engineer Performs software engineering activities in all layers of the stack, from setting up the database to programming in the back-end and the appearance at the front-end. The Senior Full Stack Engineer work assignments involve moderately complex to complex issues where the analysis of situations or data requires an in-depth evaluation of variable factors. As Centerwell builds its AI engineering function from the ground up, we need a platform foundation strong enough to support everything that comes next. As Lead Full-Stack Engineer focused on platform and API engineering, you will design and build the service layer that connects AI capabilities, data systems, and product frontends—setting the standards for how services are built, secured, and operated across the team. You will work with meaningful architectural scope, making decisions that span API design, security patterns, and deployment practices. The platform you build will serve care teams and patients across hundreds of Centerwell clinics. If you want to build platforms that others build on—and do it in service of better primary care—this is the role for you. Key Responsibilities Platform and API Architecture: ** Design and lead development of core backend services, REST and GraphQL APIs, and service-to-service integrations that connect all layers of Centerwell's AI product stack. Security and Compliance by Design: ** Establish patterns for authentication, authorization, rate limiting, PHI access control, and audit logging. Ensure HIPAA compliance is embedded in platform design from day one—not bolted on after the fact. AI and LLM Integration Patterns: ** Define and implement reusable patterns for integrating AI capabilities into product

Become a part of our caring community The Strategy Advancement Advisor is a member of Humana’s People Innovation team, which helps shape and advance the future of HR by translating enterprise and workforce priorities into focused strategies and measurable outcomes. This role partners with HR and business leaders to frame complex challenges, develop fact-based recommendations, and move high-priority initiatives from strategy through execution. The Advisor will work across a varied portfolio of enterprise people priorities and transformation efforts, bringing structure to ambiguity, connecting work across functions, and helping leaders make informed decisions that improve business, workforce, and associate outcomes. Humana is seeking an experienced strategy professional to join the People Innovation team. The team serves as a strategic partner to HR and the business, helping identify, shape, and advance high-impact opportunities that support Humana’s people strategy and broader enterprise transformation. Our work spans emerging workforce priorities, talent and organizational strategies, operating model and work redesign, and other cross-functional initiatives that require enterprise perspective, disciplined problem solving, and coordinated execution. As a Strategy Advancement Advisor, you will lead and support a portfolio of complex, high-visibility initiatives from early problem definition through recommendation and activation. You will work with senior leaders, HR partners, subject matter experts, and cross-functional teams to clarify business needs, synthesize internal and external insights, evaluate strategic choices, and translate direction into practical plans. The role requires the ability to move between strategy and execution, build alignment across a matrixed organization, and produce clear, executive-ready recommendations that enable timely decisions and

Become a part of our caring community Help shape practical, responsible AI solutions that improve healthcare experiences and outcomes. Humana’s Enterprise AI organization develops safe, scalable AI solutions across our Insurance and CenterWell businesses. We bring together product managers, data scientists, engineers, policy experts, and business leaders to apply emerging technology to meaningful healthcare challenges. As Associate Director of Applied AI, you will lead teams that design, build, and deploy enterprise AI solutions, with a focus on generative AI and intelligent agents. You will connect technical strategy to business needs, guide responsible delivery in a regulated environment, and help teams turn promising ideas into measurable outcomes for members, patients, and associates. Key Responsibilities Lead and mentor teams developing production-ready AI solutions that improve healthcare delivery, member experiences, and business operations. Help define and execute the roadmap for applied AI initiatives in alignment with enterprise priorities and business needs. Guide the evaluation and adoption of machine learning, generative AI, large language models, multimodal models, and intelligent agent technologies. Oversee scalable APIs, frameworks, data pipelines, retrieval-augmented generation solutions, and agent orchestration capabilities. Partner with product, data science, engineering, architecture, security, and business teams to translate requirements into reliable solutions. Establish standards for AI evaluation, obse

Become a part of our caring community The Actuarial Analyst 2, Analytics/Forecasting supports value-based initiatives within the HealthCare Economics team, partnering across actuarial, finance, analytics, and business teams to develop actionable financial insights and forecasting solutions. This role will play a key part in the design and development of a holistic modeling solution, building new methodologies and frameworks from the ground up to support performance measurement, financial reconciliation, and strategic decision-making across multiple value-based care programs. You work assignments are varied and frequently require interpretation of complex business issues, independent judgment, and the ability to recommend and execute appropriate courses of action with limited guidance. The Actuarial Analyst 2, Analytics/Forecasting will be responsible for analyzing and forecasting claims and revenue trends, developing innovative forecasting methodologies, and translating complex business requirements into scalable analytical solutions. This individual must be able to independently evaluate data, identify gaps, propose holistic approaches, and develop end-to-end solutions that integrate financial, operational, and claims-based insights. Responsibilities include designing forecasting methodologies, establishing reporting metrics, validating model results, and supporting the evolution of value-based care analytics. You will understand department, segment, and organizational strategy and operating objectives, including their linkages to related areas. You will make decisions regarding own work methods, occasionally in ambiguous situations, and requires minimal direction and receives guidance where needed. Use your skills to make an impact Required Qualifications Bachelor's Degree Successful completion of

Become a part of our caring community The Director, Provider Engagement leads a team of Provider Engagement associates responsible for developing and strengthening positive, long-term relationships with contracted physicians, provider groups, health systems, and ancillary providers across Humana's Medicare Advantage network. You drive provider engagement strategies focused on improving provider experience, quality outcomes, Star Ratings performance, value-based care adoption, risk adjustment performance, and operational excellence. You will report to the VP, Network Performance. You will serve as a key market leader, representing Provider Engagement across cross-functional initiatives and ensuring alignment between provider needs and Humana's strategic priorities. Lead, develop, and support a team of up to 13 Provider Engagement associates responsible for managing relationships with contracted providers and healthcare organizations. Establish team strategy, operating processes, and performance expectations aligned with organizational objectives. Conduct regular performance evaluations and provide coaching, mentoring, and professional development opportunities. Foster a culture of accountability, collaboration, continuous improvement, and provider-centric engagement. Monitor performance against established key performance indicators and organizational goals. Build and sustain strong relationships with value-based provider partners. Represent Humana at key provider meetings, association events, market forums, and strategic provider discussions. Develop and execute provider engagement strategies that support Medicare Star Ratings improvement and quality performance objectives. Partner with provider organizations to drive performance within value-based payment arrangements and alternative payment mode

Become a part of our caring community The Senior Full Stack Engineer Performs software engineering activities in all layers of the stack, from setting up the database to programming in the back-end and the appearance at the front-end. The Senior Full Stack Engineer works on problems of diverse scope and complexity ranging from moderate to substantial. You will report to the Associate Director of Software Engineering. The Senior Full Stack Engineer is involved in all stages of software development. This includes front-end development, back-end development, database integrations, network and hosting management, user interface, user experience, and back-end server management. You are experienced with Cloud-based services and development end-to-end. You can work with autonomy while collaborating with others on architectural and process decisions. You are comfortable with giving and receiving feedback from your teammates and others in the organization. You conceptualize what you need to do from incomplete and evolving requirements. Experience communicating updates and resolutions to customers and other partners. On any given day you may: Contribute to technical architecture reviews and design as well as present vision Expertise using advanced data structures and algorithms Possess the ability to evaluate problems in both strategic and tactical terms Mentor and assist in coaching a small team of high-caliber engineers Drive positive change with technical vision and innovative solutions Strong affinity to software-driven engineering & automation Partner with the business and stakeholders to define technical direction & work cross functionally Deliver business-critical systems and compon

Become a part of our caring community The Lead Software Engineer codes software applications based on business requirements. The Lead Software Engineer works on problems of diverse scope and complexity ranging from moderate to substantial. The Lead Software Engineer standardizes the quality assurance procedure for software. Oversees testing and debugging and develops fixes. Researches complaints and makes necessary adjustments and/or recommendations to resolve complex software related issues. Advises executives to develop functional strategies (often segment specific) on matters of significance. Exercises independent judgment and decision making on complex issues regarding job duties and related tasks, and works under minimal supervision, Uses independent judgment requiring analysis of variable factors and determining the best course of action. Key Responsibilities Technical Architecture and Ownership:** Design and own the end-to-end architecture of Centerwell's AI systems, including LLM-powered clinical tools, RAG pipelines, harnesses, agent-based workflows, and intelligent automation. Make and communicate foundational technical decisions in close collaboration with the broader engineering team. Model Development and Fine-Tuning:** Evaluate, select, and where appropriate guide the fine-tuning of foundation models. Establish model evaluation frameworks that prioritize safety, accuracy, and clinical relevance. Clinical and Product Partnership:** Collaborate closely with product managers, designers, clinicians, and data stakeholders to understand care delivery workflows and translate them into well-scoped, high-impact AI features. HIPAA Compliance and Responsible AI:** Ensure all AI systems are designed, deployed, and monitored in compliance with HIPAA and Humana's Responsible AI standards, including participation i

About Scale AI Scale AI is the data foundation for AI, helping organizations build and deploy reliable production AI applications. We partner with the world's leading enterprises and government organizations to accelerate their AI transformation through frontier AI systems that solve real business problems. Every day, we work with organizations across finance, healthcare, manufacturing, media and telecommunications to build production AI agents that automate complex workflows, help humans, reason over enterprise knowledge, and operate safely at scale. The Opportunity Applied AI is moving faster than ever. New foundation models, reasoning techniques, agent architectures, and research papers emerge every week. Yet building AI systems that reliably solve real-world problems remains one of the hardest engineering challenges. As a Frontier Agent Engineer (Applied AI) , you'll bridge the gap between cutting-edge AI research and production deployment. You'll work directly with enterprise customers to design, evaluate, and deploy intelligent systems that combine frontier models with structured knowledge, retrieval, traditional machine learning, and enterprise software. Unlike traditional ML roles that focus on a single model or product, you'll work across a diverse portfolio of AI challenges spanning multiple industries and use cases. You may build a multi-agent research system and then participate in designing a customer intelligence platform, a healthcare copilot, or an autonomous workflow for a Fortune 100 company. If you enjoy reading new AI papers, experimenting with the latest models, and shipping production systems that create measurable business impact, you'll fit right in. What You'll Build Frontier AI Systems Design and deploy production AI agents that leverage the latest advances in large language models, reasoning, retrieval, memory, and tool use. Architect intelligent systems that combine LLMs, traditional machine learning, structured knowledge, enterprise data

About Scale AI Scale AI is the data foundation for AI, helping organizations build and deploy reliable production AI applications. We partner with the world's leading enterprises and government organizations to accelerate their AI transformation through frontier AI systems that solve real business problems. Every day, we work with organizations across finance, healthcare, manufacturing, media and telecommunications to build production AI agents that automate complex workflows, help humans, reason over enterprise knowledge, and operate safely at scale. The Opportunity Applied AI is moving faster than ever. New foundation models, reasoning techniques, agent architectures, and research papers emerge every week. Yet building AI systems that reliably solve real-world problems remains one of the hardest engineering challenges. As a Senior Frontier Agent Engineer (Applied AI) , you'll bridge the gap between cutting-edge AI research and production deployment. You'll work directly with enterprise customers to design, evaluate, and deploy intelligent systems that combine frontier models with structured knowledge, retrieval, traditional machine learning, and enterprise software. Unlike traditional ML roles that focus on a single model or product, you'll work across a diverse portfolio of AI challenges spanning multiple industries and use cases. You may build a multi-agent research system and then participate in designing a customer intelligence platform, a healthcare copilot, or an autonomous workflow for a Fortune 100 company. If you enjoy reading new AI papers, experimenting with the latest models, and shipping production systems that create measurable business impact, you'll fit right in. What You'll Build Frontier AI Systems Design and deploy production AI agents that leverage the latest advances in large language models, reasoning, retrieval, memory, and tool use. Architect intelligent systems that combine LLMs, traditional machine learning, structured knowledge, enterpri

Do work that matters. At AlertMedia, we help organizations protect their people, operations, and brand. Our modern Risk Intelligence and Response platform empowers teams to detect emerging threats, assess impact, and respond with confidence. We believe building resilience should be simpler—and it starts with bringing critical information and workflows together in one unified platform. Our core values drive us in our important mission of keeping people safe & informed: We’re humans not robots Customers always come first We work better together Simplicity is our strength Our reputation is priceless Hard work pays off As one of the fastest-growing software companies in the country, AlertMedia is looking for a Sr. Software Engineer with full-stack experience. This role is on our AI Agent and Orchestration team, building the AI Assistant and Orchestration products and the agents and platform capabilities that power them. As an AI-forward company, we actively use modern AI tools such as Claude, Codex, and other emerging technologies to help our engineers work smarter, move faster, and build better products. Who you are: You thrive in a collaborative engineering environment that plays to everyone's strengths with a willingness to work across the entire stack. You’ve developed scalable production web applications in Python/Django/Next.js/React . You understand how LLM applications work — context, tool calling, evaluation, and the ways they fail. You care about the quality of your work, the impactful product you help build, and the team you help build it with. You have AWS (or other cloud) experience and are willing to learn from people with both more and less experience than you. You are a user of AI-assisted development tools that improve engineering workflows, code quality, and team productivity. What you get to do every day: Build and ship features

About the Team Models are becoming increasingly capable—moving from tools that assist humans to agents that can plan, execute, and adapt in the real world. Mitigating the frontier risks resulting from these capabilities is paramount to OpenAI’s ability to continue deploying models safely. The Preparedness team is dedicated to addressing these critical risks. Our work includes: Measurement. Monitoring and predicting the evolving capabilities of frontier AI systems. Mitigation. Keeping misuse and misalignment safeguards, alignment tools, and security measures on track to adequately address extreme threats that might arise in the future. Coordination. Setting mitigation targets by maintaining OpenAI’s preparedness framework , and partnering with other staff to achieve these targets. This is urgent, fast-paced work that has far-reaching implications for the company and for society. About the Role We are seeking exceptional researchers who can push the frontier of safety mitigations. You will help derisk frontier models by developing novel safety mitigations, developing and applying new techniques from domains like interpretability, control, and alignment to ensure the safety of OpenAI’s deployed models. You will play a critical role in defining how a safe AI system should look in the future at OpenAI, making a significant impact on our mission to build and deploy safe AGI. This role requires strong technical depth and close cross-functional collaboration to ensure our safety mitigations are enforceable, scalable, and effective. We seek researchers who can partner with experts across domains such as misalignment, cybersecurity, and biology in order to develop the best possible end-to-end safety stack. In this role, you will: Work on identifying emerging AI safety risks and new methodologies for exploring and mitigating the impact of such risks Build (and then continuously refine) the evaluations that enable us to assess the extent of these risks; this might include worki

Scale Labs, Research Scientist — Agent Robustness As the leading data and evaluation partner for frontier AI companies, Scale plays an integral role in understanding the capabilities and safeguarding AI models and systems. Building on this expertise, Scale Labs has launched a new team focused on policy research, to bridge the gap between AI research and global policymakers to make informed, scientific decisions about AI risks and capabilities. Our research tackles the hardest problems in agent robustness, AI control protocols, and AI risk evaluations to help governments, industry, and the public understand and mitigate AI risk while maximizing AI adoption. This team collaborates broadly across industry, the public sector, and academia and regularly publishes our findings. We are actively seeking talented researchers to join us in shaping this vision. As a Research Scientist working on Agent Robustness you will work on the fundamental challenges of building AI agents that are safe and aligned with humans. For example, you might: Research the science of AI agent capabilities with a focus on how they relate to safety, risk factors, and methodologies for benchmarking them; Design and build harnesses to test AI agents’ tendency to take harmful actions when pressured to do so by users or tricked into doing so by elements of their environment; Design and build exploits and mitigations for new and unique failure modes that arise as AI agents gain affordances like coding, web browsing, and computer use; Characterize and design mitigations for potential failure modes or broader risks of systems involving multiple interacting AI agents. Ideally you’d have: Commitment to our mission of promoting safe, secure, and trustworthy AI deployments in the industry as frontier AI capabilities continue to advance. Practical experience conducting technical research collaboratively. You should be comfortable building and leveraging agent scaffolding, designing evaluation harnesses, an

About the Team OpenAI's research training infrastructure powers how our frontier models are trained and evaluated. The Simulation team sits at the intersection between the agentic harness that powers OpenAI's products and the research infrastructure where GPT-next is trained, ensuring that our model's training environment is as realistic as possible. This team owns the integration layer that connects our production harness capabilities into the training stack. The work is highly cross-functional and high leverage: researchers depend on it to run experiments and evaluations reliably as well as to develop the next generation of harness capabilities. Failures in this surface can materially affect training velocity and correctness. About the Role We're looking for a Principal Software Engineer to lead the architecture and evolution of the Simulation Platform. You'll own a critical interface between research and engineering, building the systems, APIs, and operational patterns that let researchers use agentic coding infrastructure safely and effectively in training environments. This role is ideal for a senior backend or infrastructure engineer with strong technical judgment, product sense for highly technical users, and the ability to drive execution across multiple teams. The highest-leverage work is building robust infrastructure that supports and accelerates research without compromising engineering quality. In this role, you will Design, build, and evolve the integration between the Codex harness that powers OpenAI's products and research training infrastructure used for training GPT-next Build a platform for our LLMs to train and be evaluated in simulated environments that mimic their deployment setting as closely as possible, on every axis: agentic harness, compute substrate, timing, tools, data sources, humans in the loop, and more Own major integration surfaces end-to-end, from architecture and API design through rollout, operations, and long-term maintenance Bu