Safety Research Scholar - Human Frontier Labs

Who are we? Cohere is the leading security-first enterprise AI company. We build cutting-edge foundation AI models and end-to-end products that are designed to solve real-world business problems. We’re training and deploying frontier models for enterprises who are building AI systems. We believe that our work is instrumental to the widespread adoption of AI and we are looking for folks that want to be part of that. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. Cohere is a team of researchers, engineers, designers, and more, who are all passionate about their craft. We are a global technology company headquartered in Toronto with key offices in London, New York City, San Francisco, Montreal, Paris, Berlin and Seoul. Join us! Why this role? To have the opportunity to collaborate with Cohere researchers and tools on designing and implementing novel research ideas and shipping state-of-the-art models to production. We have openings in teams covering base model training, retrieval augmented generation, data and evaluation, safety, and finetuning, to name a few; and we are open to receiving intern applications in any research area relating to LLMs to broaden your research connections while obtaining deep experience in a growing AI startup. Please Note: To be eligible for a Research Internship, you must be currently pursuing a PhD in Machine Learning, NLP, or a related discipline. You need to be available for a full-time internship that lasts for 4-6 months. As a Cohere Research Intern, you will: Conduct cutting-edge machine learning research, building and training large language models. Focus on research projects aimed at expanding the frontier of knowledge in language modelling and associate areas such as evaluation, multimodal models, optimisation etc. Disseminate your research results through the production of publications, datasets, and code. Contrib

Who are we? Cohere is the leading security-first enterprise AI company. We build cutting-edge foundation AI models and end-to-end products that are designed to solve real-world business problems. We’re training and deploying frontier models for enterprises who are building AI systems. We believe that our work is instrumental to the widespread adoption of AI and we are looking for folks that want to be part of that. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. Cohere is a team of researchers, engineers, designers, and more, who are all passionate about their craft. We are a global technology company headquartered in Toronto with key offices in London, New York City, San Francisco, Montreal, Paris, Berlin and Seoul. Join us! Why this role? To have the opportunity to collaborate with Cohere researchers and tools on designing and implementing novel research ideas and shipping state-of-the-art models to production. We have openings in teams covering base model training, retrieval augmented generation, data and evaluation, safety, and finetuning, to name a few; and we are open to receiving intern applications in any research area relating to LLMs to broaden your research connections while obtaining deep experience in a growing AI startup. Please Note: To be eligible for a Research Internship, you must be currently pursuing a PhD in Machine Learning, NLP, or a related discipline. You need to be available for a full-time internship that lasts for 4-6 months. As a Cohere Research Intern, you will: Conduct cutting-edge machine learning research, building and training large language models. Focus on research projects aimed at expanding the frontier of knowledge in language modelling and associate areas such as evaluation, multimodal models, optimisation etc. Disseminate your research results through the production of publications, datasets, and code. Contrib

About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems. About the role Anthropic's Safeguards team is seeking a Red Team Engineer to help ensure the safety of our deployed AI systems and products. In this role, you'll take an adversarial approach to uncover vulnerabilities across our product ecosystem before they can be exploited by malicious actors. Your work will span from technical infrastructure vulnerabilities on our products to emergent risks from advanced AI capabilities. While you'll bring best practices from traditional security approaches, the focus is on broader safety implications and novel abuse unique to advanced AI systems and associated products. You'll investigate the full spectrum of potential abuse — from coordinated account manipulation and payment fraud to novel exploitation of product features — and simulate sophisticated threat actors who chain multiple attack vectors to achieve their objectives. Key responsibilities Conduct comprehensive adversarial testing across Anthropic's product surfaces, developing creative attack scenarios that combine multiple exploitation techniques Research and implement novel testing approaches for emerging capabilities, including agent systems, tool use, and new interaction paradigms Design and execute "full kill chain" attacks that emulate real-world threat actors attempting to achieve specific malicious objectives Build and maintain systematic testing methodologies that evaluate every aspect of our systems Develop automated testing frameworks to enable continuous assessment at scale Collaborate with Product, Engineering, and Policy teams to translate findings into concrete improvements Help establish metrics for measuring detection eff

About the Collective The Human Frontier Collective (HFC) ’s mission is to bring the world's best minds to shape the future of AI – because the future of AI depends not only on powerful models, but on the people who teach them to think. We bring together researchers, academics, and domain leaders across 70+ fields – over 90% of members hold doctorates – to shape how frontier AI systems are built, evaluated, and governed. HFC members have since co-authored published work including SciPredict , PropensityBench , and Professional Reasoning Benchmark . Why join the HFC HFC centers on three core pillars: the network itself, participation in selected AI/ML projects, and opportunities to publish with our research team. Join a unique and exclusive expert network: You’ll become a part of an interdisciplinary community, consisting of over 90% doctorates and field leaders representing 70+ fields from the leading institutions. We’re a collective of top innovators and thought leaders committed to advancing frontier AI to power the world’s most important decisions. Participate in AI/ML projects: Beyond the collective network, you’ll be regularly invited to work on high-impact projects with Scale AI and its affiliated lab and platform: building AI safety and policy guardrails, helping models understand real-world deep learning workflows by designing, reviewing, and optimizing PyTorch models, evaluating complex ML code and AI-generated implementations for efficiency and correctness, and more. Co-author selected research publications: Collaborate with Scale’s research team to co-author technical reports and research papers—boosting your academic visibility and professional recognition. Who should apply Background : PhD, or postdoctoral research experience, in Machine Learning, Computer Science, or a related field; or senior industry experience in Machine Learning or a related field. Skills : Strong proficiency in Python and modern ML frameworks (PyTorch, TensorFlo

Airbnb was born in 2007 when two hosts welcomed three guests to their San Francisco home, and has since grown to over 5 million hosts who have welcomed over 2 billion guest arrivals in almost every country across the globe. Every day, hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way. The Community You Will Join: When a significant severe safety incident impacts, or threatens to impact, our community or company, the Community Crisis Management (CCM) team works to coordinate a swift and effective response across the company to mitigate any continued damage. The team is responsible and accountable for immediate decisions, managing the cross-functional response to severe safety crises on the platform, keeping Airbnb leadership properly informed and involved, and for alerting relevant team leaders to any root causes or vulnerabilities that may have been uncovered throughout the response. The Difference You Will Make: As a Community Crisis Investigator, you carry your own caseload and run the day-to-day response, with a defined check-in point for the calls that set precedent or carry external exposure. You assess incidents and bring what you learn back to the team. A Typical Day: Crisis Case Work Assess incoming incidents and recommend whether a case meets the threshold for crisis implementation, drafting summaries concise and comprehensive enough that the decision-maker can act on them without sending them back. Outline and drive the case management actions needed for stakeholders to respond effectively, keeping the response moving inside the incident’s timeline. Research case precedent and consult stakeholders when a case does not fit an established pattern, and bring your recommended option with its reasoning rather than an open question. Prepare concise overviews and next steps for urgent, monitoring, or implemented cases, and note team updates or housekeeping items during regional chec

About Scale AI At Scale AI, our mission is to accelerate the development of AI applications. For 8 years, Scale has been the leading AI data foundry, helping fuel the most exciting advancements in AI, including generative AI, defense applications, and autonomous vehicles. With our recent Series F round, we’re accelerating the abundance of frontier data to pave the road to Artificial General Intelligence (AGI) and building upon our prior model evaluation work with enterprise customers and governments to deepen our capabilities and offerings for public and private evaluations. About Data Engine Our Generative AI Data Engine powers the world’s most advanced LLMs and generative models through world-class RLHF (Reinforcement Learning with Human Feedback), human data generation, model evaluation, safety, and alignment. The data we produce is some of the most critical work for how humanity will interact with AI. About Our FDE Team Generating high-quality data is the core problem our business solves. We aim to make producing and delivering high-quality data seamless and efficient for operators and customers. Our Team is building customer and operator-specific infrastructure to provide high-quality data with low turnaround time. You'll be exposed to the cutting edge of the Generative AI industry while directly interfacing with the leading model-building organizations in the space, including the top AI research labs and government agencies. Join us in shaping the future of Artificial General Intelligence. As a Forward Deployed Engineer, you'll be at the forefront of providing the critical data infrastructure that powers the most advanced AI models, directly influencing how humanity interacts with AI. You will work with the world’s leading AI companies and government agencies to solve their most complex AI data-related problems. Responsibilities: Drive Impact: Directly contribute to the advancement of AI by delivering critical data solutions for leading AI innovators and