Software Engineer, Agent Optimization & Evaluation

Fanatics Commerce is the global leader in licensed sports merchandise, operating a vertically integrated platform that designs, manufactures, and delivers officially licensed apparel, jerseys, headwear, and collectibles for major leagues, teams, and events worldwide. With more than 900 e-commerce sites and a global omnichannel presence across digital, in-venue, and retail, Fanatics Commerce reaches fans in over 180 countries and powers official fan experiences for many of the world's most iconic sports properties. At Fanatics, we bring our BOLD Leadership Principles to life every day - building championship teams, obsessing over fans, acting with entrepreneurial speed, and delivering with a determined and relentless mindset. About the team The AI/ML Platform pod builds Fanatics' internal AI platform: the paved road for every team building with LLMs, MCPs, and agents. Gateway, agent runtime, registry, retrieval, evaluation, and cost analytics, all running on a petabyte-scale lakehouse with enterprise-grade governance. You'll join a pod that owns this platform end to end. This is a Staff-level role, and a hands-on one. You will choose the architecture, define the standards other teams build to, and stay in the code, shipping the hardest parts yourself. The decisions you make are ones the organization lives with for years. Responsibilities • Own the end-to-end architecture of the AI control plane: gateway, agent runtime, registry, retrieval, evaluation, and cost. Set technical direction, sequence the roadmap, and make the build-versus-buy calls. • Build the enterprise LLM gateway: multi-provider routing, failover, caching, rate limiting, token optimization, and key management, with a path to self-hosted open-weight models where cost, latency, or data residency call for it. • Build the Agent/MCP ecosystem on a managed runtime such as AWS BedrockAgentCore: enterprise systems exposed as governed tools over MCP, a control-plane registry for discovery,

At Snowflake, we are powering the era of the agentic enterprise. To usher in this new era, we seek AI-native thinkers across every function who are energized by the opportunity to reinvent how they work. You don’t just use tools; you possess an innate curiosity, treating AI as a high-trust collaborator that is core to how you solve problems and accelerate your impact. We look for low-ego individuals who thrive in dynamic and fast-moving environments and move with an experimental mindset — who rapidly test emerging capabilities to discover simpler, more powerful ways to deliver results. At Snowflake, your role isn't just to execute a function, but to help redefine the future of how work gets done. Snowflake is about empowering enterprises to achieve their full potential, and people too. With a culture that’s all in on impact, innovation, and collaboration, Snowflake is the sweet spot for building big, moving fast, and taking technology, and careers, to the next level. About the Role The Cortex Code team is building the future of coding agents for working with data. See our flagship product in action: Cortex Code in Action: Live Demos + AMA . Your work will directly impact how developers and businesses build with data. You'll own the full AI engineering lifecycle: design, prompt/tool engineering, evals, deployment, measurement, and optimization. You'll work with a small, high-powered modeling and infrastructure team. What you will do in this role: Own features end-to-end for Snowflake Cortex Code products. Build agentic workflows, coding harnesses, evaluation pipelines. Build enterprise-grade context engineering: function calling, tool schemas, guardrails, agent teams, and verification/repair. Partner with product and infra: translate customer problems into products and experiments. Collaborate with infrastructure teams to productionize improvements. Work with an elite team of engineers towards building great products Requirements: Bachelor’s degree in Computer Scienc

The Team + The Role Our Engineering Team is focused on building AI Products for our product experience (PX) platform. We build from the ground up to explore, prototype, and ship AI-native experiences that change how software teams understand and serve their users. This is not an AI layer added to existing product; it is a deliberate bet on what product intelligence looks like next. The team operates with high autonomy, moves quickly, and builds products without clear precedents. As a Staff Software Engineer (AI), you will sit at the intersection of deep technical capability and strong product judgment. You will design and build production-grade AI systems, including RAG pipelines, agentic workflows, and LLM-powered features, while making clear tradeoffs across prompting, fine-tuning, architecture, evaluation, and deployment. You will also partner closely with product, design, and engineering stakeholders to frame the right problems and communicate technical decisions clearly. This role is based in our New York office. What this looks like day-to-day Applied AI systems: Design and build AI-native systems, including RAG pipelines, agentic workflows, and LLM-powered product features. You will take ideas from prototype through production and ensure they can support real users. Model strategy: Make principled decisions about when to prompt, when to fine-tune, and when to use a different technical approach entirely. You will explain those tradeoffs clearly to engineers and non-engineers. Evaluation and guardrails: Instrument and evaluate model outputs rigorously by defining evaluation frameworks and identifying hallucinations early. You will implement guardrails that hold up under real-world usage and load. Productionize AI ownership: Own model deployment, monitoring, latency optimization, cost management, and reliability at scale. You will ensure AI systems are observable, performant, and production-ready. Full-stack delivery: Contribute across the stack when needed to g

The Team + The Role Our Engineering Team is focused on building AI Products for our product experience (PX) platform. We build from the ground up to explore, prototype, and ship AI-native experiences that change how software teams understand and serve their users. This is not an AI layer added to existing product; it is a deliberate bet on what product intelligence looks like next. The team operates with high autonomy, moves quickly, and builds products without clear precedents. As a Sr. Software Engineer (AI), you will sit at the intersection of deep technical capability and strong product judgment. You will design and ship applied AI systems, including RAG pipelines, agentic workflows, and LLM-powered features, from prototype through production. You will make principled technical decisions, evaluate model behavior rigorously, and communicate tradeoffs clearly to engineers and non-engineers alike. This role is based in our New York office. What this looks like day-to-day Applied AI systems: Design and build AI systems including RAG pipelines, agentic workflows, and LLM-powered features. You will take work from prototype through production and ensure it can hold up in real customer environments. Technical decision-making: Make principled decisions on when to prompt, when to fine-tune, and when to use a different tool entirely. You will explain these tradeoffs clearly so the team can move quickly without sacrificing quality. Model evaluation: Instrument and evaluate model outputs rigorously by defining evaluation frameworks and catching hallucinations early. You will implement guardrails that can withstand real-world load and production use. Productionize AI ownership: Own model deployment, monitoring, latency optimization, cost management, and reliability at scale. You will help ensure AI systems are observable, efficient, and dependable in production. Full-stack product shipping: Contribute across the stack when needed because this team ships products, not just mod

Airbnb was born in 2007 when two hosts welcomed three guests to their San Francisco home, and has since grown to over 5 million hosts who have welcomed over 2 billion guest arrivals in almost every country across the globe. Every day, hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way. Airbnb was born in 2007 when two hosts welcomed three guests to their San Francisco home, and has since grown to over 5 million hosts who have welcomed over 2 billion guest arrivals in almost every country across the globe. Every day, hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way. The Community You Will Join: Machine Learning and Artificial Intelligence are at the heart of the Airbnb product. From Trust to Payments, and from Customer Service to Marketing we rely on ML to ensure that guests and hosts have the best possible experience with Airbnb. The CS AI product team is responsible for driving CSxAI (Customer Support x Artificial Intelligence) initiatives by adopting the Generative AI technologies to enable an intelligent, scalable and exceptional service experience. The team develops and enhances various AI models, ML services and tools including LLM fine-tuning, alignment and optimization, RAG/Search, LLM evaluation and testing automation, feedback-based learning and guardrail for a wide range of applications in Airbnb. What you will do: As a principal machine learning engineer, you will be responsible for fine-tuning state-of-the-art LLMs for diverse use cases while optimizing models for high-performance deployment on Airbnb’s ML Infrastructure. You will partner with product managers, software engineers, data scientists and operation teams to brainstorm, design and develop AI products such as AI Assistant, Autonomous agent, recommendation, travel planning, and many more products that make meaningful

Airbnb was born in 2007 when two hosts welcomed three guests to their San Francisco home, and has since grown to over 5 million hosts who have welcomed over 2 billion guest arrivals in almost every country across the globe. Every day, hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way. The Community You Will Join: Our web and API surfaces handle requests from guests and hosts alongside a growing volume of automated agents: AI assistants, crawlers, and scrapers. We build the systems that bring clarity to this traffic, combining in-house ML and vendor signals to decide in real time how to serve billions of daily requests. Anti-bot and anti-scraping detection is our most adversarial mandate, but the wider challenge is full traffic classification: building evaluation frameworks that tell legitimate automation apart from abusive actors, so high-stakes decisions hold up across the fleet. The Difference You Will Make: You will architect and maintain Airbnb’s end-to-end traffic classification ML systems, balancing high-performance model deployment with rigorous offline data pipelines. Success is measured by your ability to harden edge-traffic policies—targeting reduced bot-incident MTTM—and by establishing rigorous evaluation practices that ensure foundational signal accuracy and evasion-resistance across the fleet. A Typical Day: Own the complete lifecycle of traffic-scoring models, from problem framing to real-time deployment, managing the adversarial feedback loop to ensure high evasion-resistance and directly drive reductions in bot-incident MTTM. Architect robust offline-to-online pipelines that produce certified source-of-truth datasets, establishing rigorous evaluation frameworks—such as stratified benchmarks and leakage-prevention checks—to ensure every model improvement is empirically measurable and defensible. Execute model optimization within strict millisecond latency budgets at the