Senior Machine Learning Engineer - Perception

The Perception team at Zoox is responsible for the robot’s understanding of the world, fusing data from Lidar, Radar, and Cameras to create a unified representation of the environment. In this role, you will contribute to the development of our next-generation 3D occupancy and segmentation networks. You will architect and optimize high-performance deep learning models that generate dense, temporally consistent voxel representations of the driving environment. This work is critical for enabling our vehicle to navigate complex urban scenarios, handle rare obstacles, and drive safely in tight spaces by providing precise geometry and motion estimates to downstream planners. In this role, you will... Design and implement state-of-the-art multi-modal sensor fusion architectures (Lidar, Camera, Radar) to predict 3D occupancy, semantic segmentation, and flow . Develop "vision-first" fusion strategies to enhance geometric understanding and reduce dependency on sparse sensor modalities . Engineer temporal processing modules to improve the stability and consistency of predictions over time. Optimize model architectures for real-time on-vehicle inference, balancing high-fidelity range extension with strict latency constraints . Collaborate with downstream consumers (Tracking, Prediction, Planner) to refine geometric outputs, such as contours and free-space estimations, for complex maneuvering. Qualifications MS or PhD in Computer Science, Robotics, Machine Learning, or related field with 6+ years of industry experience. Deep expertise in 3D Computer Vision and Deep Learning, specifically with voxel-based or BEV (Bird's Eye View) architectures. Strong proficiency in Python and deep learning frameworks (PyTorch) for model training and design as well as some experience in C++ for model integration. Experienc

This role in the Offline Driving Intelligence team is responsible for developing/learning behavior models for road users such as cars, bicycles, and pedestrians. These agents populate Zoox's simulations and must be indistinguishable from real road users, yet fully controllable: promptable into the rare, adversarial, safety-critical behaviors we need to test against. This means the team’s models directly impact how fast Zoox can train, validate and ship its driving stack. Our team collaborates closely with Planner, Simulation and Validation teams to develop and validate our driving performance. As an ML Agents Machine Learning Engineer, you will work on the bleeding edge of the industry, developing novel machine learning pipelines and models to predict the behavior of other agents in the world and planning the best course of action for the ego vehicle. In this role, you will... Develop new deep learning models that use imitation learning and reinforcement learning to generate driving plans for human-like driving agents. Work on novel techniques to estimate the quality of those driving plans along the dimensions of safety, progress, comfort and realism. Build generative behavior models (e.g. autoregressive, diffusion) that are conditionable on scenario intent — "cut off the ego vehicle," "jaywalk here" — for targeted stress-testing. Leverage our compute, infrastructure and large corpus of data to push boundaries of the field. Develop metrics and tools to analyze errors and understand improvements of our systems. Collaborate with engineers on Perception, Planning, Simulation, and Validation to solve the overall Autonomous Driving problem. Qualifications PhD degree in computer science or related field and 4+ years of relevant professional experience or master's degree and 7+ years of relevant professional experience

The Zoox 3D Sensor Simulation team’s work is at the core of creating in-the-loop simulation and high fidelity synthetic data for training and testing autonomous vehicle AI. We are looking for a machine learning professional with experience applying ML to sensor data, such as LiDAR, radar, and cameras. This role lies at the intersection of machine learning and 3D simulation at Zoox, ranging from learned sensor models to neural 3D reconstruction. By improving the realism of simulated sensors (lidar, radar, and cameras), you’ll enable the Zoox robotics autonomy stack to learn, improve, and enhance the mobility of humans. In this role, you will: Design, train, and validate ML models that improve the realism of simulated lidar, radar, and camera data in autonomous vehicle (AV) simulation pipelines. Collaborate with Synthetic Data and HIL Sim teams to integrate 3DGS and NeRF-based 3D reconstruction with simulation of all sensor modalities. Build and curate training datasets from real and simulated sensor data, including GenAI-based data augmentation. Partner with Sim V&V and Perception engineers to define sensor realism metrics and close the sim-to-real gaps with the greatest impact on autonomy. Mentor engineers and interns, and establish best practices for training and deploying ML on Zoox sensor data. Qualifications: 2+ years of prior experience as a Technical Lead 5+ years of experience developing ML models in Python with PyTorch or TensorFlow <

The Autonomy Behaviors team is responsible for producing our vehicle’s driving trajectory. As a Principal ML Engineer, you will lead the technical development of machine learning algorithms for our next-generation ML Planner. Your impact will range from delivering innovations for immediate milestones to pioneering exploratory projects. You will collaborate closely with teams specializing in Foundation Models, Perception, Simulation, and Safety Validation, in influencing our autonomy stack. Your role will look at problems in a way that crosses team boundaries to prototype new approaches that influence the long term technical direction of multiple organizations within the company. In this role, you will: Set the technical direction of ML development for the future of our Planner Push the frontiers of imitation learning, reinforcement learning, and model scaling to develop new ML models for our onboard Planner Identify and drive force-multipliers for ML development, including metrics, data, and training frameworks Develop new methods to leverage Zoox’s Foundation Models into the ML Planner Provide technical mentorship to the broader group of ML developers at Zoox Collaborate with engineers across Autonomy Software, Simulation, System Design and Mission Assurance, and other departments to solve the overall Autonomous Driving problem in complex urban environments Qualifications BS, MS, or PhD degree in computer science or related field Proven track record of technical leadership in large-scale AI organizations Significant experience with training and deploying Deep Learning models to production Fluency in C++ or Fluency in Python with a basic understanding of C++ Extensive experience with programming and algorithm design-Strong mathematics skills, 10+ years of experience Bonus Qualifications Conference or Journal publications in Machine Learning or Robotics related v

The Prediction & Behavior ML team is responsible for developing machine learning (ML) algorithms that learn and predict behaviors from data, applying them both on-vehicle to influence driving behavior and off-vehicle to provide ML capabilities to simulation and validation. Given the tight integration of behavior forecasting and motion planning, our team collaborates closely with the Planner team to advance overall vehicle behavior. We also work closely with our Perception, Simulation, and Systems Engineering teams to accelerate our ability to validate our driving performance. As a Learned Trajectory Machine Learning Engineer you will be responsible for developing deep learned models that produce trajectories for our vehicles to drive. Given the tight integration of behavior prediction and motion planning, you will closely collaborate with the Planner and Perception teams in the advancement of our overall vehicle behavior. In this role, you will: You will develop new deep learning models that use imitation learning and reinforcement learning to generate driving plans for our autonomous vehicle. You will also work on techniques to estimate the quality of those driving plans along the dimensions of safety, progress, comfort etc. You will leverage our large-scale machine learning infrastructure to discover new solutions and push the boundaries of the field You will develop metrics and tools to analyze errors and understand improvements of our systems You will collaborate with engineers on Perception, Planning, and Simulation to solve the overall Autonomous Driving problem in complex urban environments Qualifications BS, MS, or PhD degree in computer science or related field Experience with training and deploying transformer-based model architectures and reinforcement learning Experience with production Machine Learning pipelines: dataset creation, training frameworks, metrics pipelines Fluency in Pyt

At Zoox, you will collaborate with a team of world-class engineers with diverse backgrounds in areas such as AI, robotics, mechatronics, planning, control, localization, computer vision, rendering, simulation, distributed computing, design, and automated testing. You’ll master new technologies while working on code, algorithms, and research to impact key systems to move Zoox forward. Responsibilities Lead the Sensor Change Playbook workflow, defining the validation process for our next-generation, low-cost vehicle platforms. Partner with PCP V&V to define the Autonomy Technical Specification (ATS) by identifying "bounding" (worst-case) scenarios, particularly those related to winter driving (e.g., heavy snow, ice, sensor occlusion). Work with PCP model owners to architect low-cost stacks that reduce reliance on expensive sensors and accommodate reduced compute (e.g., migrating from 8 GPUs to 1 Thor). Drive the Clearance MoC (Method of Compliance) process for hardware changes, balancing cost, compute constraints, performance and safety. Qualifications Deep expertise in the PCP Stack , multi-modality fusion, and optimizing architectures for severely constrained compute budgets. Experience in V&V (Validation & Verification) for safety-critical systems, specifically expanding into complex Operational Design Domains (ODDs) like winter weather. Proven ability to influence cross-functional stakeholders (SDMA, AHE, PCP) to align on cost vs. performance trade-offs. The ability or experience of managing or curating datasets for sensor validation or model training Bonus Qualifications Experience with Autonomy Technical Specifications (ATS) and defining winter/adverse weather bounding scenarios. Experience working with VLM’s. Ab