At Vanta, our mission is to help businesses earn and prove trust. We believe that security should be monitored and verified continuously, and we empower companies to practice better security and prove it with ease. Vanta has a kind and talented team, and while some have prior security experience, many have been successful at Vanta without it. As a Staff Software Engineer on the Automation - Foundations team, you will lead the re-platforming of the data layer underneath Vanta’s entire compliance product. This is a migration that spans multiple teams, has to preserve every public API contract along the way, and cannot lose a single customer’s evidence while it happens. Foundations is Vanta’s data platform team. We ingest security and compliance data from across our customer’s environments, currently tens of thousands of resources per second, with single-customer bursts running into the millions. We store the data, catalog it, make it queryable, and turn it into evidence that has to survive a real SOC 2 or FedRAMP audit. We are in the middle of moving our platform from a Mongo-centric architecture to a schema-aware, Postgres-backed architecture, on a Kafka and S3 pipeline that decouples data fetching from processing. Both pipelines run in parallel today, the hard problems here are correctness under migration, eventual consistency, and multi-tenancy, in a domain where “mostly right” is not an acceptable failure mode. Visit our Vanta Engineering Blog to learn more about what our team is working on. What you'll do as a Staff Software Engineer at Vanta: Lead the migration of Vanta’s resource data model from a Mongo-centric solution to a schema-aware Postgres-backed solution and running both generations in parallel without breaking a customer integration. Drive solutions across teams that you do not own but are dependent on the platform built by your team. Design for correctness under eventual consistency with idempotent session handling, conditional writes that survive out
