Data Engineer with DataBricks

Overview: We are looking for a Senior Data Engineer with deep expertise in Lakehouse architecture, real-time data streaming, cloud data infrastructure, and microservices development on Azure Kubernetes Service (AKS). You will play a central role in designing and delivering next-generation data pipelines, BI solutions, AI/ML platforms, streaming APIs, and scalable microservices that power Guidepoint's research and analytics products. This is a high-impact, hands-on engineering role. You will work closely with data architects, data scientists, analysts, frontend engineers, QA, and DevOps teams to translate complex business requirements into scalable, reliable, and observable data systems. This is a Hybrid role from our Pune office. What You'll Do: Data Engineering & Lakehouse Design, build, and maintain ETL pipelines, data ingestion workflows, and table schemas on Azure Databricks to support BI, analytics, and AI/ML use cases Architect and optimize the Lakehouse using Delta Lake on Databricks, ensuring reliability, performance, and cost efficiency Build and support data pipelines from business applications such as Salesforce, NetSuite, and other enterprise systems Develop and maintain Knowledge Graph models, entity relationship structures, and NLP-based insight pipelines Maintain data governance, data privacy standards, and compliance best practices throughout the data lifecycle Perform root cause analysis on data and processes to identify opportunities for improvement Collaborate with data architects, scientists, and business consumers to populate and optimize the data warehouse for reporting and analytics Microservices & AKS Development Develop and support scalable web APIs and microservices using Python and Azure Platform Services Build new applications, services, and platforms; optimize existing solutions and refactor legacy components using modern, scalable architec

Who we are Artefact is a leading global consulting firm dedicated to accelerating the adoption of data and AI. We work with a variety of businesses, from supermarket chains, to private equity firms and telecoms; including Nissan, L'Oréal, Carrefour, WHSmith, Orange, Beiersdorf, BNP Paribas, and Samsung. Our success stems from combining advanced data technologies, agile methods for quick delivery, and dedicated teams of data scientists, data engineers, business consultants, and data analysts. Our 2000 employees operate in 26 countries (Americas, Europe, Asia, Middle East, India, Africa) and we partner with 1,000+ clients. What you will be doing As a Senior Engineer in our India office, you will support and develop client data platforms, with a focus on metadata management and data governance using enterprise tools such as Informatica Intelligent Data Management Cloud (IDMC). Your role would include: Platform maintenance: Run and support IDMC day to day, including Secure Agent management, connectivity, access control (asset groups, roles, SSO/SCIM provisioning), and environment promotion. Metadata engineering: Configure and troubleshoot metadata scans across cloud data warehouses, databases, CRM platforms, APIs and reporting tools, and resolve scan and profiling failures at scale. Data quality and governance: Build and maintain data quality rules, scorecards, business glossaries and data classifications, and keep them consistent across sources. Lineage: Map end-to-end data flow across the enterprise, so that data pipelines are visible and trusted. Automation: Use IDMC REST APIs, Python and bulk import/export to automate glossary loads, catalog clean-up, and governance workflows. Qualifications & Experience Informatica Experience: Hands-on experience with Informatica IDMC in production (CDGC, Metadata Command Center or Data Quality), including platform administration, metadata

Who We Are Addepar is a global data and AI platform empowering investment professionals to turn complex financial information into actionable intelligence. Addepar unifies portfolio, market and client data in a total portfolio view and delivers AI-powered insights within investment and client workflows. More than 1,500 firms in 60 countries use Addepar to manage and advise on nearly $10 trillion in assets. Its open platform integrates with 650 software, data and consulting partners to power end-to-end investment operations across firms of all sizes and levels of complexity. Addepar supports clients worldwide with offices in New York City, Salt Lake City, São Paulo, London, Edinburgh, Geneva, Warsaw, Dubai, Pune and Singapore. The Role As a Field Engineer, Team Lead you will serve as a player-coach — guiding a team of field engineers while personally executing the most complex, mission-critical technical work yourself. Operating at the intersection of engineering and client success, you are both the technical anchor for your team and Addepar's technical vanguard in the field, translating complex business requirements into production-grade integrations, automations, and data solutions. You will be the technical expert across Addepar's full solution suite — from API integrations and automation frameworks to advanced data platforms (including our white-labeled Databricks offering, ADX) and AI-driven tooling. At the same time, you will develop your people-management skills: providing daily guidance, coordinating work allocation, and turning the vision set by leadership into reality while keeping your team aligned to our core processes. This role combines a forward-deployed delivery model with hands-on technical depth and functional leadership of a high-performing engineering team. Applicants must have legal authorization to work in the country where this role is based on the first day of employment. Visa sponsorship is not available for this position. Important Note: Thi

About AlphaSense: The world’s most sophisticated companies rely on AlphaSense to remove uncertainty from decision-making. With market intelligence and search built on proven AI, AlphaSense delivers insights that matter from content you can trust. Our universe of public and private content includes equity research, company filings, event transcripts, expert calls, news, trade journals, and clients’ own research content. The acquisition of Tegus by AlphaSense in 2024 advances our shared mission to empower professionals to make smarter decisions through AI-driven market intelligence. Together, AlphaSense and Tegus will accelerate growth, innovation, and content expansion, with complementary product and content capabilities that enable users to unearth even more comprehensive insights from thousands of content sets. Our platform is trusted by over 6,000 enterprise customers, including a majority of the S&P 500. Founded in 2011, AlphaSense is headquartered in New York City with more than 2,000 employees across the globe and offices in the U.S., U.K., Finland, India, Singapore, Canada, and Ireland. Come join us! About AlphaSense: AlphaSense is an equal-opportunity employer. We are committed to a work environment that supports, inspires, and respects all individuals. All employees share in the responsibility for fulfilling AlphaSense’s commitment to equal employment opportunity. AlphaSense does not discriminate against any employee or applicant on the basis of race, color, sex (including pregnancy), national origin, age, religion, marital status, sexual orientation, gender identity, gender expression, military or veteran status, disability, or any other non-merit factor. This policy applies to every aspect of employment at AlphaSense, including recruitment, hiring, training, advancement, and termination. In addition, it is the policy of AlphaSense to provide reasonable accommodation to qualified employees who have protected disabilities to the extent requi

JOB TITLE: Senior Security Data Engineer CLASSIFICATION: Exempt Approved by: Jitendra Patil EXPERIENCE: 5–10 Years Approved Date: REPORTS TO: Shivendra Ashish Revised Dates: POSITIONS SUPERVISED: Core Responsibilities: Pipeline Management: Maintain high-throughput streaming pipelines to ingest logs from various sources (Firewalls, Cloud, Endpoints) to a central destination. Log Normalization: Write parsers to convert raw, messy logs into standard schemas (e.g., OCSF or ECS) for consistent querying. Cost Optimization: Implement routing logic to send "high-value" data to the SIEM and "bulk" data to low-cost Object Storage (Data Lake). Data Preparation: Clean and structure data to enable AI/ML detection models and advanced analytics. Must-Have Skills: Data Engineering: Proficiency in Python (for ETL) and SQL (for complex querying). Streaming Tech: Experience with Message Queues (e.g., Kafka, Pub/Sub) and stream processing concepts. Log Handling: Mastery of Regex and log parsing strategies for standard formats (Syslog, CEF, JSON). Storage Architecture: Understanding of Data Lake principles (Parquet/Avro formats) vs. Data Warehouses. Preferred / Nice to Have: Experience with Vector Databases for storing embeddings. Knowledge of Log Observability/Routing tools (middleware that routes logs). Familiarity with Big Data frameworks (e.g., Spark, Flink). Ability to embrace Clearwater's CLEAR core values (Commitment to Client Success, Lead with Accountability, Integrity & Collaboration, Excellence in All That We Do, Advance Colleague Success, Respect & Transparency) and culture. Disclaimer: The above statements are intended to describe the general nature and level of work being performed by people assigned to this position. They are not to be construed as an exhaustive list of all responsibilities, duties, and skills required of colleagues in the role. All colleagues may be required to perform duties outside of their normal responsibilities fr

About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery. WHY JOIN CAPCO? You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry. MAKE AN IMPACT Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services. #BEYOURSELFATWORK Capco has a tolerant, open culture that values diversity, inclusivity, and creativity. CAREER ADVANCEMENT With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands. DIVERSITY & INCLUSION We believe that diversity of people and perspective gives us a competitive advantage. MAKE AN IMPACT Role: GCP Data Engineer Job Summary: We are looking for an enthusiastic GCP Data Engineer to join our dynamic and growing project team. In this role, you will have the opportunity to develop your skills while contributing to large-scale data projects on Google Cloud Platform. You will work closely with senior engineers and data analysts to build and maintain robust data pipelines, helping to turn raw data into actionable insights. This is an excellent opportunity for a foundational career in data engineering within a supportive and collaborative environment. Key Responsibilities: · Develop, test, and maintain data pipelines and ETL/ELT processes using Python, SQL,