Senior Data Engineer - Full Stack

Job summary

We are looking for a Sr. Data Engineer - Full Stack to design and deliver end-to-end data solutions on Databricks — from stakeholder discovery and data ingestion through modeling, APIs, applications, and production operations.This hands-on role combines deep data engineering expertise with strong stakeholder partnership, product thinking, and end-to-end solution ownership. You will work directly with business and technical teams to clarify ambiguous problems, rapidly prototype solutions, and turn validated concepts into secure, scalable, and reliable data products.You will bridge data engineering, analytics, software engineering, and business teams to make trusted data actionable.

 

What you will do

• Partner directly with business and technical stakeholders to understand workflows, define desired outcomes, and translate ambiguous needs into technical requirements and delivery plans.
• Design, build, and maintain end-to-end data products on Databricks, spanning ingestion, Delta Lake storage, transformation, serving, APIs, and user-facing experiences.
• Develop and operate reliable batch, incremental, and streaming pipelines using SQL, Python, PySpark, Kafka, and Databricks-native capabilities.
• Design event-driven and near-real-time data solutions that integrate with operational systems and downstream consumers.
• Build backend services, APIs, and integrations that make governed data available to applications and operational workflows.
• Develop lightweight applications, dashboards, and interfaces in partnership with product, analytics, BI, and user-experience teams.
• Rapidly prototype solutions, validate them through stakeholder feedback, and prepare successful concepts for production use.
• Create scalable data models and curated datasets that support analytics, reporting, AI/ML, and operational decision-making.
• Implement data-quality, security, lineage, and governance controls using Databricks and Unity Catalog.
• Establish automated testing, CI/CD, monitoring, alerting, and documentation across the data-product lifecycle.
• Optimize pipelines, streaming workloads, queries, services, and applications for reliability, performance, scalability, and cost.
• Diagnose and resolve issues across source systems, streaming platforms, pipelines, data models, APIs, applications, and downstream consumers.
• Collaborate with platform and product engineering teams to turn recurring stakeholder needs into reusable capabilities.
• Lead technical design and code reviews, mentor engineers, and help elevate full-stack data-engineering practices.

 

What you will have

• 5+ years of experience in data engineering, software engineering, or a related role, including ownership of production data solutions.
• Strong proficiency in SQL, Python, and PySpark, with experience building reliable, production-grade pipelines and data products.
• Hands-on experience with Databricks, Apache Spark, Delta Lake, and Unity Catalog or comparable data-governance capabilities.
• Experience designing and supporting streaming or near-real-time data pipelines using Kafka, Kinesis, Event Hubs, or similar event-streaming technologies.
• Strong understanding of event-driven architecture, message processing, schema evolution, data consistency, and streaming reliability.
• Experience delivering full-stack solutions that include data pipelines, backend services or APIs, and lightweight user-facing applications.
• Experience building REST APIs, services, and integrations using Python frameworks such as FastAPI, Flask, or comparable technologies.
• Experience with AWS, Azure, or GCP and cloud-native architecture patterns.
• Strong understanding of data modeling, data warehousing, distributed processing, and analytics-friendl