N
NielsenIQvia muse
Senior Data Engineer
TN, INPosted 2w ago
Data ScientistSeniorFull-time
About the Role
**Job Description**
As a Senior Data Engineer, you will design, develop, and operate data platform components and data\-intensive products. You will own significant data engineering work from requirements and design through implementation and production support, partnering with Data Engineering, Data Science, Cloud, DevOps, Application, Product, and client\-facing teams to deliver reliable solutions that support business growth.
This is a hands\-on role for an experienced engineer who can solve distributed\-data challenges, contribute to sound technical decisions, support other engineers, and help bring emerging technologies into reliable production capabilities.
Responsibilities
* Design, develop, and maintain data pipelines, platform components, and data\-intensive services.
* Own data engineering work from requirements and technical design through implementation, deployment, and production support.
* Contribute to data acquisition strategies, dataset contracts, acceptance criteria, quality standards, lineage, and lifecycle policies.
* Build scalable batch and real\-time data pipelines, lakehouse architectures, and distributed processing solutions for high\-volume workloads.
* Help evolve the data ecosystem to support growth in data volume, velocity, variety, and product complexity.
* Apply and improve reusable patterns for data ingestion, transformation, storage, orchestration, testing, observability, and delivery.
* Build production systems that meet expectations for reliability, security, performance, scalability, cost efficiency, and SLA\-driven delivery.
* Contribute to operational practices for monitoring, incident response, failure recovery, replay, backfills, schema evolution, and disaster recovery.
* Partner with Data Science and ML teams to productionize ML, AI, and LLM\-based workloads and integrate them into customer\-facing products.
* Collaborate with Cloud, DevOps, Application, Security, Product, and client teams to deliver robust solutions for business and customer needs.
* Evaluate technologies and architectural patterns, contribute to technical proofs of concept, and recommend adoption based on measurable value.
* Document technical decisions, trade\-offs, risks, and implementation approaches.
* Participate in technical reviews and help improve design, code quality, testing, maintainability, and production ownership.
* Support engineering roadmaps, hiring, and capability development across the data engineering discipline.
**Qualifications**
* **6\+ years of hands\-on software or data engineering experience, including experience designing, building, and operating production\-grade data systems.**
* Experience contributing to complex data initiatives across teams.
* B.Sc. or M.Sc. in Computer Science, Computer Engineering, or an equivalent technical field, or comparable practical experience.
* Strong understanding of data\-intensive and **distributed systems**, including scalability, reliability, consistency, and performance trade\-offs.
* **Strong proficiency in Python and working proficiency in Scala.**
* **Strong expertise with at least one major cloud platform: AWS, Microsoft Azure, or Google Cloud Platform.**
* Experience with large\-scale data processing technologies such as **Apache Spark and Databricks.**
* Experience designing and operating event\-driven or **streaming systems** using technologies such as Apache Kafka.
* Experience with workflow orchestration and platform tooling such as **Apache Airflow**, including maintainable DAGs and operational business\-critical workflows.
* Advanced SQL skills and strong experience with **data modeling**, query optimization, and data storage systems.
* Hands\-on experience with at least one of the following: data lake or lakehouse architectures; columnar or analytical databases; or NoSQL or distributed database systems.
* Experience owning production\-grade batch and/or streaming pipelines, including data quality, observability, schema evoluti
As a Senior Data Engineer, you will design, develop, and operate data platform components and data\-intensive products. You will own significant data engineering work from requirements and design through implementation and production support, partnering with Data Engineering, Data Science, Cloud, DevOps, Application, Product, and client\-facing teams to deliver reliable solutions that support business growth.
This is a hands\-on role for an experienced engineer who can solve distributed\-data challenges, contribute to sound technical decisions, support other engineers, and help bring emerging technologies into reliable production capabilities.
Responsibilities
* Design, develop, and maintain data pipelines, platform components, and data\-intensive services.
* Own data engineering work from requirements and technical design through implementation, deployment, and production support.
* Contribute to data acquisition strategies, dataset contracts, acceptance criteria, quality standards, lineage, and lifecycle policies.
* Build scalable batch and real\-time data pipelines, lakehouse architectures, and distributed processing solutions for high\-volume workloads.
* Help evolve the data ecosystem to support growth in data volume, velocity, variety, and product complexity.
* Apply and improve reusable patterns for data ingestion, transformation, storage, orchestration, testing, observability, and delivery.
* Build production systems that meet expectations for reliability, security, performance, scalability, cost efficiency, and SLA\-driven delivery.
* Contribute to operational practices for monitoring, incident response, failure recovery, replay, backfills, schema evolution, and disaster recovery.
* Partner with Data Science and ML teams to productionize ML, AI, and LLM\-based workloads and integrate them into customer\-facing products.
* Collaborate with Cloud, DevOps, Application, Security, Product, and client teams to deliver robust solutions for business and customer needs.
* Evaluate technologies and architectural patterns, contribute to technical proofs of concept, and recommend adoption based on measurable value.
* Document technical decisions, trade\-offs, risks, and implementation approaches.
* Participate in technical reviews and help improve design, code quality, testing, maintainability, and production ownership.
* Support engineering roadmaps, hiring, and capability development across the data engineering discipline.
**Qualifications**
* **6\+ years of hands\-on software or data engineering experience, including experience designing, building, and operating production\-grade data systems.**
* Experience contributing to complex data initiatives across teams.
* B.Sc. or M.Sc. in Computer Science, Computer Engineering, or an equivalent technical field, or comparable practical experience.
* Strong understanding of data\-intensive and **distributed systems**, including scalability, reliability, consistency, and performance trade\-offs.
* **Strong proficiency in Python and working proficiency in Scala.**
* **Strong expertise with at least one major cloud platform: AWS, Microsoft Azure, or Google Cloud Platform.**
* Experience with large\-scale data processing technologies such as **Apache Spark and Databricks.**
* Experience designing and operating event\-driven or **streaming systems** using technologies such as Apache Kafka.
* Experience with workflow orchestration and platform tooling such as **Apache Airflow**, including maintainable DAGs and operational business\-critical workflows.
* Advanced SQL skills and strong experience with **data modeling**, query optimization, and data storage systems.
* Hands\-on experience with at least one of the following: data lake or lakehouse architectures; columnar or analytical databases; or NoSQL or distributed database systems.
* Experience owning production\-grade batch and/or streaming pipelines, including data quality, observability, schema evoluti
Similar Jobs
S
Scotiabank• 1mo ago
Software Engineer (AI Engineer)
Toronto, Ontario, CanadaFull-time
E
EY• 1mo ago
EY - GDS Consulting - AIA - Build AI - Senior
Kolkata, West Bengal, IndiaFull-time
C
CapTech• 3w ago
Lead Machine Learning / Data Science Engineer
Denver, COFull-time
C
Citi• 1mo ago
Assistant Vice President Data Science and Gen AI
Pune Division, Maharashtra, IndiaFull-time