V
Vanguardvia muse
Senior Lead Data & AI Engineer
Toronto, ON, CAPosted 2w ago
Data EngineerLeadFull-time
About the Role
**Position Overview**
=====================
Join us in a high\-impact, high\-visibility role where you will lead the design and delivery of enterprise\-scale Data \& AI solutions that power analytics, operational decision\-making, and AI innovation across the organization.
We are seeking an experienced Senior Lead Data \& AI Engineer with 9\-12\+ years of experience in Data Engineering and modern cloud\-based data platforms, with deep expertise in Databricks, Lakehouse architecture, and AWS. This role will be responsible for building scalable data foundations, reusable engineering frameworks, and AI\-ready data products that accelerate business value and enable advanced analytics and Generative AI use cases.
The ideal candidate combines strong hands\-on data engineering expertise with solution architecture, technical leadership, and a passion for modernizing data ecosystems through automation, metadata\-driven engineering, and cloud\-native technologies.
**Key Responsibilities**
========================
**Data Engineering Leadership**
-------------------------------
* Lead the architecture, design, and delivery of enterprise\-scale data engineering solutions leveraging Databricks and AWS.
* Design and implement scalable, resilient, and reusable data pipelines supporting batch, streaming, and event\-driven workloads.
* Drive engineering best practices across data ingestion, transformation, orchestration, testing, monitoring, and deployment.
* Establish reusable frameworks, patterns, and standards to accelerate delivery and improve engineering quality across teams.
**Databricks \& Lakehouse Engineering**
---------------------------------------
* Architect and implement modern Lakehouse solutions utilizing Databricks, Delta Lake, Unity Catalog, and cloud\-native services.
* Develop metadata\-driven and configuration\-based frameworks that standardize ingestion, transformation, quality validation, and deployment processes.
* Design and implement Delta Live Tables (DLT), Lakeflow Pipelines, and Databricks Workflows to automate enterprise data processing.
* Build trusted, governed, and reusable data products that support analytics, reporting, data science, and AI initiatives.
* Optimize platform performance, scalability, reliability, and cost efficiency across large\-scale workloads.
**Data Platform Modernization**
-------------------------------
* Lead migration and modernization initiatives from legacy platforms to cloud\-native data architectures.
* Implement data observability, monitoring, lineage, metadata management, and quality controls throughout the data lifecycle.
* Partner with architecture and platform teams to advance enterprise data platform capabilities and engineering standards.
* Support implementation of CI/CD, Infrastructure\-as\-Code, automated testing, and DevSecOps practices.
**AI \& Advanced Analytics Enablement**
---------------------------------------
* Build AI\-ready data foundations that support machine learning, Generative AI, and intelligent business applications.
* Partner with Data Scientists, AI Engineers, and business teams to operationalize data products and AI solutions.
* Support implementation of vector search, semantic models, retrieval\-augmented generation (RAG), and knowledge\-driven AI capabilities.
* Contribute to AI orchestration and agent\-based solutions where business value can be accelerated through intelligent automation.
* Enable reliable integration between AI applications and enterprise data platforms.
**Governance, Security \& Data Quality**
----------------------------------------
* Implement enterprise\-grade governance using Unity Catalog, metadata repositories, and data quality frameworks.
* Ensure compliance with enterprise security, privacy, and regulatory requirements.
* Design automated controls for data validation, observability, lineage, and operational monitoring.
* Partner with governance organizations to improve trust and accessibility of crit
=====================
Join us in a high\-impact, high\-visibility role where you will lead the design and delivery of enterprise\-scale Data \& AI solutions that power analytics, operational decision\-making, and AI innovation across the organization.
We are seeking an experienced Senior Lead Data \& AI Engineer with 9\-12\+ years of experience in Data Engineering and modern cloud\-based data platforms, with deep expertise in Databricks, Lakehouse architecture, and AWS. This role will be responsible for building scalable data foundations, reusable engineering frameworks, and AI\-ready data products that accelerate business value and enable advanced analytics and Generative AI use cases.
The ideal candidate combines strong hands\-on data engineering expertise with solution architecture, technical leadership, and a passion for modernizing data ecosystems through automation, metadata\-driven engineering, and cloud\-native technologies.
**Key Responsibilities**
========================
**Data Engineering Leadership**
-------------------------------
* Lead the architecture, design, and delivery of enterprise\-scale data engineering solutions leveraging Databricks and AWS.
* Design and implement scalable, resilient, and reusable data pipelines supporting batch, streaming, and event\-driven workloads.
* Drive engineering best practices across data ingestion, transformation, orchestration, testing, monitoring, and deployment.
* Establish reusable frameworks, patterns, and standards to accelerate delivery and improve engineering quality across teams.
**Databricks \& Lakehouse Engineering**
---------------------------------------
* Architect and implement modern Lakehouse solutions utilizing Databricks, Delta Lake, Unity Catalog, and cloud\-native services.
* Develop metadata\-driven and configuration\-based frameworks that standardize ingestion, transformation, quality validation, and deployment processes.
* Design and implement Delta Live Tables (DLT), Lakeflow Pipelines, and Databricks Workflows to automate enterprise data processing.
* Build trusted, governed, and reusable data products that support analytics, reporting, data science, and AI initiatives.
* Optimize platform performance, scalability, reliability, and cost efficiency across large\-scale workloads.
**Data Platform Modernization**
-------------------------------
* Lead migration and modernization initiatives from legacy platforms to cloud\-native data architectures.
* Implement data observability, monitoring, lineage, metadata management, and quality controls throughout the data lifecycle.
* Partner with architecture and platform teams to advance enterprise data platform capabilities and engineering standards.
* Support implementation of CI/CD, Infrastructure\-as\-Code, automated testing, and DevSecOps practices.
**AI \& Advanced Analytics Enablement**
---------------------------------------
* Build AI\-ready data foundations that support machine learning, Generative AI, and intelligent business applications.
* Partner with Data Scientists, AI Engineers, and business teams to operationalize data products and AI solutions.
* Support implementation of vector search, semantic models, retrieval\-augmented generation (RAG), and knowledge\-driven AI capabilities.
* Contribute to AI orchestration and agent\-based solutions where business value can be accelerated through intelligent automation.
* Enable reliable integration between AI applications and enterprise data platforms.
**Governance, Security \& Data Quality**
----------------------------------------
* Implement enterprise\-grade governance using Unity Catalog, metadata repositories, and data quality frameworks.
* Ensure compliance with enterprise security, privacy, and regulatory requirements.
* Design automated controls for data validation, observability, lineage, and operational monitoring.
* Partner with governance organizations to improve trust and accessibility of crit