A
ACH Groupvia muse
Data Engineer
Mile End, SA, AUPosted 2w ago
Data EngineerMid LevelFull-time
About the Role
We are looking for a Data Engineer with hands\-on experience in Azure Data Factory to develop, build and maintain data pipelines and analytics infrastructure that support operational efficiencies and informed business decisions. This is a full\-time role is based at our Head Office in Mile End.
**The role:**
* Develop and maintain data architecture and infrastructure that support reliable analytics and reporting
* Build, maintain and monitor Azure Data Factory pipelines to integrate and transform data across systems
* Develop and troubleshoot Apache Spark data processing workloads and support their operation in Kubernetes environments
* Design, develop and optimise Power BI semantic models, including dimensional modelling, data relationships, DAX measures, row\-level security and performance tuning
* Develop and support advanced analytics platforms, collaborating with analytics teams to prepare, validate and transform data for reproducible machine learning (ML) workflows
* Design scalable database schemas and administer, maintain and optimise databases and queries to support dependable data access and use
* Apply data governance standards through metadata and lineage management, access controls, and data validation, reconciliation and quality monitoring to safeguard data integrity
* Partner with stakeholders to deliver practical data solutions that meet business needs
* Contribute to other data engineering activities as required
**About you:**
You translate business needs into reliable, well\-governed data solutions and communicate clearly with non\-technical stakeholders. The requirements below are essential unless marked desirable:
* A degree or postgraduate qualification in computing or a related technical discipline, or equivalent experience
* Experience delivering and supporting reliable production data pipelines integrating complex sources in a large\-scale organisation
* Essential: Azure Data Factory expertise in ETL/ELT orchestration, incremental loads, monitoring, error handling and performance troubleshooting
* Experience with Apache Spark (PySpark or Spark SQL) for distributed data processing, and familiarity with Kubernetes\-based deployments, monitoring and troubleshooting
* Desirable: Microsoft Fabric experience with lakehouses, warehouses, data pipelines or notebooks
* Strong coding and technical skills to develop efficient, maintainable solutions for data transformation, automation and analytics
* Hands\-on experience with modern data engineering tools for data integration, transformation, orchestration and monitoring
* Power BI semantic model development, including dimensional modelling, relationships, DAX, row\-level security and performance tuning
* Understanding of ML concepts and practical data preparation, validation and transformation for reproducible workflows with analytics teams
* Database architecture and data modelling experience, including scalable schemas and database and query optimisation
* Practical data governance experience covering metadata, lineage, access controls and data integrity through validation, reconciliation and quality issue resolution
* Strong analytical and root\-cause investigation skills to evaluate solutions and resolve complex data issues using sound architecture and governance principles
* Strong stakeholder engagement to clarify requirements, explain trade\-offs and collaborate across business and technical teams
* Ability to prioritise competing demands, meet deadlines and own solutions from development to operational support
* Commitment to clear documentation and continuous improvement in data reliability, performance and maintainability
A satisfactory National Criminal History check less than 12 months old at commencement is required for the successful candidate.
**How to Apply:** To apply online press ‘Apply’ now.
ACH Group celebrates and encourages workforce diversity and does not discriminate on the basis of gender, age, race, sexual orien
**The role:**
* Develop and maintain data architecture and infrastructure that support reliable analytics and reporting
* Build, maintain and monitor Azure Data Factory pipelines to integrate and transform data across systems
* Develop and troubleshoot Apache Spark data processing workloads and support their operation in Kubernetes environments
* Design, develop and optimise Power BI semantic models, including dimensional modelling, data relationships, DAX measures, row\-level security and performance tuning
* Develop and support advanced analytics platforms, collaborating with analytics teams to prepare, validate and transform data for reproducible machine learning (ML) workflows
* Design scalable database schemas and administer, maintain and optimise databases and queries to support dependable data access and use
* Apply data governance standards through metadata and lineage management, access controls, and data validation, reconciliation and quality monitoring to safeguard data integrity
* Partner with stakeholders to deliver practical data solutions that meet business needs
* Contribute to other data engineering activities as required
**About you:**
You translate business needs into reliable, well\-governed data solutions and communicate clearly with non\-technical stakeholders. The requirements below are essential unless marked desirable:
* A degree or postgraduate qualification in computing or a related technical discipline, or equivalent experience
* Experience delivering and supporting reliable production data pipelines integrating complex sources in a large\-scale organisation
* Essential: Azure Data Factory expertise in ETL/ELT orchestration, incremental loads, monitoring, error handling and performance troubleshooting
* Experience with Apache Spark (PySpark or Spark SQL) for distributed data processing, and familiarity with Kubernetes\-based deployments, monitoring and troubleshooting
* Desirable: Microsoft Fabric experience with lakehouses, warehouses, data pipelines or notebooks
* Strong coding and technical skills to develop efficient, maintainable solutions for data transformation, automation and analytics
* Hands\-on experience with modern data engineering tools for data integration, transformation, orchestration and monitoring
* Power BI semantic model development, including dimensional modelling, relationships, DAX, row\-level security and performance tuning
* Understanding of ML concepts and practical data preparation, validation and transformation for reproducible workflows with analytics teams
* Database architecture and data modelling experience, including scalable schemas and database and query optimisation
* Practical data governance experience covering metadata, lineage, access controls and data integrity through validation, reconciliation and quality issue resolution
* Strong analytical and root\-cause investigation skills to evaluate solutions and resolve complex data issues using sound architecture and governance principles
* Strong stakeholder engagement to clarify requirements, explain trade\-offs and collaborate across business and technical teams
* Ability to prioritise competing demands, meet deadlines and own solutions from development to operational support
* Commitment to clear documentation and continuous improvement in data reliability, performance and maintainability
A satisfactory National Criminal History check less than 12 months old at commencement is required for the successful candidate.
**How to Apply:** To apply online press ‘Apply’ now.
ACH Group celebrates and encourages workforce diversity and does not discriminate on the basis of gender, age, race, sexual orien