Our client, a global FMCG organization, is looking to bring on an experienced Data Engineer for a contract role. In this position, you will be instrumental in designing, developing, and maintaining robust data solutions that enable seamless data ingestion, processing, and transformation for our global data landscape. The ideal candidate will have extensive experience with data engineering, a deep understanding of Change Data Capture (CDC), and expertise in ingesting data into data lakes. Knowledge of Microsoft Fabric is highly preferred, although familiarity with Azure Data Factory or Synapse Analytics is also valuable.
Key Responsibilities:
-
Data Ingestion & Integration:
Design, build, and manage efficient data ingestion pipelines to handle structured and unstructured data from various sources into data lakes, ensuring scalability and reliability.
-
Change Data Capture (CDC):
Implement and optimize Change Data Capture processes to support near real-time data integration and analytics requirements.
-
Data Transformation & Processing:
Develop ETL/ELT workflows and data transformation processes to ensure data quality, consistency, and usability across the organization's data platforms.
-
Data Lake Architecture & Management:
Set up, manage, and optimize data lakes on Azure (or other cloud platforms), ensuring compliance with data governance and security standards.
-
Platform Development:
Work with Microsoft Fabric (or Azure Data Factory, Synapse Analytics) to design data pipelines that serve multiple data consumers, supporting batch and streaming data ingestion.
-
Collaboration & Support:
Collaborate closely with data scientists, data analysts, business stakeholders, and other engineering teams to understand data requirements and deliver optimal data solutions.
-
Performance Optimization:
Monitor and optimize data workflows and processes to enhance performance, reduce costs, and ensure scalability.
Key Qualifications:
-
Education:
Bachelor's or Master's degree in Computer Science, Data Engineering, Information Technology, or a related field.
-
Experience:
- 5+ years of experience in data engineering or a similar role within a multinational or complex organizational environment.
- Proven experience with Change Data Capture (CDC) and managing real-time data flows.
-
Technical Skills:
- Data Platform Expertise: Strong preference for experience with Microsoft Fabric; expertise with Azure Data Factory, Synapse Analytics, or similar tools will also be considered.
- Data Lakes: Demonstrated experience in data lake architecture, management, and data ingestion techniques.
- Programming & Scripting: Proficiency in SQL, Python, or other relevant languages for data transformation and automation.
- Cloud Platforms: Strong experience with Azure; familiarity with other cloud environments (AWS, GCP) is a plus.
- ETL/ELT Pipelines: Solid understanding of ETL/ELT processes, with hands-on experience in designing, developing, and managing large-scale data pipelines.
-
Additional Skills:
- Knowledge of data governance, data quality, and data privacy best practices.
- Experience with data visualization tools such as Power BI is a plus.
- Familiarity with DevOps practices and CI/CD pipelines in data engineering is advantageous.
Personal Attributes:
- Strong problem-solving skills with a focus on delivering practical solutions.
- Excellent communication skills, capable of working effectively with both technical and non-technical stakeholders.
- A collaborative team player with a commitment to continuous learning and improvement.
Morgan McKinley is acting as an Employment Agency and references to pay rates are indicative.
BY APPLYING FOR THIS ROLE YOU ARE AGREEING TO OUR TERMS OF SERVICE WHICH TOGETHER WITH OUR PRIVACY STATEMENT GOVERN YOUR USE OF MORGAN MCKINLEY SERVICES.
