My client is looking for a Data Engineering Tech Lead with a minimum of 5 years experience to join their organisation based in Cork. This is a permanent role.
Responsibilities:
- Architect and manage modern data platforms, leveraging Databricks (Spark, Delta Lake, MLflow) and AWS services (S3, Glue, Lambda) to deliver robust, scalable pipelines.
- Design and oversee ETL/ELT frameworks for real-time and batch processing, implementing CI/CD practices and infrastructure automation (Terraform or CloudFormation).
- Optimize performance and reliability of data workflows through Spark tuning, cost management, and observability tools such as Datadog or CloudWatch.
- Enforce security and governance standards, applying best practices in IAM, data lineage, and compliance frameworks (e.g., GDPR, ISO).
- Lead and mentor a team of data engineers, fostering technical excellence, agile delivery, and collaboration with cross-functional partners.
Skills & Experience:
- Bachelor's or Master's degree in Computer Science, Engineering, or related field, with 5+ years of experience in data engineering leadership.
- Proven expertise with Databricks and AWS services, including S3, Glue, Lambda, and Step Functions, alongside deep understanding of Spark optimization.
- Proficiency in Python (PySpark), Scala, and SQL, with strong experience in data modeling, warehousing, and integration techniques.
- Hands-on experience with infrastructure as code (Terraform/CloudFormation) and modern CI/CD pipelines (Git-based workflows, Jenkins, etc.).
- Strong leadership, communication, and stakeholder management skills, with the ability to translate complex data architecture concepts into actionable insights
For more information, reach out today!
