Senior - Principal Data Engineer

Job Description

Morgan McKinley is currently working with a client in the Energy sector. You will be part of the Digital Technology Team to innovate and deploy digital products that can create impact on a national sc

Responsibilities:

• Create and maintain multiple robust and high-performance data processing pipeline within Cloud, Private Data Centre and Hybrid data ecosystem
• Assemble large, complex data sets from a wide variety of data sources
• Collaborate with Data Scientist, Machine Learning Engineer, Business Analyst and Business users to derive actionable insights and reliable foresights into customer acquisition, operational efficiency and other key business performance metrics
• Develop, deploy and maintain multiple microservices, rest API and reporting services
• Design and implement internal processes to automate manual workflow, optimize data delivery and redesigning infrastructure for greater scalability, etc
• Establish expertise in designing, analyzing and troubleshooting large-scale distributed systems

Requirements:

• Proficiency in managing extensive data lakes and data warehouses.
• Familiarity with the Hadoop ecosystem and prominent big data tools, encompassing Spark and Kafka.
• Hands-on involvement in stream-processing frameworks, including Spark-Streaming.
• In-depth comprehension of SQL, with the capability to enhance data queries for optimal performance.
• Skillset spanning object-oriented and functional scripting languages like Python, Java, Scala, and more.
• Demonstrated track record of effectively handling, manipulating, and deriving insights from vast and disparate datasets.
• Application of contemporary development methodologies such as Scrum, TDD, continuous integration, and code reviews.
• Utilization of ETL tools like Talend Big Data, Apache Nifi, and others.
• Prior experience with Hortonworks Data Platform or Cloudera Data Platform.
• Familiarity with Metadata Management tools.
• Exposure to Data Governance processes and associated tools.
• Proven adeptness in collaborating with cross-functional teams within a dynamic work environment

Those who are keen for the role and would like to discuss the opportunity further, please click "Apply Now" or email Kin Long at with your updated CV.

Only shortlisted candidates will be responded to, therefore if you do not receive a response within 14 days, please accept this as notification that you have not been shortlisted.

Morgan McKinley Pte Ltd

EA Licence No: 11C5502 | EAP Registration No: R2095054