Data Engineer

We are seeking a skilled Data Engineer with extensive experience in the Cloudera Data Platform (CDP) to join our dynamic team. The ideal candidate will have over four years of experience in designing, developing, and managing data pipelines, and will be proficient in big data technologies. This role requires a deep understanding of data engineering best practices and a passion for optimizing data flow and collection across a diverse range of sources.

Required Skills and Qualifications:

  • Experience: 4+ years of experience in data engineering, with a strong focus on big data technologies.
  • Cloudera Expertise: Proficient in Cloudera Data Platform (CDP) and its ecosystem, including Hadoop, Spark, HDFS, Hive, Impala, and other relevant tools.
  • Programming Languages: Strong programming skills in Python, Scala, or Java.
  • ETL Tools: Experience with ETL tools and processes.
  • Data Warehousing: Knowledge of data warehousing concepts and experience with data modeling.
  • SQL: Advanced SQL skills for querying and manipulating large datasets.
  • Linux/Unix: Proficiency in Linux/Unix shell scripting.
  • Version Control: Familiarity with version control systems like Git.
  • Problem-Solving: Strong analytical and problem-solving skills.
  • Communication: Excellent verbal and written communication skills, with the ability to explain complex technical concepts to non-technical stakeholders.

 

Preferred Qualifications:

  • Cloud Experience: Experience with cloud platforms such as AWS, Azure, or Google Cloud.
  • Data Streaming: Experience with real-time data streaming technologies like Kafka.
  • DevOps: Familiarity with DevOps practices and tools such as Docker, Kubernetes, and CI/CD pipelines.

Education:

  • Bachelor?s degree in computer science, Information Technology, or a related field.

 

Domain Skill:

Data Engineer

Main Skill:

Hadoop, Spark,Hive,Impala,Scala,Python,Java,Linux

Not sure where to start? Ask IVY.
IVY