GCP Data Engineer

Role: GCP Data Engineer (Mid Level)

Position Overview:

We are seeking a mid-level GCP Data Engineer with 4+ years of experience in ETL, Data Warehousing, and Data Engineering. The ideal candidate will have hands-on experience with GCP tools, solid data analysis skills, and a strong understanding of Data Warehousing principles.

 

Qualifications:

�      4+ years of experience in ETL & Data Warehousing

�      Should have excellent leadership & communication skills

�      Should have experience in developing Data Engineering solutions Airflow, GCP BigQuery, Cloud Storage, Dataflow, Cloud Functions, Pub/Sub, Cloud Run, etc.

�      Should have built solution automations in any of the above ETL tools

�      Should have executed at least 2 GCP Cloud Data Warehousing projects

�      Should have worked at least 2 projects using Agile/SAFe methodology

�      Should Have mid level experience in Pyspark and Teradata

�      Should Have mid level experience in

�      Should have working experience on any DevOps tools like GitHub, Jenkins, Cloud Native, etc & on semi-structured data formats like JSON, Parquet and/or XML files & written complex SQL queries for data analysis and extraction

�      Should have in depth understanding on Data Warehousing, Data Analysis, Data Profiling, Data Quality & Data Mapping

 

Responsibilities:

�      Analyze the different source systems, profile data, understand, document & fix Data Quality issues

�      Gather requirements and business process knowledge in order to transform the data in a way that is geared towards the needs of end users

�      Write complex SQLs to extract & format source data for ETL/data pipeline

�      Create design documents, Source to Target Mapping documents and any supporting documents needed for deployment/migration

�      Design, Develop and Test ETL/Data pipelines

�      Design & build metadata-based frameworks needs for data pipelines

�      Write Unit Test cases, execute Unit Testing and document Unit Test results

�      Deploy ETL/Data pipelines

�      Use DevOps tools to version, push/pull code and deploy across environments

�      Support team during troubleshooting & debugging defects & bug fixes, business requests, environment migrations & other adhoc requests

�      Do production support, enhancements and bug fixes

�      Work with business and technology stakeholders to communicate EDW incidents/problems and manage their expectations

�      Leverage ITIL concepts to circumvent incidents, manage problems and document knowledge

�      Perform data cleaning, transformation, and validation to ensure accuracy and consistency across various data sources

�      Stay current on industry best practices and emerging technologies in data analysis and cloud computing, particularly within the GCP ecosystem

 

Education: B.Tech. /B.E. in Computer Science or related field.

  • Certifications: Google Cloud Professional Data Engineer Certification.
Not sure where to start? Ask IVY.
IVY