Data Engineer Lead

Roles and Responsibilities:

  • Analyze source and target Salesforce instances to understand data structures and migration requirements.
  • Develop and optimize ETL processes using tools like Informatica or Talend
  • Map data fields between source and target systems, implementing necessary transformations.
  • Execute data cleansing, deduplication, and validation processes to ensure data quality.
  • Perform incremental and full data loads using Salesforce Data Loader or APIs (REST, Bulk).
  • Enable and leverage Change Data Capture (CDC) or similar mechanisms for incremental extraction.
  • Troubleshoot and resolve migration-related issues, ensuring performance and data consistency.
  • Collaborate with stakeholders to define migration strategies and validation criteria.
  • Document data mappings, transformation rules, migration processes, and results.
  • Assist in deployment activities, including cutover execution and post-migration support.

Critical Skills to Possess:

Expert level Snowflake knowledge and Expert level SQL, SOQL and Exposure to DBT and data querying languages.

? Work Experience in CI-CD Pipeline 

? Any one Orchestration tool (Tidel, AirFlow or Control M)

? Hands-on experience with Salesforce APIs (SOAP, REST, Bulk).

? Expertise in ETL tools such as Informatica or Talend

? Strong data modelling skills for schema design and relationship mapping.

? Strong understanding of Salesforce data models, objects, and metadata management.

? Experience with data quality tools and processes (cleansing, deduplication).

? Familiarity with Change Data Capture (CDC) and incremental data extraction mechanisms.

? Knowledge of deployment and migration tools, such as Salesforce Data Loader or ANT Migration Tool.

? Ability to troubleshoot data discrepancies and optimize data flows for performance.

? Experience with data validation and reporting tools.

? Familiarity with PySpark for large-scale data processing is a plus.

? Familiarity with cloud-based platforms and integration patterns is a plus.

Preferred Qualifications:

  • BS degree in Computer Science or Engineering or equivalent experience
Not sure where to start? Ask IVY.
IVY