Be wary of WhatsApp messages impersonating Jobline Resources's staff offering job opportunities. Those who encounter suspicious messages can contact Jobline at +65 6339 7198

Responsibilities

  • Build and maintain end-to-end data pipelines for ingestion, processing and transformation using Informatica IDMC and Databricks (PySpark) ensuring reliability and testability.
  • Build and maintain end-to-end data pipelines to ingest, validate, transform and load data from multiple sources into data lake house using Informatica IDMC and Databricks (PySpark) ensuring reliability and testability.
  • Monitor and tune pipeline and query performance; apply basic resource-scaling techniques to meet performance targets.
  • Follow security and privacy practices for handling sensitive data, including encryption and access controls
  • Automate routine tasks (scheduling, retries, basic alerting) using IDMC workflows, Databricks Jobs, AWS Step Functions or Lambda
  • Document pipeline designs, configurations and runbooks; record key metadata and lineage details as required.
  • Troubleshoot production data issues, perform root-cause analysis and implement fixes; escalate complex incidents to Technical Leads.
  • Contribute to cost and performance optimization by suggesting right-sizing or schedule changes for compute resources.
  • Contribution for AI enablement with core AWS, Databricks and IDMC environment

Requirements

  • Bachelor’s / Master’s degree in computer science, data engineering, or a related field.
  • Minimum 3 years of experience in cloud data engineering, with expertise in AWS, Databricks, and/or Informatica IDMC.
  • Proficiency in programming languages such as Python/PySpark for building data pipelines.
  • Strong Database SQL hands-on.
  • Strong knowledge in data modelling and schema design.
  • Basic Linux and command-line skills to run scripts in a shell environment.
  • Excellent problem-solving and analytical skills.
  • AWS, Databricks and Informatica certifications are a plus.
  • Familiarity with data visualization tools like Tableau.
  • Knowledge of data governance and data cataloguing tools, especially Informatica IDMC.
  • Experience with version control systems (e.g., Git) and CI/CD pipelines.
  • Understanding of DevOps principles for managing and deploying.
  • Strong communication and collaboration skills.

Shortlisted candidates will be offered a 1 Year agency contract employment.