You are viewing a preview of this job. Log in or register to view more details about this job.

Data Engineer

InterSources Inc
Email

Job Description:

We are seeking a talented Data Engineer to join our dynamic team. As a Data Engineer, you will be responsible for designing, implementing, and maintaining scalable data pipelines to support our data-driven initiatives. You will work closely with cross-functional teams to understand data requirements, develop ETL processes, and ensure the reliability and efficiency of our data infrastructure.

Responsibilities:

  • Design, develop, and maintain robust data pipelines and ETL processes to collect, process, and store data from various sources.
  • Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and translate them into technical specifications.
  • Optimize data pipelines for performance, scalability, and reliability, ensuring timely and accurate data delivery.
  • Implement data quality checks and monitoring mechanisms to ensure the integrity and consistency of data.
  • Work closely with DevOps and IT teams to deploy and manage data infrastructure in cloud environments (e.g., AWS, Azure, GCP).
  • Stay current with emerging technologies and best practices in data engineering and contribute to the continuous improvement of our data architecture and processes.
  • Provide support and troubleshooting for data-related issues and participate in on-call rotation as needed.

Qualifications:

  • Bachelor's degree or higher in Computer Science, Engineering, or a related field.
  • Proven experience as a Data Engineer or similar role, with a strong background in data warehousing, ETL, and data modeling.
  • Proficiency in programming languages such as Python, Java, or Scala, with experience in developing data pipelines using frameworks like Apache Spark, Apache Beam, or Apache Flink.
  • Hands-on experience with relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra).
  • Familiarity with cloud computing platforms and services, particularly AWS (e.g., S3, EC2, Glue, Redshift) or Azure (e.g., Blob Storage, Azure Data Factory, Azure Databricks).
  • Experience with containerization and orchestration tools such as Docker and Kubernetes is a plus.
  • Strong analytical and problem-solving skills, with the ability to work independently and as part of a team in a fast-paced environment.
  • Excellent communication and collaboration skills, with the ability to effectively interact with technical and non-technical stakeholders.