DevOps Engineer (Data Engineering and Pipeline Management)

Responsibilities:

• Design, build, and optimize scalable data pipelines for ETL processes.
• Ensure high availability and reliability of data flow across systems.
• Implement Infrastructure as Code (IaC) using tools like Terraform or Ansible.
• Automate deployment processes and system configurations.
• Set up and manage CI/CD pipelines using tools such as Jenkins, GitLab CI/CD, or CircleCI.
• Collaborate with development teams to streamline code integration and deployment.
• Deploy and manage applications on cloud platforms like AWS, Azure, or Google Cloud.
• Optimize cloud resources for performance and cost-efficiency.
• Implement monitoring solutions using tools like Prometheus, Grafana, or ELK Stack.
• Proactively identify and resolve issues to maintain system health.
• Ensure data security protocols are in place and comply with industry regulations.
• Conduct regular security assessments and audits.
• Work closely with cross-functional teams including data scientists, software engineers, and product managers.
• Participate in architectural discussions and contribute to technical decision-making.

Requirements:

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• Minimum of 3 years of experience in DevOps, Data Engineering, or a similar role.
• Proficiency in programming languages such as Python, Java, or Scala.
• Experience with big data technologies like Hadoop, Spark, or Kafka.
• Strong knowledge of containerization tools like Docker and orchestration platforms like Kubernetes.
• Familiarity with database systems, both SQL and NoSQL.
• Experience with version control systems like Git.
• Excellent problem-solving and analytical abilities.
• Strong communication and collaboration skills.
• Ability to work independently and within a team.
• Adaptability to fast-paced environments and changing requirements.