Description:
We are seeking a highly skilled Platform Engineering Lead to architect, build, and manage enterprise-scale cloud-native data platforms. The ideal candidate will have strong expertise in Databricks, AWS Cloud, Terraform, Python, dbt, DevOps, and Platform Engineering to build scalable, secure, and highly available data platform solutions.
This role is responsible for platform engineering, infrastructure automation, cloud operations, DevOps enablement, platform reliability, and platform security. The successful candidate will work closely with cross-functional teams to deliver modern data platform capabilities that support enterprise data and analytics initiatives.
Key Responsibilities
- Design, build, and manage enterprise-scale cloud-native data platforms.
- Lead Databricks platform administration and engineering activities.
- Design and implement Lakehouse Architecture using Delta Lake.
- Manage Databricks Workflows, Unity Catalog, security configurations, and access controls.
- Optimize platform performance, scalability, availability, and cost efficiency.
- Implement governance controls across development, testing, and production environments.
- Design and implement AWS-based infrastructure using services such as S3, Glue, Lambda, ECS, EKS, CloudWatch, IAM, Secrets Manager, and VPC.
- Develop Infrastructure as Code (IaC) solutions using Terraform.
- Build reusable Terraform modules for infrastructure provisioning.
- Design and implement CI/CD pipelines for automated deployments.
- Improve deployment automation and platform reliability using DevOps best practices.
- Utilize GitHub Actions, GitLab CI/CD, Azure DevOps, Jenkins, Docker, and Kubernetes.
- Implement monitoring, observability, and incident management processes.
- Perform root cause analysis and implement preventive actions for platform issues.
- Collaborate with development, infrastructure, security, and operations teams to deliver secure and scalable platform solutions.
Required Qualifications
- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
- 10+ years of experience in Platform Engineering, Cloud Engineering, Data Engineering, or a related field.
- Hands-on experience designing and supporting enterprise-scale cloud-native data platforms.
- Strong understanding of DevOps, Infrastructure Automation, Cloud Security, and Platform Operations.
- Excellent analytical, troubleshooting, communication, and collaboration skills.