Data Engineer

Description:

What You Will Do

• Perform complex queries using SQL and / or other tools (e.g., R, Python Pandas, PySpark)
• Manage ETL (Extract, Transform, Load) functions
• Develop, build, test, and maintain scalable data pipeline architectures and tools
• Partner with data scientists/analysts and cross-functional teams to discover, collect, cleanse, and refine the data needed for analysis and modeling
• Operate in a cloud environment and assess environment specifications and requirements for big data
• Work with data integration and management tools such as databases (cloud and on-prem; relational and NoSQL), Databricks and a variety of tools from major cloud providers (AWS, Azure, GCP)
• Understand client needs and translate to technical requirements and Jira cards for implementation

What You Will Need

• U.S. citizenship and Secret Clearance.
• B.S./B.A. is required plus 3+ years of relevant professional experience
• Experience developing data pipelines / ETLs and performing data engineering using Spark and Databricks
• Experience performing data engineering to enable data science and machine learning
• Experience in Python or Scala
• Understanding relational databases, columnar data warehouses, NoSQL, and other storage types
• Proficiency in SQL
• Ability to instrument basic automation and CI/CD
• Familiarity with cloud-based services (AWS, Azure, GCP)
• Knowledge/experience designing and building distributed systems for scalability and security
• Understanding of Agile development methodologies