Pyspark Developer

Description:

🚀 We’re Hiring | PySpark Developer (Contract) 🚀
📍 Location: New York
📄 Job Type: Contract


We are seeking an experienced PySpark Developer to join our data engineering team. The ideal candidate will have strong hands-on expertise in PySpark and cloud-based data platforms, along with the ability to lead and mentor team members.

🔧 Key Skills & Requirements:
In-depth knowledge of PySpark, including DataFrames, Datasets, RDDs, Spark Streaming, and Parquet partitioning
Strong experience with Databricks
Hands-on experience with AWS services such as S3, Athena, and Glue (Athena understanding is mandatory)
Strong skills in Python (Data Python – not application development)
Advanced SQL expertise
Experience with ETL / ELT processes and data warehousing
Familiarity with Git, CI/CD tools, and Agile methodologies
Understanding of data governance and security best practices
Excellent communication skills
Ability to train, mentor, and lead developers to bridge skill gaps
Willingness to work onsite from the office all 5 days

🧠 Responsibilities:
Design, develop, and optimize scalable data pipelines
Work with large datasets using PySpark and Spark frameworks
Collaborate with cross-functional teams to deliver high-quality data solutions
Ensure best practices in data quality, security, and governance
Support team growth through technical guidance and mentoring