Pyspark Engineer (Charlotte)

Job Description: Our client is seeking a highly skilled and experienced Senior PySpark Developer to join our dynamic data engineering team. This role is ideal for a motivated professional with a strong background in big data technologies, ETL processes, and database systems. The successful candidate will play a key role in designing, developing, and optimizing data pipelines and analytics solutions that support critical business operations. Note: Only candidates eligible to work on a W2 basis will be considered. Key Responsibilities: Design, develop, and maintain scalable data pipelines using PySpark and Spark SQL. Collaborate with data architects, analysts, and business stakeholders to understand data requirements and deliver robust solutions. Optimize and troubleshoot complex ETL workflows and data transformation processes. Write efficient SQL queries for data extraction, transformation, and loading. Ensure data quality, integrity, and compliance with enterprise standards. Participate in code reviews, testing, and deployment activities. Monitor and maintain production workflows, including job scheduling and performance tuning. Required Skills & Qualifications: 5 years of experience in data engineering or software development. Strong proficiency in PySpark, Python, and Spark SQL. Solid experience with SQL and relational database systems. Proven expertise in ETL development and data pipeline orchestration. Familiarity with job scheduling tools; Autosys experience is a plus. Excellent problem-solving skills and attention to detail. Strong communication and collaboration abilities. Preferred Qualifications: Experience working in the financial services industry. Familiarity with cloud platforms and big data ecosystems.