Data Engineer
437 madison Avenue 27th Floor New York, NY 10001 US
Job Description
Our financial services client in Manhattan is seeking a Data Engineer to join their team. This is a full-time, salaried position. Three days in the office (Manhattan, New York) and two days work from home.
Position Overview
As a Data Engineer, you will develop a deep understanding of our client's data landscape and collaborate closely with cross-functional teams to gather requirements and deliver data-driven solutions. You will be responsible for providing data services through reports, systems, and interface applications to support various business functions.
The ideal candidate has a strong background in data engineering, is proficient in mainstream database systems, and possesses solid development skills, including the ability to perform in-depth debugging and troubleshooting. A passion for solving complex data challenges and enabling data-driven decision-making is essential.
Key Responsibilities:
- Data Analysis and Reporting: Work with business analysts and stakeholders to understand their data needs and deliver data solutions. Requires a strong understanding of data processing and hands-on experience with tools such as SQL, Pandas, and Jupyter.
- Data Service Platform Development: Identify recurring analytical and reporting needs and implement reusable solutions within the internal data service platform. Proficiency in Python, Django, and JavaScript is required, along with strong debugging skills.
- Data Processing Pipelines: Design, build, and maintain reliable and scalable daily data pipelines. Experience with Apache Airflow or other ETL orchestration tools is essential.
- External Data Interfaces: Develop and maintain data services for external systems based on internal datasets. Requires knowledge of RESTful APIs and message queue technologies such as IBM MQ or Apache Kafka.
- Data Platform Infrastructure: Work with cloud-based infrastructure components including EC2, Docker, RDS, Redshift, and S3 to support the data platform.
Qualifications:
- Must:Bachelor’s degree in Data Engineering, Computer Science, Information Technology, or a related field
- Minimum of 3 years’ experience in a data engineering role
- Proficiency in Python and Pandas
- Strong SQL skills
- Preferred:Experience with JavaScript and web debugging tools
- Familiarity with Apache Airflow
- Experience with Django
- Experience with AWS and related cloud technologies
