Experience working in agile software development teams: 5+ years;
Experience with SQL language: 3+ years;
Python experience: 3+ years;
Experience working with cloud platforms like Amazon, Azure or Google Cloud Platform: 2+ years;
Experience working with Docker: 2+ years;
Experience in software release management: Git, CI/CD;

Implement and maintain different data ingestion/processing pipelines for existing and new projects, extending and improving our existing data pipeline stack built using Airflow, Cloud Dataflow, and Bigquery.
Handle architectural and design considerations such as performance, scalability, reusability, and flexibility constraints.
Evaluate the data integrity from each source, finding the way to ingest, curate, and normalize it.
Document and communicate changes inline code, PR, tickets and release notes with the engineering, product and data team.
Work jointly with the research team to make prototypes out of proof-of-concepts and operationalize prototype pipelines.
Work jointly with the product team to understand the needs of the output datasets and find interconnections between projects.
Work jointly with the front-end and data to make sure the data is being used correctly.
Implement uptime monitoring for the multiple ingestion pipelines and set up continuous QA metrics on processing pipelines.
Debug data issues working closely with team members from other areas of the organization (subject matter experts or ML scientists).
Modify/Fix/Implement new API endpoints owning the full development cycle. Including supporting QA for test automation and product for feature release and documentation.

Data & Back-End Engineer

Apply for this position