Staff Data Engineer at Unite Us

<div class="p-pdf_iframe__page" data-page-number="1"> <p><strong>Unite USA Inc. dba Unite Us<br></strong><br><strong>Staff Data Engineer</strong><br><br><strong>Location: </strong>New York, New York<br><br><strong>Job Description<br></strong><br>The Staff Data Engineer will lead enterprise-wide data architecture strategy and long-term data<br>platform design. Evaluate and approve major technical designs, including establishing<br>engineering best practices. Serve as technical advisor to leadership and influence product/data<br>roadmap. Champion data governance, democratization, and scalable data standards. Work with<br>cross functional teams including the Solutions Delivery, Business Intelligence; Predictive<br>Analytics, and Enterprise Services teams to assist with and support any data-related technical<br>issues and/or needs. Analyze, manipulate, and process large sets of data using statistical<br>software. Develop ETL (Extract, Transform, Load) and data pipeline solutions using Airflow,<br>DBT, Python, and Snowflake to load a data warehouse. Design and manage ETL workflows to<br>support large-scale ingestion, transformation, and loading of data. Build and scale CI/CD<br>pipelines and enterprise-wide data quality frameworks. Test internal data pipelines for reliability<br>and performance. Test, validate, and reformulate models to ensure accuracy. Complete and<br>improve monitoring and validation processes to maintain pipeline reliability and data accuracy.<br>Write new functions or applications in programming languages to conduct analysis. Build<br>streaming pipelines with Confluent Kafka for event-driven processing. Build dashboards and<br>monitors for pipeline health, freshness, uptime, and SLA compliance. Apply data modeling to<br>extract and analyze information from large structured and unstructured datasets. Telecommuting<br>available anywhere in the United States. Annual salary between $175,000 - $200,000.</p> <p><br><strong>Job Requirements<br></strong><br>Requires at least a Master’s degree or foreign equivalent in Data Science, Applied Computer<br>Science, Computer Science or a related field. Must possess 4 years (48 months) of experience<br>with all of the following: (a) developing and optimizing scalable pipelines using Apache Airflow<br>and Python; (b) building streaming pipelines using Kafka; (c) tuning big data systems using<br>Kafka and Hive; (d) using Spark to create jobs for batch and streaming analytics; (e) writing and<br>using complex SQL queries to retrieve data from data warehouses for analysis and reporting (f)<br>using AWS for data processing and large-volume workloads; (g) automating deployment of code<br>and processes using Python; (h) working with cloud-based infrastructure; (i) building Datadog<br>dashboards to monitor pipelines and system performance; (j) using Docker and Kubernetes to<br>improve and validate containerization and automation reliability; (k) implementing Continuous<br>Integration/Continuous Deployment (CI/CD) using Jenkins; and (l) utilizing all of the following:<br>GitHub as a code repository, Windows and Linux operating systems. Experience may be gained<br>concurrently. Telecommuting available anywhere in the United States.<br>Benefits: Paid Parental Leave; 401K Match; Flexible Vacation Time; Medical, Dental and<br>Vision Coverage; Commuter Benefits.</p> </div>