REMOTE opportunity with a FULL suite of benefits
Title: Data Engineer
Location: Remote with occasional travel
Type: Permanent
JOB DESCRIPTION
Your expertise in data engineering and software engineering will enable and empower our organization to build and deploy data driven solutions to production. You will be a champion of DataOps, and agile practices; actively participating in project teams to drive value.
Responsibilities:
- Agile Project Work: Work in cross-functional, geographically distributed agile teams of highly skilled data engineers, software/machine learning engineers, data scientists, DevOps engineers, designers, product managers, technical delivery teams, and others to continuously innovate analytic solutions.
- Design, develop, and review real-time/bulk data pipelines from a variety of sources (streaming data, APIs, data warehouse, messages, images, video, etc)
- Follow established design patterns for data ingest, transformation, and egress
- Develop documentation of Data Lineage and Data Dictionaries to create a broad awareness of the enterprise data model and its applications
- Apply best practices within DataOps (Version Control, P.R. Based Development, Schema Change Control, CI/CD, Deployment Automation, Test Automation, Shift left on Security, Loosely Coupled Architectures, Monitoring, Proactive Notifications)
- Problem Solving/Project Management: Constructively challenge while soliciting participation in problem solving to enrich possible solutions.
- Architecture: Utilize modern cloud technologies and employ best practices from DevOps/DataOps to produce enterprise quality production Python and SQL code with minimal errors. Participate in regular code review sessions and collaboratively discuss opportunities for continuous improvement in all solutions.
- Self-Development: Flexibly seek out new work or training opportunities to broaden experience. Independently research latest technologies and openly discuss applications within the department.
- Perform other duties as requested.
Qualifications
Minimum Requirements:
- Bachelor’s degree in engineering, computer science, analytical field (Statistics, Mathematics, etc.) or related discipline and three (3) years of relevant work experience
- OR
- Master’s or Ph.D. in engineering, computer science, analytical field (Statistics, Mathematics, etc.) or related discipline and one (1) year of relevant work experience
- Strong experience in at least two areas:
- Knowledgeable Practitioner of SQL development with experience designing high quality, production SQL codebases
- Knowledgeable Practitioner of Python development with experience designing high quality, production Python codebases
- Knowledgeable Practitioner in data engineering, software engineering, and ML systems architecture
- Knowledgeable Practitioner of data modeling
- Experience applying software development best practices in data engineering projects, including Version Control, P.R. Based Development, Schema Change Control, CI/CD, Deployment Automation, Test Driven Development/Test Automation, Shift left on Security, Loosely Coupled Architectures, Monitoring, Proactive Notifications using Python and SQL
- Data science experience wrangling data, model selection, model training, modeling validation, e.g., Operational Readiness Evaluator and Model Development and Assessment Framework, and deployment at scale
Preferred Qualifications:
- Working knowledge of Azure Stream Architectures, DBT, Schema Change tools, Data Dictionary tools, Azure Machine Learning Environment, GIS Data
- Working knowledge of Software Engineering and Object Orient Programming Principles
- Working knowledge of Distributed Parallel Processing Environments such as Spark or Snowflake
- Working knowledge of problem solving/root cause analysis on Production workloads
- Working knowledge of Agile, Scrum, and Kanban
- Working knowledge of workflow orchestration using tools such as Airflow, Prefect, Dagster, or similar tooling
- Working knowledge with CI/CD and automation tools like Jenkins or Azure DevOps
- Experience with containerization tools such as Docker
- Strong verbal and written communication skills in English language
Criteria/Conditions:
- Our client promotes a drug/alcohol-free work environment through the use of mandatory pre-employment drug testing and on-going random drug testing as allowed by applicable state laws