Senior Data Engineer - Airflow, Oracle DB, YAML Workflows - Level IV
- San José, San José, Costa Rica
- Fully Remote
CONSULTFIZ LATAM, a fully owned subsidiary of Confiz LLC is looking for a Senior Data Engineer - Airflow, Oracle DB, YAML Workflows to join our Finance Technology Delivery team as we intensify the pace of innovation and support future growth through technology solutions. The Senior Data Engineer will be responsible for supporting mission-critical Integrations between source data with financial SaaS and client applications.
Responsibilities:
- Leads multidimensional projects that involve multiple teams.
- Leads and works with other software engineers on design best practices and conducts code reviews.
- Resolves complex engineering problems, collaborating with others.
- Facilitates cross functional troubleshooting, root cause analysis and engages others when needed.
- Responsible for creating, evaluating, and contributing to feature detailed designs.
- Design, develop, and implement software utilizing an agile project cycle.
- Mentor others on the team and raise the bar for technical knowledge across a wide spectrum.
- Demonstrates thorough knowledge of information technology concepts issues, trends and best practices as they relate to Cloud technologies and system integrations.
- Apply and share knowledge of security coding practices and secure system fundamentals.
- This role involves working in rotational shifts. Shift timings may vary based on business needs.
- Employees are required to complete a proper handover at the end of their shift to ensure smooth workflow continuity.
- Coordination with the next shift team member is essential for task updates and seamless transition.
- Flexibility to adjust shift timings when needed is expected.
Requirements:
You must be able to:
- Design, implement, and optimize data pipelines and workflow orchestration using Apache Airflow (including YAML-based DAGs).
- Manage and maintain large-scale, standalone server-based infrastructures for data processing and job execution.
- Develop and optimize SQL/PLSQL queries and stored procedures for Oracle databases.
- Write clean and modular Python code for Airflow tasks, data transformations, and automation logic.
- Can manage Shell scripts and PowerShell scripts for task automation and system integration.
- Maintain configuration, monitoring, and scaling of Airflow deployments in high-load, multi-tenant environments.
- Use GitLab or GitHub for source control, code review, and integration with CI/CD pipelines.
- Collaborate with platform engineers, DevOps, and data analysts to deliver scalable solutions across large environments.
- Monitor workflows using Airflow UI, metadata DB, and custom alerting mechanisms; troubleshoot failures and bottlenecks.
- Ensure data pipeline reliability, consistency, and security within complex infrastructure boundaries.
You must have:
- Familiarity with Airflow Security configurations, logging and monitoring.
- Strong problem-solving skills with a focus on writing clean, maintainable, and well-tested code.
- Excellent communication skills (verbal and written) and the ability to collaborate effectively with cross-functional teams.
- 7+ years of professional experience in data pipeline development, preferably in enterprise or high-scale environments.
- Bachelor’s or Master’s degree in Computer Science, Engineering, or equivalent practical experience.
We have a global team of amazing individuals working on highly innovative enterprise projects & products. Our customer base includes Fortune 100 retail and CPG companies, leading store chains, fast growth fintech, and multiple Silicon Valley startups.
What makes Confiz stand out is our focus on processes and culture. Confiz is ISO 9001:2015 (QMS), ISO 27001:2022 (ISMS), ISO 20000-1:2018 (ITSM) and ISO 14001:2015 (EMS) Certified. We have a vibrant culture of learning via collaboration and making workplace fun.
People who work with us work with cutting-edge technologies while contributing success to the company as well as to themselves.
To know more about Confiz Limited, visit https://www.linkedin.com/company/confiz/