DATAOPS REMOTE

Job Description MAIN RESPONSABILITIES • Incident Response o Understand problems from a user perspective and communicate to clearly  understand the issue.
o Reproduce bugs or issues that users are facing.
o Apply root cause analysis to quickly and efficiently  o Find the root cause of the problem, patch it, test it, and communicate with the  end user.
o Write postmortems summarizing every step of resolution and helping the team  to track all issues.
o Monitor existing flows and infrastructure and perform the same tasks when  discovering bugs/issues through monitoring and alerting.
• Maintenance o Monitor flows and infrastructure to identify potential issues.
o Adapt configurations to keep flows and infrastructures working as expected,  keeping the operations without incident.
• Database Optimization o Track costs and time of processing through dedicated dashboards.
o Alert people who query tables the wrong way, involving high costs.
o Track down jobs, views, and tables that are running inefficiently and occur  either high costs or low speed of execution.
o Optimize jobs, queries, and tables to optimize both costs and speed of  execution.
• Infrastructure Management o Manage infrastructure through Terraform.
o Share and propose good practices.
o Decommission useless infrastructures such as services, tables, or virtual  machines.
• Deployments o Track future deployments with a Data Architect and participate in Deployment  Reviews.
o Share and propose good practices of deployment.
o Accompany Data Engineers during the entire process of deployments.
o Accompany Data Engineers in the following period of active monitoring.
o Ensure diligent application of deployment process, logging, and monitoring  strategy.
o Take over newly deployed flows in the run process.
REQUESTED HARD SKILLS • Google Cloud Platform.
General knowledge of the platform and various services,  and at least one year of experience with GCP.
• Apache Airflow.
At least two years of experience with the Airflow orchestrator,  experience with Google Composer is a plus.
• Google BigQuery.
Extensive experience (at least 4 years) with GBQ, know how to  optimize tables and queries, and able to design database architecture.
• Terraform.
At least two years of experience with Terraform, and know good  practices of GitOps.
• Apache Spark.
this is an optional expertise we would value.
Some of our pipelines  use pySpark.
• Additional Knowledge and Experience that are a Plus.
o Pub/Sub o Kafka o Azure Analysis Services o Google Cloud Storage optimization I'm interested I'm interested Privacy Policy