EMPREGO SPARK SCALA DATA ENGINEERS

Connaissance des architectures data (datalake, lakehouse... contribuer à l’animation d’un collectif de data engineers et data architects pour partager les savoir-faire...

scala and pyspark for big data processing... tools like apache spark and big data pipelines... apply best practices in data engineering and cloudbased solutions...

A psa sines, multinacional de referência e líder de mercado no sector de movimentação de carga contentorizada, encontra-se atualmente a recrutar para a posição de...

Experience with big data and data modelling... big data processing: engage in large-scale data projects, utilizing frameworks like apache spark for efficient data...

• experience building and optimizing ‘big data’ data pipelines, architectures, and data sets... • build processes supporting data transformation, data structures...

Understanding of big data technologies such as hadoop or spark...   data quality and governance: support data quality initiatives by performing data validation...

• familiarity with data warehousing and data modeling concepts... • proficiency in data integration techniques, etl processes and data pipeline architectures...

• establish data architecture design patterns and best practices... data lake/raw, sand box, curated/cleansed layer, analytics/integrated layer, etc...

* data engineering expertise: proficient in sql and experienced in data pipeline frameworks (e... * data engineering: design and implement efficient, scalable data...

Daily data validation: conduct daily data validation checks against operational data to ensure accuracy and completeness... data management: design and implement...

data is your driving force... you do this by working closely together in a multidisciplinary team of data scientists, engineers and architects...

Your mission you will be part of a central data enterprise team within the data technology pillar that aims at building the best data technology solutions for the...

• proficiency in sql, data modeling, and data warehousing concepts... • hands-on knowledge of apache spark... • develop and manage data pipelines using databricks...

Aws s3, azure data factory)... design, build, and maintain scalable data platforms... ensure data security and privacy through proper implementation of access controls...

• experience with data architecture: hands-on experience designing and managing scalable data architectures, including data lakes and data warehouses...

Mongodb);familiarity with big data platforms (hadoop, spark) and experience in cloud environments (e... this professional will be responsible for exploring large...

This professional will be responsible for exploring large volumes of data and transforming this information into actionable recommendations, using statistical models...

• proficiency in data integration techniques, etl processes and data pipeline architectures... • developing and optimizing data processing and data storage systems...

Reporting to the data leadership, you will collaborate closely with data analysts, data scientists, and business stakeholders to ensure the seamless flow of data...