Avenga is a global IT and digital transformation champion. We deliver strategy, customer experience, solution engineering, managed services, software products and outsourcing services like: staff augmentation, team leasing and permanent employment.
Together, we are more than 2500 professionals with over 20 years of experience. We are present in Europe, Asia and the USA.
Data Engineer - Senior
Miejsce pracy: woj.
mazowieckie
Nr ref.: LP/DES/ZD/01
Nr ref.: LP/DES/ZD/01
Requirements:
- 4+ years of working with programming language focused on data pipelines,eg. Python or R
- 4+ years of experience working with SQL
- 3+ years of experience in data pipelines maintenance
- 3+ years of experience with different types of storage (filesystem, relation, MPP, NoSQL) and working with various kinds of data (structured, unstructured, metrics, logs, etc.)
- 3+ years of experience in working in data architecture concepts (in any of following areas data modeling,
- metadata mng., workflow management, ETL/ELT, real-time streaming, data quality, distributed systems)
- 3+ years of cloud technologies with emphasis on data pipelines (Airflow, Glue, Dataflow - but also other
- smart solutions of handling data in the cloud - elastic, redshift, bigquery, lambda, s3, EBS etc.)
Requirements:
- 4+ years of experience working with SQL
- 1+ years of experience in Java and/or Scala
- Very good knowledge of relational databases (optional)
- Very good knowledge of data serialization languages such as JSON, XML, YAML
- Excellent knowledge of Git, Gitflow and DevOps tools (e.g. Docker, Bamboo, Jenkins, Terraform
- Capability to conduct performance analysis, troubleshooting and remediation (optional)
- Excellent knowledge of Unix
- Pharma data formats is a big plus (SDTM)