Senior Data Engineer
In today's digital age, instant information is the norm. But this transparency disappears when it comes to consumer products, and their ethical and sustainable production through supply chains.
Addressing the global climate challenge and safeguarding both humanity and the planet requires a transformation in food systems.
This is why we built OpenSC.
Co-founded by WWF and The Boston Consulting Group, OpenSC is on a mission to revolutionize the way we buy and produce products by providing data on transparency and traceability for supply chains.
We deliver social and environmental impact by providing supply chain transparency that empowers brands to know, influence, and prove the sustainability of their supply chain practices.
This is achieved through the OpenSC platform, which leverages AI, Blockchain and Remote Sensing technologies to enable automated ingestion, analysis, and sharing of a supply chain's data.
Tasks
- Further design and build OpenSC’s data platform across all data domains : collection, storage, transformation and analyses
- Develop high-quality data pipelines that transform structured, semi-structured, and unstructured data from various sources into a standardized data model
- Expand OpenSC’s data model to accommodate a growing number of use cases
- Assume responsibility for the business interpretation of data and actively contribute to data science and analytics projects
- Provide support and guidance on data security and cloud infrastructure topics, ensuring a secure and efficient handling of data
- Adhere to software engineering best practices regarding version control, testing, code reviews, deployment etc.
- Communicate effectively with technical and non-technical stakeholders
- Contribute elevating the team through knowledge sharing on data engineering best practices
Requirements
- 6+ years of relevant industry experience building production-level data products
- Proven solid foundation of core data concepts (e.g. integration, modeling, security, lineage, governance etc.)
- Excellent Python skills and advanced competencies with main data ecosystem libraries (pandas, great expectations, dbt etc.)
- Advanced knowledge of SQL and relational databases (e.g. PostgreSQL)
- Extensive experience with data pipeline orchestration tools (e.g. Dagster, Airflow, etc.)
- Extensive experience working with cloud infrastructure and services (ideally on AWS)
- Experience working with different data modeling patterns and ability to abstract complex real-life information into a standardized data model description
- Demonstrated familiarity with Version control (Git), Containerization (Docker, Kubernetes), CI / CD (e.g. CircleCi, Github Actions), IaC (Terraform)
- Strong interpersonal skills and ability to work in a multidisciplinary and multicultural team
Senior Google Cloud Data Engineer (m/f/x) - Anti-Financial Crime Technology
Work as a senior developer for developing analytics algorithms against large data sets residing on data lakes and/or on Google Cloud. Work as a senior developer for various data sourcing in Hadoop and/or Google Cloud. Proficient in Hadoop, Python, Spark, SQL, Unix, Pentaho, Hive, DataFlow, CloudComp...
Senior DevOps Engineer (f/m/x) - Cash Data Technology
As a Senior DevOps Engineer (f/m/x) you'll be part of the team engineering distributed systems and automating clusters management. We have a strong focus on promoting technical excellence - our engineers work at the forefront of financial services innovation using cutting-edge technologies. We are c...
Senior Data Engineer (m/w/d) - Python Entwickler
Datenplattform für Business Intelligence, Digital Analytics, Marketing Controlling und Data Science betreiben und kontinuierlich weiterentwickeln. Datenquellen an das Data Warehouse anbinden, die Qualitätssicherung der importierten Daten übernehmen, Datenarchitekturen sowie ETL-Prozesse konzipieren,...
Senior Data Engineer
Develop scalable data storage solutions, data warehousing, and data architecture. Implement best practices for data governance, data security, and data privacy. Design and develop data models, data pipelines, and ELT processes. Optimize data systems for efficiency and performance to ensure reliable ...
Senior Data Engineer
Ensure data consistency and integrity during the integration process, performing data quality checks, data validation, and cleaning as needed. Data Engineer opportunity with a global biopharmaceutical company based in Germany and wider Europe. Design and develop data pipelines that extract data from...
Senior Google Cloud Data Engineer (f/m/x) - Anti-Financial Crime Technology
Work as a senior developer for developing analytics algorithms against a large data sets residing on data lakes and/or on Google Cloud. Work as a senior developer for various data sourcing in Hadoop and/or Google Cloud. Proficient in Hadoop, Python, Spark, SQL, Unix, Pentaho, Hive, DataFlow, Cloupos...
Senior Data Engineer
You would be joining a new programme of work to transform how Springer Nature uses its data: building up data capabilities, creating a data platform and engineering capability (technology, people and process) to create a foundation for the future, adding value to cross-organisation Initiatives and k...
Senior Data Engineer (f/m/x) – Origination & Advisory Technology
As a Data Engineer (f/m/x) you will be part of product team, responsible for all steps of bringing high-quality data to end users: ETL processes, data analysis and preparation, platform development and evolution. The team consumes billions of records for reporting, deep data analysis, and advanced d...
Senior Data Engineer
The data engineer will provide product data science engineering the tools and best practices that will enable them to complete their mission quickly in an automated way, improve data quality and data consistency across our pipelines and dashboards. We are seeking a talented Snowflake Data Engineer t...
Senior Data Engineer with a twist (M/F/D) $$Senior Data Engineer mit einem besonderen Kniff (M/W/D)$$
Skills and Qualifications Finished studies in Computer Science or other relevant field Good organizational / project management skills T-SQL Data Warehouse Modeling SSIS PowerShell Ideally SSAS Ideally PowerBI Fluent German (at least C1) and good English language skills What you will do Further deve...