Beschreibung
- IT - Software Engineering
- Bern
- Contracting
- Teilzeit, 80-100%
- ASAP - 3-4 months
- Snowflake, CDC, Apache
Your tasks:
- Analyse and evaluate the CDC approach provided by TigerData based on Apache Iceberg
- Define and design a suitable target architecture for continuous data synchronisation from TigerData to Snowflake
- Configure and optimise Snowflake for processing data provided through Apache Iceberg
- Design and implement the required Apache Iceberg components and data structures
- Ensure reliable processing of inserts, updates and deletes
- Address schema evolution, data consistency, data latency, recovery and reconciliation
- Establish monitoring, logging and error-handling mechanisms to ensure stable production operations
- Coordinate technical activities with TigerData and, where required, directly with Snowflake
- Support the implementation, validation and production rollout of the end-to-end solution
- Document the technical architecture and facilitate knowledge transfer to the internal team
Your profile:
- Several years of hands-on experience in designing and implementing data integration and replication solutions
- Strong knowledge of Snowflake, supported by practical project experience
- Proven hands-on project experience with Apache Iceberg
- Solid understanding of Change Data Capture (CDC), data replication, and streaming or synchronisation methods
- Experience in building and operating production-ready CDC and data integration platforms
- Knowledge of schema evolution, data consistency, recovery, reconciliation, monitoring and error handling
- Experience integrating different data platforms and building robust end-to-end data pipelines
- Ability to analyse and drive technical solutions in collaboration with software vendors and internal stakeholders
- Independent, structured and solution-oriented working approach
- Experience with Fivetran HVR or comparable CDC and data replication solutions is an advantage
- Experience integrating time-series databases or related data platforms is an advantage
- Fluent in Englisch or German