Project Overview
CDC Technology Procurement. We eliminated downstream delays by transitioning from batch-based legacy extraction to real-time data streaming via Qlik Replicate and Kafka.
CDC Technology Procurement. We eliminated downstream delays by transitioning from batch-based legacy extraction to real-time data streaming via Qlik Replicate and Kafka.
The Challenge
Batch-based data extraction from AS400 and SQL server Core Banking caused replication delays to downstream platforms.
Batch-based data extraction from AS400 and SQL server Core Banking caused replication delays to downstream platforms.
The Solution
Implemented Realtime Change Data Capture (CDC) using Qlik Replicate from AS400 core system & streamed data into Kafka.
Implemented Realtime Change Data Capture (CDC) using Qlik Replicate from AS400 core system & streamed data into Kafka.
The Result
Enabled near real-time data replication from core banking systems to significantly improve data availability for operational reporting.
Enabled near real-time data replication from core banking systems to significantly improve data availability for operational reporting.
​
About
OCBC is a leading financial services group in Asia, dedicated to helping individuals and businesses build wealth and achieve sustainable growth through innovative banking and investment solutions.
The Business Challenges
- Batch-based extraction involves pulling massive datasets at scheduled intervals, which frequently saturates system resources and creates significant processing bottlenecks. During these heavy read operations, the primary AS400 and SQL Server databases prioritize extraction tasks, leading to a "sync gap" where downstream platforms lag behind the source data. This delay results in data latency across reporting and analytical layers, often requiring a transition to change data capture (CDC) for real-time consistency.
Sibernetik Solutions
- By implementing real-time Change Data Capture (CDC) via Qlik Replicate, the system now identifies and captures row-level database changes as they occur without the overhead of traditional batch queries. These captured events are immediately streamed into Kafka, creating a high-throughput data pipeline that ensures downstream consumers have access to the latest core banking updates. This architecture effectively eliminates the previous replication delays, providing a scalable and decoupled environment for real-time analytics and reporting.
The Business Benefits
Enabled near real-time data replication from core banking systems by transitioning to a streaming architecture that synchronizes transactions across all platforms with minimal latency.
Improved data availability for operational reporting by decoupling workloads from core systems to provide stakeholders with immediate access to live metrics without compromising primary performance.
