Project Overview
CDC Data Stream. We partnered
with BRI to implement CDC streaming
architecture designed to provide
real-time data access across diverse endpoints while preserving
core system performance.
CDC Data Stream. We partnered
with BRI to implement CDC streaming
architecture designed to provide
real-time data access across diverse endpoints while preserving
core system performance.
The Challenge
BRI needed to eliminate system latency and high batch-processing costs while integrating complex AS400 journals with modern platforms like Hadoop and HBase.
BRI needed to eliminate system latency and high batch-processing costs while integrating complex AS400 journals with modern platforms like Hadoop and HBase.
The Solution
We utilized Qlik Replicate to capture delta changes from AS400 and leveraged Confluent Kafka to stream data into a unified HBase and Hadoop storage layer.
We utilized Qlik Replicate to capture delta changes from AS400 and leveraged Confluent Kafka to stream data into a unified HBase and Hadoop storage layer.
The Result
The project enabled real-time analytics for faster decision-making and improved efficiency by reducing data transfer costs and core system dependency.
The project enabled real-time analytics for faster decision-making and improved efficiency by reducing data transfer costs and core system dependency.
​
About
Bank Rakyat Indonesia (BRI) is one of the largest state-owned banks in Indonesia, primarily focused on providing comprehensive banking services to micro, small, and medium enterprises (MSMEs) to drive national economic growth
The Business Challenges
- System Latency and Load: The need to access real-time data without overburdening the core/source systems (AS400), specifically avoiding the high costs and delays associated with traditional batch processing.
- Complex Data Integration: Integrating diverse endpoints including AS400 journals, Qlik Replicate, Confluent (Kafka), HBase, Hadoop, and various RDBMS platforms while ensuring secure data transmission.
Sibernetik Solutions
CDC Technology Implementation: Deploying Change Data Capture (CDC) via Qlik Replicate to read AS400 journals, allowing only changed data (delta) to be transferred.
Unified Streaming & Storage Architecture: Utilizing Confluent (Kafka) as a temporary buffer and HBase/Hadoop for replication and historical storage to support both operational and analytical needs.
The Business Benefits
- Real-Time Data Capability: Achieved real-time data access for faster business decisions and streaming analytics without impacting the core system's performance.
- Optimized Resource Efficiency: Significantly reduced data transfer costs and improved extraction efficiency by moving away from dependency on core system batch processes.
