Can Batch evolution be completely replaced by Real-time in Banking?

Traditionally, Core banking Systems are developed using Mainframe technologies which leverages batch processing for multiple banking operations. Batches were developed due to the manual processes and operating hours of banks were limited.

Batch processes are nothing but scheduled processes which executes series of steps at defined time. Traditionally batch processes were leveraged for multiple banking operations including interest calculation, bulk transaction processing & generating reports for regulators etc.

Digital transformation

Nowadays banks operate 24*7 days in online mode. Hence, it needs replacing batch with real-time processes. Most of these processes are automated and processed in real-time as and when information is available by leveraging streaming and event-driven solutions

Many banks are analyzing, planning and initiating digital transformation of Core banking systems which are traditionally developed in large mainframe systems. Although Mainframe systems are very efficient in processing millions of transactions, some of the limitations for such implementations are –

• Scarcity of Legacy Skillset
• Higher CapEx to handle increased load and higher OpEx
• Integration challenges with modern systems
• Agility is limited

With cloud technologies which provides on-demand infrastructure that can be easily scalable to required capacity, banks are looking for cloud as an option for replacing Core banking operations with modernized technical platforms.

While deciding whether an existing batch process can be replaced in transformed system, analyze the current process and identify whether it is still required in new scenario or if it can be migrated to real-time.

In scenario, where other interfacing applications has dependency on batches, migrating it in phases when all dependent applications are transformed could be an option or replacing it to real-time or near real-time process could be investigated.

Majority of public cloud vendors provides managed or cloud-native PaaS services which are easily provisioned and provides in-built features for handling real-time processing of events. With this easy to use, highly scalable and secure streaming and event-driven services, real-time or near real-time processing flows can be implemented with an agility.

Challenges in replacing Batch Processing in Banking

Ensuring stable processing and providing correct output is vital to keep banking business running. Failure or delays in providing appropriate output may lead to direct impact on banking operations.

While embarking a journey to digital transformations, banks always face challenges with transforming legacy batches. Here are some of the challenges that should be considered while transforming batch workloads.

• Knowledge Scarcity: Few challenges while modernizing the legacy batch workloads includes – missing documentation of existing system, source code not available for third party components or developers worked on the system no longer exist in banks. In such a case, reverse engineering of an application needs to be done rigorously to identify functionalities of batch workload and its upstream and downstream interfaces.
• File based Interfacing Applications: Multiple legacy applications interface each other via intermediate files which are processed by batches. Batch may provide file to dependent application or process a file or invoke a process in dependent application.This I/O file-based interface between different applications can be re-engineered by passing file data directly between calls. While modernizing batch processes, all the interfacing applications have impact hence proper analysis and planning needs to be done to see how all the interfacing applications can adapt the change and implement any façade layer if required to avoid impact on other applications.
• Impact on interfacing Applications: Although batch workloads are transformed to modern technology platform, it should have minimum impact for upstream and downstream interfaces. In case, batches are transformed and hosted in cloud platform, mechanism should be set to access interfaces in on-premises environment so that existing business functionalities will continue functioning as it is.
• Manual Submission of batch jobs: Certain batches run in ad-hoc manner, where end user manually run it on-demand as per the requirement. In such cases, if it’s difficult to replace batch workloads with real-time processing then still modern batch frameworks and cloud native services can be leveraged for the implementation. In such cases, thorough analysis is required to find an appropriate solution for such a requirement.
• Infrastructure Requirement: Legacy applications have batches implemented in Mainframe technologies which provides greater processing performance. While modernizing batch processing workloads, appropriate capacity planning and scaling requirements should be understood and implemented to ensure the same performance can be achieved using modernized technology platform.
• Cutover Requirements: Batches generally interface with multiple applications, hence while modernizing batch applications, cutover requirements should be taken into consideration which includes downtime accepted, no. of applications to be deployed together in a move group and rollover strategy.

Replacing Batch with Real-time?

• With the change in banks operating model where it’s always in online mode, and with increased customer expectations such as always on, speedy response to banking operations, real-time processing is the need of an hour. However, certain background processes still need to be processed in batch mode. Not all banking processes can be replaced as real-time. Still some batch processing is required for banking processes such as payment settlement, insurance claim approvals, fraud check, risk check, balance check with core systems for payment operations or check approvals etc.
• New generations core banking systems should be reconsidered to see if real-time capabilities can be implemented wherever possible and use modern batch capabilities for certain unavoidable processes which cannot run in real-time or near real-time mode.