Bidirectional Replication - An Analysis
B.O.S. Software Service und Vertrieb GmbH and its tcVISION solution have been an integral part of the data replication, synchronization, transformation, and exchange market for over a decade.
No other manufacturer in this field covers the technical diversity of operating systems, databases, and cloud environments as consistently as B.O.S. Software. In this context we would like to point out the supported systems.
"Data Synchronization in Real Time" is the headline of our website and we have already explained the difference between "Real-time and Near Real-time" in a previous blog.
Our topic today is bidirectional replication, i.e. replication with equal partners (platforms/databases) with the aim to detect any change on one platform and replicate it to the other platform without sending the change back to the triggering platform.
This type of replication is also called "master-master" replication. Usually replications are performed from a production database (e.g. Db2, IMS, DL/I, VSAM, Adabas) to another environment (RDBMS under Windows/Unix/Linux or Linux on z, or a cloud system) where the replication itself is one-way. This so-called "master-slave" replication serves to synchronize the data of the determining, productive database on a parallel environment. The reason for this is to have the possibility to work with current data for analysis purposes, reporting, business intelligence, data warehousing, etc.
In the case of application modernization or even application relocation from the mainframe to an open system platform, bidirectional replication plays an important role, especially if both platforms are of strategic importance.
There are special forms of bidirectional replication. These are always present when mainframe databases are involved, whose data and records are referenced via so-called internal indices and keys. We will come back to this special type of processing later in the blog.
Let's take a look at "normal" bidirectional replication and assume the following example:
The replication is performed between a mainframe data source (Db2, IMS, DL/I, VSAM, CA DATACOM/DB) and a relational database under Windows/Unix/Linux or Linux on z (e.g. ORACLE, MS SQL Server, Db2 LUW, PostgreSQL, etc.) Both platforms have equal rights, i.e. users work with the respective systems and make changes. These changes must be detected and replicated to the other platform in real time. This type of processing usually also requires certain organizational measures (e.g. different key and number ranges for the respective platform). However, this is not a requirement for a functioning bidirectional replication.
A user of the mainframe system makes a change for a certain record. This change is "captured" in real time, transferred to the target system, and applied to the target database. This change on the target system, in turn, would also be captured and replicated back to the source system without special precautions. This processing would lead to circulating changes.
To prevent this, precautions must be taken to identify the change and not send it back to the source system. These precautions are available in the tcVISION solution and are defined as a "BackUpdateCheck" in the tcVISION repository at the connection between source and target. Depending on the source, indicators are defined to identify the change during the backward replication. These can be different indicators: User Name, Online Transaction ID, Job Name, PSB Name, and many more. The definition of the BackUpdateCheck is very easy to perform, as the following example shows:
The tcVISION Repository definition shows a replication between a VSAM file that is modified via CICS and a PostgreSQL table. A double click on the connection line between the input and output table shows the properties of the replication connection:
"BackUpdateCheck" was selected as replication type, and CICS Id, CICS Transaction, and CICS userid are offered as identification.
If a change in the VSAM file is found to be caused by tcVISION's CICS userid or by the tcVISION CICS transaction, the change will not be replicated further because the change in this example was triggered by PostgreSQL.
Changes that were prevented by a BackUpdateCheck are of course logged in a special log and are available for revision purposes at any time.
As mentioned before, there are also special BackUpdateChecks that can be important in conjunction with certain mainframe databases.
Mainframe database systems like ADABAS from Software AG or IDMS/DB from Computer Associates (BROADCOM) use internal keys to identify a record. For example, the ISN (Internal Sequence Number) in the case of ADABAS or DBKEY in IDMS. If a record is added to the replica in a replication scenario, the ISN number or DBKEY is not affected. After the INSERT has been performed on the mainframe (e.g. via tcVISION), the change is captured by tcVISION. However, since the INSERT on the mainframe has created a new ISN or DBKEY for the record, it must be transmitted to the target system via an UPDATE statement.
The implementation of this procedure is done via the tcVISION repository using the type "BackUpdateCheck for Inserts". This type of processing is also called “LOOPBACK processing”.
Bidirectional replication is an important part of enterprise initiatives in the context of application modernization or application migration to a new platform. Especially if a mainframe plays the central role in this scenario.
Our tcVISION solution is characterized by the fact that bidirectional replication can be implemented without problems and that traditional mainframe resources can be used as output targets without the need for additional software components.
Since the market launch of tcVISION many customers have successfully implemented their migration scenarios using tcVISION's bidirectional replication. Each of these scenarios had its own focus and characteristics which were finally solved and found their way into the tcVISION solution.