Using the Object Synchronization Feature

Introduction to Object Synchronization

Object synchronization is a set of tools available with InterSystems IRIS® objects that allows application developers to set up a mechanism to synchronize databases on “occasionally connected” systems. By this process, each database updates its objects. Object synchronization offers complementary functionality to InterSystems IRIS system tools that provide high availability. Object synchronization is not designed to provide support for real-time updates; rather, it is most useful for a system that needs updates at discrete intervals.

For example, a typical object synchronization application would be in an environment where there is a master copy of a database on a central server and secondary copies on client machines. Consider the case of a sales database, where each sales representative has a copy of the database on a laptop computer. When Mary, a sales representative, is off site, she makes updates to her copy of the database. When she connects her machine to the network, the central and remote copies of the database are synchronized. This can occur hourly, daily, or at any interval.

Object synchronization between two databases involves updating each of them with data from the other. However, InterSystems IRIS does not support bidirectional synchronization as such. Rather, updates from one database are posted to the other; then updates are posted in the opposite direction. For a typical application, if there is a main database and one or more local databases (as in the previous sales database example), it is recommended that updates are from the local to the main database first, and then from the main database to the local one.

For object synchronization, the idea of client and server is by convention only. For any two databases, you can perform bidirectional updates; if there are more than two databases, you can choose what scheme you use to update all of them (such as local databases synchronizing with a main database independently).

This section addresses the following topics:

• The GUID

• How updates work

• The SyncSet and SyncTime objects

The SyncSet and SyncTime Objects

When two databases are to be synchronized, each has transactions in it that the other lacks. This is illustrated in the following diagram:

Two Unsynchronized Databases

Here, database A and database B have been synchronized at transaction 536 for database A and transaction 112 for database B. The subsequent transactions for each database need to be updated from each to the other. To do this, InterSystems IRIS uses what is called a SyncSet object. This object contains a list of transactions that are used to update a database. For example, when synchronizing database B with database A, the default contents of the SyncSet object are transactions 547, 555, 562, and 569. Analogously, when synchronizing database A with database B, the default contents of the SyncSet object are transactions 117, 124, 130, and 136. (The transactions do not use a continuous set of numbers, because each transaction encapsulates multiple inserts, updates, and deletes — which themselves use the intermediate numbers.)

Each database holds a record of its synchronization history with the other. This record is called a SyncTime table. For database, its contents are of the form:

Database     Namespace     Last Transaction Sent     Last Transaction Received
------------------------------------------------------------------------------
B            User          536                       112

Note:

The numbers associated with each transaction do not provide any form of a time stamp. Rather, they indicate the sequence of filing for transactions within an individual database.

Once database B has been synchronized with database A, the two databases might appear as follows:

Two Databases, Where One Has Been Synchronized with the Other

Because the transactions are being added to database B, they result in new transaction numbers in that database.

Analogously, the synchronization of database B with database A results in 117, 124, 130, and 136 being added to database A (and receiving new transaction numbers there):

Two Synchronized Databases

Note that the transactions from database B that have come from database A (140 through 162) are not updated back to database A. This is because the update from B to A uses a special feature that is part of the synchronization functionality. It works as follows:

1. Each transaction in a database is labeled with what can be called “a database of origin.” In this example, transaction 140 in database B would be marked as originating in database A, while its transaction 136 would be marked as originating in itself (database B).

2. The SyncSet.AddTransactions() method, which bundles a set of transactions for synchronization, allows you to exclude transactions that originate in a particular database. Hence, when updating from B to A, AddTransactions() excludes all transactions that originate in database A — because those have already been added to the transaction list for database B.

This functionality prevents creating infinite loops in which two databases continually update each other with the same set of transactions.

Modifying the Classes to Support Synchronization

Object synchronization requires that the sites have data with matching sets of GUIDs. If you are starting with an already existing database that does not yet have GUIDs assigned for its records, you need to assign a GUID to each instance (record) in the database, and then make sure there are matching copies of the database on each site. In detail, the process is:

1. For each class being synchronized, set the value of the OBJJOURNAL parameter to 1.

   Parameter OBJJOURNAL = 1;

   

   This activates the logging of filing operations (that is, insert, update, or delete) within each transaction; this information is stored in the ^OBJ.JournalT global. An OBJJOURNAL value of 1 specifies that the property values that are changed in filing operations are stored in the system journal file; during synchronization, data that needs to be synchronized is retrieved from that file.

   Note:

   OBJJOURNAL can also have a value of 2, though the possible use of this value is restricted to special cases. It is never for classes using the default storage mechanism (%Storage.Persistent). A value of 2 specifies that property values that are changed in filing operations are stored in the ^OBJ.Journal global; during synchronization, data that needs to be synchronized is retrieved from that global. Also, storing information in the global increases the size of the database very quickly.

2. Optionally also set the value of the JOURNALSTREAM parameter to 1.

   Parameter JOURNALSTREAM = 1;

   

   By default, object synchronization does not support synchronization of file streams. The JOURNALSTREAM parameter controls whether or not streams are journaled when OBJJOURNAL is true:

   • If JOURNALSTREAM is false and OBJJOURNAL is true, then objects are journaled but the streams are not.

   • If JOURNALSTREAM is true and OBJJOURNAL is true, then streams are journaled. Object synchronization tools will process journaled streams when the referencing object is processed.

3. For each class being synchronized, set the value of its GUIDENABLED parameter to 1; this tells InterSystems IRIS to allow the class to be stored with GUIDs.

   Parameter GUIDENABLED = 1;

   

   Note that if this value is not set, the synchronization does not work properly. Also, you must set GUIDENABLED for serial classes, but not for embedded objects.

4. Recompile the class.

5. For each class being synchronized, give each object instance its own GUID by running the AssignGUID() method:

    Set Status = ##class(%Library.GUID).AssignGUID(className,displayOutput)

   

   where:

   • className is the name of class whose instances are receiving GUIDs, such as "Sample.Person".

   • displayOutput is an integer where zero specifies that no output is displayed and a nonzero value specifies that output is displayed.

   The method returns a %StatusOpens in a new tab value, which you should check.

6. Put a copy of the database on each site.

Performing the Synchronization

This section describes how to perform the synchronization. The database providing the updates is known as the source database; and the database receiving the updates is the target database. To perform the actual synchronization, the process is:

1. Each time you wish to synchronize the two databases, go to the instance with the source database. On the source database, create a new SyncSet using the %New() method of the %SYNC.SyncSet class:

    Set SrcSyncSet = ##class(%SYNC.SyncSet).%New("unique_value")

   

   The integer argument to %New(), unique_value, should be an easily identified, unique value. This ensures that each addition to the transaction log on each site can be differentiated from the others.

2. Call the AddTransactions() method of the SyncSet instance:

    Do SrcSyncSet.AddTransactions(FirstTransaction,LastTransaction,ExcludedDB)

   

   Where:

   • FirstTransaction is the first transaction number to synchronize.

   • LastTransaction is the last transaction number to synchronize.

   • ExcludedDB specifies a namespace within a database whose transactions are not included in the SyncSet.

   This method collects the synchronization data and puts it in a global, ready for export.

   Or, to synchronize all transactions since the last synchronization, omit the first and second arguments:

    Do SrcSyncSet.AddTransactions(,,ExcludedDB)

   

   This gets all transactions, beginning with the first unsynchronized transaction to the most recent transaction. The method uses information in the SyncTime table to determine the values.

   ExcludedDB is a $LIST created as follows:

    Set ExcludedDB = $ListBuild(GUID,namespace)

   

   Where:

   • GUID is the system GUID of the target system. This value is available through the %SYS.System.InstanceGUIDOpens in a new tab class method; to invoke this method, use the ##class(%SYS.System).InstanceGUID() syntax.

   • namespace is the namespace on the target system.

3. Call the ErrCount() method to determine how many errors were encountered. If there have been errors, the SyncSet.Errors query provides more detailed information.

4. Export the data to a local file using the ExportFile() method:

    Do SrcSyncSet.ExportFile(file,displaymode,bUpdate)

   

   Where:

   • file is the file to which the transactions are being exported; it is a name with a relative or absolute path.

   • displaymode specifies whether or not the method writes output to the current device. Specify “d” for output or “-d” for no output.

   • bUpdate is a boolean value that specifies whether or not the SyncTime table is updated (where the default is 1, meaning True). It may be helpful to explicitly set this to 0 at this point, and then set it to 1 after the source receives assurance that the target has indeed received the data and performed the synchronization.

5. Move the exported file from the source machine to the target machine.

6. Create a SyncSet object on the target machine using the SyncSet.%New() method. Use the same value for the argument of %New() as on the source machine — this is what identifies the source of the synchronized transactions.

7. Read the SyncSet object into the InterSystems IRIS instance on the target machine using the Import() method:

    Set Status = TargetSyncSet.Import(file,lastSync,maxTS,displaymode,errorlog,diag)

   

   Where:

   • file is the file containing the data for import.

   • lastSync is the last synchronized transaction number (default from synctime table).

   • maxTS is the last transaction number in the SyncSet object.

   • displaymode specifies whether or not the method writes output to the current device. Specify “d” for output or “-d” for no output.

   • errorlog provides a repository for any error information (and is called by reference to provide information for the application).

   • diag provides more detailed diagnostic information about what is happening when importing

   This method puts data into the target database. It behaves as follows:

   1. If the method detects that the object has been modified on both the source and target databases since the last synchronization, it invokes the %ResolveConcurrencyConflict() callback method; like other callback methods, the content of %ResolveConcurrencyConflict() is user-supplied. (Note that this can occur if either the two changes both modified a common property or the two changes each modified non-intersecting sets of properties.) If the %ResolveConcurrencyConflict() method is not implemented, then the conflict remains unresolved.

   2. If, after the Import() method executes, there are unsuccessfully resolved conflicts, these remain in the SyncSet object as unresolved items. Be sure to take the appropriate action regarding the remaining conflicts; this may involve resolution, leaving the items in an unresolved state, and so on.

   Important:

   The Import() method returns a status value but that status value simply indicates that the method completed without encountering an error that prevented the SyncSet from being processed. It does not indicate that every object in the SyncSet was processed successfully without encountering any errors. For information on synchronization error reporting, see Import() in the class reference for %SYNC.SyncSet.

8. Once the first database updates the second database, perform the same process in the other direction so that the second database can update the first one.

Translating Between GUIDs and OIDs

To determine the OID of an object from its GUID or vice versa, there are two methods available:

• %GUIDFind() is a class method of the %GUIDOpens in a new tab class that takes a GUID of an object instance and returns the OID associated with that instance.

• %GUID() is a class method of the %PersistentOpens in a new tab class that takes an OID of an object instance and returns the GUID associated with that instance; the method can only be run if the GUIDENABLED parameter is TRUE for the corresponding class. This method dispatches polymorphically and determines the most specific type class if the OID does not contain that information. If the instance has no GUID, the method returns an empty string.

Manually Updating a SyncTime Table

To perform a manual update on the SyncTime table for a database, invoke the SetlTrn() method, which sets the last transaction number:

 Set Status=##class(%SYNC.SyncTime).SetlTrn(syncSYSID, syncNSID, ltrn)

Where:

• syncSYSID is the system GUID of the target system. This value is available through the %SYS.System.InstanceGUIDOpens in a new tab class method; to invoke this method, use the ##class(%SYS.System).InstanceGUID() syntax.

• syncNSID is the namespace on the target system, which is held in the $Namespace variable.

• ltrn is the highest transaction number known to have been imported. You can get this value by invoking the GetLastTransaction() method of the SyncSet.

The SetlTrn() method sets the highest transaction number synchronized on the target system instead of the default behavior (which is to set the highest transaction number exported from the source system). Either approach is fine and is a choice available during application development.