PRODUCTS AND SERVICES INDUSTRIES SUPPORT PARTNERS COMMUNITIES ABOUT
  The Coherence Incubator
  Push Replication Pattern 2.1.1
 Browse Space
Version 4 by Brian Oliver
on Nov 10, 2008 14:32.

<< Changes from 3 to 4
compared with
Current by Rob Misek
on Aug 12, 2009 10:14.

 
Key
These lines were removed. This word was removed.
These lines were added. This word was added.

View page history


There are 1 changes. View first change.

  {note}This documentation applies to the Push Replication Pattern 2.1.1. The latest Push Replication Pattern documentation is available is [here|Push Replication Pattern].{note}
  {note}This documentation applies to the Push Replication Pattern 2.1.1. The latest Push Replication Pattern documentation is available [here|Push Replication Pattern].{note}
  
 {section}
 {column:width=50%}
 h3. The Push Replication Pattern
 This is an {color:green}*example implementation*{color} of Push Replication between two or more Coherence data grids.
  
 *The Push Replication Pattern advocates that;*
 # _Operations_ {color:gray}(such as insert, update and delete){color} occurring on _Data_ in one _Location_ should be *pushed* to one or more _Devices_, each _Device_ being capable of *optimistically replicating* the said _Operations_ {color:gray}(in the order in which the said _Operations_ originally occurred){color} in another _Receiving_ _Location_.
 # If a _Receiving_ _Location_ is unavailable or uncontactable for some reason, the _Operations_ to be replicated at the said _Location_ will be queued and executed {color:gray}(in the original order){color} at a later point in time.
  
 *This implementation of the Push Replication Pattern additionally advocates that;*
 # The _Data_ on which _Operations_ occur are standard Coherence Cache _Entries_.
 # The _Operations_ are called _EntryOperations_.
 # A _Location_ is a Coherence Cluster.
 # A _Location_ may act as both a sender of _Operations_ and receiver of _Operations_. That is, multi-way multi-location push replication is permitted.
 # A _Device_ is a class that implements the {{BatchPublisher}} interface.
 {column}
  
 {column:width=50%}
 h3. Outline
 || *Project Lead:* | Brian Oliver, Oracle |
 || *Release Name:* | Version 2.1.1: October 14th, 2008 |
 || *Target Platforms:* | Java Standard Edition 5+ |
 || *Requires Coherence Version:* | 3.3.1 or 3.4.0 |
 || *Other Dependencies:* | [coherence-common-1.1.1|Coherence Common] \\ [coherence-commandpattern-2.1.1|Command Pattern] \\ [coherence-messagingpattern-2.1.1|Store And Forward Messaging Pattern]|
 || *Download:* | [^coherence-pushreplicationpattern-2.1.1.jar] \\MD5:d86b2e72eb6d9b815ae340ba85dc1bba |
 || *Source Code and Documentation:* | [^coherence-pushreplicationpattern-2.1.1-src.zip] \\MD5:4d521be40e831171831abee0b68535b9 |
  
 h3. Rationale
 The purpose of this pattern is to provide extensible, flexible, high-performance, highly-available and scalable infrastructure to support the replication of _EntryOperations_ occurring in one Coherence Cluster to one or more possibly globally distributed other Coherence Clusters.
  
 While this naturally forms a "hub-and-spoke" architecture of federated Coherence Clusters, by configuring each Coherence Cluster as a "hub" and a "spoke", multi-way push replication may be achieved.
 {column}
 {section}
  
 h3. Implementation Discussion
  
 {tip:title=Before you start}
 The Push Replication Pattern makes extensive use of the [Store and Forward Messaging Pattern] as infrastructure to capture {{EntryOperations}} and deliver them in-order of occurrence to {{BatchPublishers}}. Consequently it is advised that you make yourself familiar with the [Store and Forward Messaging Pattern] (and it's implementation) to understand the implementation of this pattern.
 {tip}
  
 The Push Replication Pattern is an extension of the [Store and Forward Messaging Pattern]. The implementation provides several new Coherence {{CacheStore}} implementations, namely the {{PublishingCacheStore}} {color:gray}(for "hub-and-spoke" push replication){color} and the {{SafePublishingCacheStore}} {color:gray}(for "multi-way" push replication){color}, that are used to capture cache mutation operations (like insert, update and delete) on cache entries.
  
 After such operations are captured {color:gray}(represented as {{EntryOperations}}){color} they are sent to the messaging layer (implemented using the [Store and Forward Messaging Pattern]) for distribution to specialized {{TopicSubscriptions}} called {{PublishingSubscriptions}}.
  
 When {{PublishingSubscriptions}} receive the {{EntryOperations}} from the messaging layer, an internal event {color:gray}(captured by a backing-map-listener executing in the same JVM in which the {{PublishingSubscription}} is being managed by Coherence){color} signals to a {{PublishingService}} to commence publishing the currently queued {{EntryOperations}} with a {{BatchPublisher}} implementation {color:gray}(provided by the said {{PublishingSubscription}} ){color}. How the ultimate publishing occurs depends on the implementation of the {{BatchPublisher}}.
  
 The following diagram provides an overview of the push replication process through the various layers of the system.
 !process-flow.png!
  
 The following section provides an example use of the push replication pattern, including establishing publishers.
  
 h3. Example
  
 *Step 1: Configuring a Cache (with a {{PublishingCacheStore}})*
  
 As push replication occurs on a cache-by-cache basis, the first step in making use of this pattern is configuring your application cache(s) to use an appropriate {{PublishingCacheStore}} implementation. For the "hub-and-spoke" model, where one cluster is a "master" and the other clusters are "spokes", you should use a {{PublishingCacheStore}}. For multi-way push replication, your cache(s) should be configured to use a {{SafePublishingCacheStore}}.
  
 By default, the Push Replication cache configuration {color:gray}({{coherence-pushreplicationpattern-[pof-]cache-config.xml}}){color} defines a "publishing-*" mapping to a {{distributed-scheme}} that is configured to use a {{PublishingCacheStore}}. Consequently any cache name prefixed with "publishing-" will automatically support Push Replication.
  
 {code:java}
 String cacheName = "publishing-cache";
  
 NamedCache namedCache = CacheFactory.getCache(cacheName);
 {code}
  
 When your application first accesses a "publishing-*" based scheme {color:gray}(which in turn creates the {{PublishingCacheStore}}){color}, the underlying messaging infrastructure will be created to support Push Replication. This includes establishing a {{Topic}} that is named exactly the same as your cache. In this case, a {{Topic}} called "publishing-cache" will be created in the messaging layer.
  
 From here on any operation that mutates the "publishing-cache" will have the said operation captured and sent {color:Gray}(as an {{EntryOperation}}){color} to the "publishing-cache" {{Topic}}, after which the operation will be forwarded in-order to the {{Subscriptions}} on the said {{Topic}}.
  
 *Step 2: Registering a {{BatchPublisher}} with a {{PublishingSubscription}} to perform Push Replication*
  
 Until a {{PublishingSubscription}} is registered on a {{Topic}}, no publishing will occur. To create a {{PublishingSubscription}} for a {{Topic}} we use the [Store And Forward Messaging Pattern] layer.
  
 {code:java}
 MessagingSession messagingSession = DefaultMessagingSession.getInstance();
  
 PublishingSubscription publishingSubscription = new DefaultPublishingSubscription(cacheName, "my-publishing-subscription", new BatchPublisherAdapter(new StdErrPublisher(), 5000, 10)));
  
 messagingSession.createTopicSubscription(publishingSubscription);
 {code}
  
 A standard implementation of a {{PublishingSubscription}} is provided by the {{DefaultPublishingSubscription}} class. The first parameter is the name of the cache {color:gray}(which is used for the {{Topic}}){color}, the second is the name of the subscription {color:gray}(all subscriptions may have a unique identity){color} and the last parameter specifies the {{BatchPublisher}} implementation to use.
  
 There are essentially two forms of publishers; those that support "batching" and those that don't. By default, the Push Replication pattern only works with {{BatchPublishers}}. Consequently to use a non-batching publisher {color:gray}(ie: an implementation of the {{Publisher}} interface){color}, an adapter has been provided. The package {{com.oracle.coherence.pushreplication.publishers}} provides numerous implementations of {{Publishers}} and {{BatchPublishers}} together with adapters to convert between them.
  
 *Step 3: Using a {{publishing-*}} cache.*
  
 Once you've established appropriate {{PublishingSubscriptions}} for each cache on which you would like to Push Replicate, you may simply use the said cache(s) as regular caches (as they are!).
  
 {code:java}
 namedCache.put("australian-welcome", "Gudday");
 namedCache.put("regular-welcome", "Hello");
 namedCache.put("french-welcome", "Bonjour");
 {code}
  
 h3. Frequently Asked Questions
  
 (*b) *How can the Push Replication be monitored on a subscription-by-subscription basis?*
  
 Like the [Command Pattern] and [Store And Forward Messaging Pattern]s, this pattern is JMX ready. By simply enabling JMX on the Coherence Cluster, each of the {{PublishingServices}} will be presented in the JMX tree, detailing current replication state and statistics.
  
 (*b) *How do I enable Push Replication to one or more (remote) Coherence Clusters?*
  
 In order to publish to a remote cluster, you must;
 # Configure and enable one or more proxies on the remote cluster(s).
 # Ensure that the members of the remote cluster(s) have the Push Replication Pattern (and dependencies) in the class path
 # Configure and enable one or more Remote Invocation Services in the "hub" with the addresses of the remote cluster proxy members (of it you're using Coherence 3.4+, use an {{AddressProvider}}). An example scheme is defined in the {{coherence-pushreplicationpattern-cache-config.xml}} file.
 # Create appropriate {{RemoteInvocationPublisher}} subscriptions for each cache (as follows).
  
 {code:java}
 messagingSession.createTopicSubscription(
  new DefaultPublishingSubscription(cacheName,
  "my-remote-subscription",
  new RemoteInvocationPublisher("RemoteSiteInvocationService",
  new BatchPublisherAdapter(new CachePublisher(cacheName)),
  10000, 100, 10000)));
 {code}
  
 (*b) *How are conflicts resolved when replicating EntryOperations in remote clusters?*
  
 The current implementation assumes it is *safe* to perform optimistically perform all {{EntryOperations}} remotely. The next release of this pattern will demonstrate the use of a pluggable {{ConflictResolver}} that may be used to reconcile and resolve all conflicts.
  
 h3. References and Additional Information
  
 * The Coherence Incubator - [Store And Forward Messaging Pattern]