PRODUCTS AND SERVICES INDUSTRIES SUPPORT PARTNERS COMMUNITIES ABOUT
  The Coherence Incubator
  Push Replication Pattern 2.3.0
 Browse Space
Version 18 by Brian Oliver
on Mar 30, 2009 11:01.

<< Changes from 17 to 18
compared with
Current by Rob Misek
on Aug 12, 2009 10:16.

 
Key
These lines were removed. This word was removed.
These lines were added. This word was added.

View page history


There are 1 changes. View first change.

  {note}This documentation applies to the Push Replication Pattern 2.3.0. The latest Push Replication Pattern documentation is available [here|Push Replication Pattern].{note}
  
 {section}
 {column:width=50%}
 h3. The Push Replication Pattern
 This is an {color:green}*example implementation*{color} of Push Replication pattern.
  
 *The Push Replication Pattern advocates that;*
 # _Operations_ {color:gray}(such as insert, update and delete){color} occurring on _Data_ in one _Location_ should be *pushed* using one or more _Publishers_ to an associated _Device_.
 # A _Publisher_ is responsible for *optimistically replicating* _Operations_ {color:gray}(in the order in which the said _Operations_ originally occurred){color} on or with the associated _Device_.
 # If a _Device_ is unavailable for some reason, the _Operations_ to be replicated using the associated _Publisher_ will be queued and executed {color:gray}(in the original order){color} at a later point in time.
  
 *This implementation of the Push Replication Pattern additionally advocates that;*
 # The _Data_ on which _Operations_ occur are standard Coherence Cache _Entries_.
 # The _Operations_ are called _EntryOperations_.
 # A _Location_ is a Coherence Cluster.
 # A _Location_ may act as both a sender of _Operations_ and receiver of _Operations_. That is, multi-way multi-location push replication is permitted.
 # A _Device_ may be anyone of the following; a local cluster, a remote cluster, a file system, a database, an i/o stream, a logging system etc.
  
 h3. Rationale
 The purpose of this pattern is to provide extensible, flexible, high-performance, highly-available and scalable infrastructure to support the replication of _EntryOperations_ occurring in one Coherence Cluster to one or more possibly globally distributed other Coherence Clusters.
  
 While this naturally forms a "hub-and-spoke" architecture of federated Coherence Clusters, by configuring each Coherence Cluster as a "hub" and a "spoke", multi-way push replication may be achieved.
 {column}
  
 {column:width=50%}
 h3. Outline
 || *Release Name:* | *{color:red}Version 2.3.0: March 30th, 2009{color}* |
 || *Target Platforms:* | Java Standard Edition 5+ |
 || *Requires Coherence Version:* | 3.3.1+ or 3.4+ |
 || *Dependencies:* | [Coherence Common 1.3.0] \\ [Command Pattern 2.3.0] \\ [Messaging Pattern 2.3.0]|
 || *Download:* | [^coherence-pushreplicationpattern-2.3.0.jar] \\MD5:553f5da91092208f855576d2888b7750 |
 || *Source Code and Documentation:* | [^coherence-pushreplicationpattern-2.3.0-src.zip] \\MD5:716beb52547c7b282040abc2c9cbda08 |
 || *Previous Releases:* | [Push Replication Pattern 2.2.0] \\ [Push Replication Pattern 2.1.1] |
  
 h3. Dependencies
 This project (like other Coherence Incubator projects) uses [Apache Ivy|http://ant.apache.org/ivy/] for dependency specification and management. While a standard {{ivy.xml}} definition file ships with the source and documentation distribution, the following diagram visually indicates the current dependencies.
  
 {center}
 !Incubator-Dependencies-PushReplication.png!
 {center}
 {column}
 {section}
  
 h3. What's New?
 The following changes have been made since the [Push Replication Pattern 2.2.0] release.
  
 * Migrated to use [Apache Ivy|http://ant.apache.org/ivy/] for dependency management, building and publishing artifacts
  
 * Added support for "draining" objects to be published (replicated) with specific Publishers. This allows users to "reset" and "discard" objects for sites that have been down a long time.
  
 * Refactored to use the new DelegatingMultiplexingBackingMapListener (from [Coherence Common 1.3.0]) and thus removed the custom PublishingSubscriptionBackingMapListener. This a). allows clean co-existance of the push-replication pattern, messaging pattern and command pattern potentially on the same cache (though no one has asked for this just yet) and b). simplifies the Push Replication cache configuration.
  
 * Resolved issue when PublishingService JMX mBeans were not being unregistered during scale-out,
 thus creating multiple mBean registrations in the JMX tree.
  
 h3. Implementation Discussion
  
 {tip:title=Before you start}
 The Push Replication Pattern makes extensive use of the [Messaging Pattern] as infrastructure to capture {{EntryOperations}} and deliver them in-order of occurrence to {{BatchPublishers}}. Consequently may be beneficial for you to get familiar with the [Messaging Pattern] (and it's implementation) to understand the implementation of this pattern.
 {tip}
  
 The Push Replication Pattern is an extension of the [Messaging Pattern]. The implementation provides several new Coherence {{CacheStore}} implementations, namely the {{PublishingCacheStore}} {color:gray}(for "hub-and-spoke" push replication){color} and the {{SafePublishingCacheStore}} {color:gray}(for "multi-way" push replication){color}, that are used to capture cache mutation operations (like insert, update and delete) on cache entries.
  
 After such operations are captured {color:gray}(represented as {{EntryOperations}}){color} they are sent to the messaging layer (implemented using the [Messaging Pattern]) for distribution to specialized {{TopicSubscriptions}} called {{PublishingSubscriptions}}.
  
 When {{PublishingSubscriptions}} receive the {{EntryOperations}} from the messaging layer, an internal event {color:gray}(captured by a backing-map-listener executing in the same JVM in which the {{PublishingSubscription}} is being managed by Coherence){color} signals to a {{PublishingService}} to commence publishing the currently queued {{EntryOperations}} with an associated {{BatchPublisher}} implementation. How the ultimate publishing occurs is entirely dependent on the implementation of the {{BatchPublisher}}.
  
 The following diagram provides an overview of the push replication process through the various layers of the system.
 !process-flow.png!
  
 h3. Supported Deployment Models
 The following section outlines common deployment models supported by the Push Replication Pattern.
  
 {tip}Each model additionally supports Conflict Resolution at the destination site through the specification of ConflictResolvers. The default conflict resolver (called a BruteForceConflictResolver) will simply overwrite the existing value - that is, last write wins.{tip}
  
 {section}
 {column:width=50%}
 h4. Active - Single Passive
 In the "Active - Single Passive" deployment model updates to data made in the active grid are are sent to the passive grid asynchronously and ordered per {{NamedCache}}.
  
 !Incubator-PushReplication-Active-Passive-One.png!
 {column}
 {column:width=50%}
 h4. Active - Multiple Passive
 In the "Active - Multiple Passive" deployment model updates to data made in the active grid are are sent to any number of passive grids asynchronously and ordered per {{NamedCache}}.
  
 !Incubator-PushReplication-Active-Passive-Many.png!
 {column}
 {section}
  
 {section}
 {column:width=50%}
 h4. Active - Active
 In the "Active - Active" deployment model updates to data made in either of the active grids are are sent to other active grid asynchronously and ordered per {{NamedCache}}.
  
 !Incubator-PushReplication-Active-Active.png!
 {center}
 {column}
 {column:width=50%}
 h4. Active - Active ^n^
 In the "Active - Active ^n^" deployment model updates to data made in any active grid are are sent all other active grids asynchronously and ordered per {{NamedCache}}.
  
 !Incubator-PushReplication-Active-Active-Many.png!
 {center}
 {column}
 {section}
  
  
  
 h3. Example
  
 The following section provides an example use of the push replication pattern, including establishing publishers.
  
 *Step 1: Configuring a Cache (with a {{PublishingCacheStore}})*
  
 As push replication occurs on a cache-by-cache basis, the first step in making use of this pattern is configuring your application cache(s) to use an appropriate {{PublishingCacheStore}} implementation. For the "hub-and-spoke" model, where one cluster is a "master" and the other clusters are "spokes", you should use a {{PublishingCacheStore}}. For multi-way push replication, your cache(s) should be configured to use a {{SafePublishingCacheStore}}.
  
 By default, the Push Replication cache configuration {color:gray}({{coherence-pushreplicationpattern-pof-cache-config.xml}}){color} defines a "publishing-*" mapping to a {{distributed-scheme}} that is configured to use a {{PublishingCacheStore}}. Consequently any cache name prefixed with "publishing-" will automatically support Push Replication.
  
 {code:java}
 String cacheName = "publishing-cache";
  
 NamedCache namedCache = CacheFactory.getCache(cacheName);
 {code}
  
 When your application first accesses a "publishing-*" based scheme {color:gray}(which in turn creates the {{PublishingCacheStore}}){color}, the underlying messaging infrastructure will be created to support Push Replication. This includes establishing a {{Topic}} that is named exactly the same as your cache. In this case, a {{Topic}} called "publishing-cache" will be created in the messaging layer.
  
 From here on any operation that mutates the "publishing-cache" will have the said operation captured and sent {color:Gray}(as an {{EntryOperation}}){color} to the "publishing-cache" {{Topic}}, after which the operation will be forwarded in-order to the {{Subscriptions}} on the said {{Topic}}.
  
 *Step 2: Registering a {{BatchPublisher}} to perform Push Replication*
  
 Until a {{BatchPublisher}} is registered for a cache configured with a {{PublishingCacheStore}}, no publishing will occur. To register a {{BatchPublisher}} to publish {{EntryOperations}}, you need to use a {{PushReplicationManager}}.
  
 {code:java}
 PushReplicationManager pushReplicationManager = DefaultPushReplicationMananger.getInstance();
  
 pushReplicationManager.registerBatchPublisher("publishing-cache", "stderr-publisher", new BatchPublisherAdapter(new StdErrPublisher()));
 {code}
  
 There are essentially two forms of publishers; those that support "batching" and those that don't. By default, the Push Replication pattern only works with {{BatchPublishers}}. Consequently to use a non-batching publisher {color:gray}(ie: an implementation of the {{Publisher}} interface){color}, an adapter has been provided. The package {{com.oracle.coherence.pushreplication.publishers}} provides numerous implementations of {{Publishers}} and {{BatchPublishers}} together with adapters to convert between them (as demonstrated in the example above)
  
 *Step 3: Using a {{publishing-*}} cache.*
  
 Once you've registered one or more {{BatchPublishers}} for each cache from which you would like to publish {{EntryOperations}}, you may simply use the said cache(s) as regular caches (as they are!).
  
 Hence the following code, registered with the above "stderr-publisher", would output the "put" operations to {{stderr}}.
  
 {code:java}
 namedCache.put("australian-welcome", "Gudday");
 namedCache.put("regular-welcome", "Hello");
 namedCache.put("french-welcome", "Bonjour");
 {code}
  
 h3. Frequently Asked Questions
  
 (*b) *How can the Push Replication Publishers be monitored?*
  
 Like the [Command Pattern] this pattern is JMX ready. By simply enabling JMX on the Coherence Cluster, each of the {{PublishingServices}} will be presented in the JMX tree, detailing current replication state and statistics.
  
 (*b) *How do I enable Push Replication to one or more (remote) Coherence Clusters?*
  
 In order to publish to a remote cluster, you must;
 # Configure and enable one or more proxies on the remote cluster(s).
 # Ensure that the members of the remote cluster(s) have the Push Replication Pattern (and dependencies) in the class path
 # Configure and enable one or more Remote Invocation Services in the "hub" with the addresses of the remote cluster proxy members (of it you're using Coherence 3.4+, use an {{AddressProvider}}). An example scheme is defined in the {{coherence-pushreplicationpattern-cache-config.xml}} file.
 # Register appropriate {{RemoteInvocationPublishers}} for each cache (as follows).
  
 {code:java}
 String remoteCacheName = cacheName; //we'll publish to the same cache name as here in the remote site
 pushReplicationManager.registerBatchPublisher(cacheName,
  "remote-site",
  new RemoteInvocationPublisher("RemoteSiteInvocationService", //the underlying service to use
  new BatchPublisherAdapter(new LocalCachePublisher(remoteCacheName)),
  false, //don't automatically start publishing
  10000, //delay between batches being published
  100, //maximum batch size
  10000, //restart delay if failure occurs
  5)); //maximum consecutive failures before suspension of publishing
 {code}
  
 (*b) *How are conflicts resolved when replicating EntryOperations in remote clusters?*
  
 The {{LocalCachePublisher}}, {{SafeLocalCachePublisher}} and {{RemoteCachePublisher}} classes support the specification of a {{CachePublisher.ConflictResolver}} implementation during construction. By implementing a {{CachePublisher.ConflictResolver}} your application may resolve any underlying Entry conflicts in an appropriate manner. If not specified, the {{LocalCachePublisher.BruteForceConflictResolver}} is used.
  
 h3. References and Additional Information
  
 * The Coherence Incubator - [Messaging Pattern]