Push Replication Pattern

Skip to end of metadata
Go to start of metadata

The Push Replication Pattern

This is an example implementation of Push Replication pattern.

The Push Replication Pattern advocates that;

  1. Operations (such as insert, update and delete) occurring on Data as one Site should be pushed using one or more Publishers to associated Destinations.

Where;

  1. A Publisher is responsible for optimistically reproducing the said Operations (typically in the order in which they originally occurred) at the associated Destination.
  2. If a Destination becomes unavailable for some reason, the Operations to be reproduced should be queued so that they may be reproduced (in the original order in which they occurred) at a later point in time.

This implementation of the Push Replication Pattern additionally advocates that;

  1. The Data on which Operations occur are Coherence Cache Entries.
  2. Operations to be reproduced include inserts, updates, and deletions of Cache Entries*. This includes NamedCache put, remove method calls together with side-effects created by invoking EntryProcessors.
  3. The Operations that are enqueued for reproduction are called Entry Operations. They contain the key, old and new values of the cache entries in Binary form.
  4. The Destination at which Entry Operations will be replicated/reproduced is dependent on the implementation of the associated Publisher
  5. Custom Publisher implementations may be implemented and configured to publish Entry Operations to any Destination.
  6. Typical Publishers and thus Destinations may be anyone of the following; a local cluster, a remote cluster, a file system, a database, an i/o stream, a logging system etc.
  7. A Cluster may act as both a sender of Operations and receiver of Operations. That is, multi-way multi-site push replication is permitted.

Rationale

The purpose of this pattern is to provide an extensible, high-performance, highly-available, general purpose scalable framework to support the reproducing Entry Operations occurring in one or more Caches in a Coherence Cluster to one or more possibly globally distributed Coherence Clusters, Caches or other Destinations.

Screencasts

Coherence Screencasts
     Push Replication Pattern Overview
     Push Replication Pattern - Auction Site Demo

Coherence 3.6 Support

Release Name: Version 3.0.3: November 22, 2010
Target Platforms: Java Standard Edition 6+
Requires Coherence Version: 3.6+
Dependencies: Coherence Common
Messaging Pattern *
Download: coherence-3.6-pushreplicationpattern-3.0.3.20019.jar
MD5:2c122ee7a18b92891478f96b1aef3964
Source Code and Documentation: coherence-3.6-pushreplicationpattern-3.0.3.20019-src.zip
MD5:24f16888fb7fb5ad6034251422e7edad
Build Instructions: Building the Coherence Incubator Projects

Coherence 3.5 Support

Release Name: Version 3.0.3: November, 2010
Target Platforms: Java Standard Edition 6+
Requires Coherence Version: 3.5.3 Patch 3+
Dependencies: Coherence Common
Messaging Pattern *
Download: coherence-3.5-pushreplicationpattern-3.0.3.20019.jar
MD5:2d7b559218e417739f437cae0b64672a
Source Code and Documentation: coherence-3.5-pushreplicationpattern-3.0.3.20019-src.zip
MD5:edcc7af7c7dfab080b676c751c451c65
Build Instructions: Building the Coherence Incubator Projects

* if using the Coherence Push Replication Provider.

Previous Releases

The following releases are provided for historical purposes only.
Push Replication Pattern 2.6.1, Push Replication Pattern 2.6.0, Push Replication Pattern 2.5.0, Push Replication Pattern 2.4.0 , Push Replication Pattern 2.3.0, Push Replication Pattern 2.2.0, Push Replication Pattern 2.1.1

Dependencies

This project (like other Coherence Incubator projects) uses Apache Ivy for dependency specification and management. While a standard ivy.xml definition file ships with the source and documentation distribution, the following diagram visually indicates the current dependencies.

What's New?

This release of the Push Replication implementation encapsulates many significant changes since the release of Push Replication Pattern 2.6.0. While the approach and central concepts used in the implementation remain mostly unchanged, almost all of the interfaces and implementing classes have been refactored to achieve the goals of the release, namely to allow developers to;

a). Completely specify Push Replication configuration declaratively in a standard Coherence Cache Configuration file. For the most part this means that Push Replication may now be adopted without changing implementing application classes or developing initialization applications for the pattern.

b). Replace as needed the internal infrastructure Push Replication uses for "messaging" and managing queues when asynchronously publishing Entry Operations. The goal being to enable organizations to leverage existing messaging infrastructure for Push Replication. With in the implementation this infrastructure is now known as the "Push Replication Provider".

Changes made as a part of version 3.0.3:

  • Upgraded to use coherence-common-1.7.2 and optionally coherence-messagingpattern-2.7.2
  • Resolved INC-815 PushReplication can call ensureCluster on the wrong service leading to a deadlock.

Changes made as a part of version 3.0.2:

  • Upgraded to use coherence-common-1.7.2 and optionally coherence-messagingpattern-2.7.2
  • Fixed PushReplication where it can call ensureCluster on the wrong service leading to a deadlock.

The following changes have been made since the original Push Replication 3.0.0 release:

  • Resolved issue where ConflictResolvers were required to be serializable. This is no longer the case as we use ClassScheme<ConflictResolver>s instead (these are naturally serializable)
  • Added support for use of the cache-name macro parameter in local/remote cache names
  • Resolved issue where the JMX MBeans for PublishingServices where not being removed from local JMX registries when they moved from one node to another in the cluster.
  • Resolved issue where the JMX MBean "suspend" operation would not work during recovery of a PublishingService
  • Resolved issue where the first few updates on a cache when a cluster started would not be published with a Publisher.
  • Corrected misspelling of the get/setRemotePublisherScheme in the RemoteClusterPublisherScheme class
  • Resolved issue where numerous Remote Cluster/Cache publishers may indefinitely hold a lock on the local cluster configuration, thus making it impossible to start other local services and/or all extend members to connect to the cluster lazily.
  • Resolved issue where the use of a RemoteCachePublisher may cause infinite cluster publishing loops between clusters.

The following changes were made as part of in the original Push Replication 3.0.0 release:

  • Removed all uses of static classes. Push Replication now uses the Extensible Environments provided by Coherence Common 1.7.0.
  • Removed need to hard code individual site names in configuration. Instead site and cluster names are automatically detected based on cluster provided information.

Configuring the Push Replication Pattern

As mentioned above, this release provides developers with the ability to use xml declarations with in Coherence Cache Configuration files to configure how Push Replication operates. Like the processing pattern, this has been achieved through the introduction of the Push Replication Namespace, the reference documentation for which is available here.

The following outlines the xml configuration structure for the Push Replication Pattern and where changes typically occur with in a Coherence Cache Configuration document.

Legend:

Symbol Meaning
1 Exactly One
0..1 None or One
? One of
* Zero or more
+ One or more

Examples

Detailed example source code and uses of this pattern can now be found in the Coherence Incubator Examples project.

Supported Deployment Models

The following section outlines common deployment models supported by the Push Replication Pattern.

Each model additionally supports Conflict Resolution at the destination site through the specification of ConflictResolvers. The default conflict resolver (called a BruteForceConflictResolver) will simply overwrite the existing value - that is, last write wins.
We highly recommend that all clusters using Push Replication are uniquely identifiable. To achieve this, each cluster should be configured such that the combination of their site and cluster names are unique. To declare the name of the geographical site in which a cluster is located you can either use the Coherence system override property tangosol.coherence.site or configure the <cluster-config>. Likewise to declare the name of a cluster use either the Coherence system override property tangosol.coherence.cluster or again, configure the the <cluster-config>.

Frequently Asked Questions

What are the ordering characteristics of push replication?

Push Replication publishes information based on mutations, called Entry Operations, of cache entries. It relies on the Push Replication Provider to provide ordering of Entry Operations for publishing. This means that for a given entry in a cache, any updates made to that entry will be published in the same order, unless of course the publishers are using Publishing Transformers that may mutate the Entry Operations prior to publishing. The ordering of publication across different entries in a cache, or between updates to different caches is not enforced.

How can the Push Replication Publishers be monitored?

Like other patterns in the Incubator and Coherence itself, Push Replication supports monitoring and some management via JMX. By simply enabling JMX on the Coherence Cluster, each of the PublishingServices will be presented in the JMX tree, detailing current publishing state and statistics.

How do I enable Push Replication to one or more (remote) Coherence Clusters?

In order to publish to a remote cluster, you must;

  1. Configure and enable one or more proxies on the remote cluster(s).
  2. Ensure that the members of the remote cluster(s) have the Push Replication Pattern (and dependencies) in the class path
  3. Configure and enable one or more Remote Invocation Services in the "hub" with the addresses of the remote cluster proxy members (or if you're using Coherence 3.4+, use an AddressProvider). An example scheme is defined in the coherence-pushreplicationpattern-cache-config.xml file.
  4. Declare an appropriate RemoteClusterPublisherScheme or RemoteCachePublisherScheme for each cache.

How are conflicts resolved when publishing Entry Operations into caches in local or remote clusters?

The LocalCachePublisherSchemes and RemoteCachePublisherSchemes classes support the specification of a ConflictResolvers in their configuration. By specifying a ConflictResolver in your application configuration, Push Replication may then detect and appropriately resolve any underlying conflicts. When not specified, the BruteForceConflictResolver is used as the default.

References and Additional Information

Labels:
current current Delete
pattern pattern Delete
project project Delete
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.