coherence-processing-pattern

Skip to end of metadata
Go to start of metadata
Deprecation Announcement
This release of the Coherence Incubator is now deprecated. For the latest supported release, help and documentation, please refer to the Coherence Incubator Project on java.net at http://cohinc.java.net.

The Processing Pattern Project
Provides an mechanism to perform distributed grid computing with Coherence.

Overview

This project provides a generalized approach for submitting and asynchronously processing (ie: executing) "work" in distributed system, based on Oracle Coherence.

The Processing Pattern advocates that;

  1. ProcessingSessions are used to submit "work" (typically regular objects) for asynchronous processing.
  2. A Submission Outcome must be returned after each submission to track the progress of said Submission.
  3. A Processing Pattern client will only have to concern itself with the above two concepts.
  4. A Processing Session can submit work and abandon it and later (or from a different session) reconnect to the submitted work.
  5. Dispatchers are responsible for routing Submissions for processing.
  6. A TaskDispatcher will dispatch special "work" which implements the Task interface.
  7. A Task that implements ResumableTask interface can checkpoint state (checkpoint as in saving intermediate state which can be retrieved when resuming), yield execution and resume.
  8. A TaskProcessor will process Tasks
  9. A TaskDispatchPolicy will determine how Tasks are assigned to TaskProcessors.

This implementation of the Processing Pattern additionally advocates that;

  1. Many Dispatchers may be configure to enable specific processing of different types of Submission.
  2. Dispatchers are consulted in configuration order for each Submission to determine suitability for routing. (This is called "dispatch chaining").
  3. Out of the box, three Dispatcher implementations are provided, one to Log the submissions, one to execute Java Runnables or Java Callables, and one (the DefaultTaskDispatcher) that dispatches Tasks to TaskProcessor implementations.
  4. The available TaskProcessors are configured in the cache-config file
  5. TaskProcessors can be of two different types, the GRID type instantiates automatically on each grid node. The SINGLE type is instantiated only on the node where it is configured.
  6. Tasks are mapped to TaskProcessors using TaskDispatchPolicies
  7. TaskDispatchPolicies include Round Robin, Random and Attribute Matching, the latter enabling an attribute supplied with a Submission to match against an attribute supplied with a TaskProcessorDefinition

This implementation of the Processing Pattern provides a simple and powerful extensible framework for the processing of "work" across a Coherence Cluster. Work can be submitted without needing to know the internals of Coherence or building custom configurations. Unlike the traditional hub-and-spoke / master-worker / broker-agent, the Processing Pattern implementation avoids the classic "single-point-of-failure", "single-point-of-dispatch", "single-point-of-contention" problems as the core state management and distribution is managed by the Data Grid.

Additionally, it provides runtime support to ResumableTasks to enable checkpointing of long running work and a suspend/resume model enabling long running tasks.

Further it supports execution of work based on a run anywhere in the grid paradigm, as well as a more controlled execution model where work is run on specific nodes, either running as grid nodes or over Coherence*Extend.

Revision History
Version 1.4.4 (changes since 1.4.3)

Version 1.4.4 is a patch release to upgrade dependencies to Coherence Common.

Version 1.4.3 (changes since 1.4.2)

Version 1.4.3 is a patch release to upgrade dependencies to Coherence Common.

Version 1.4.2 (changes since 1.4.1)

Version 1.4.2 is a patch release to upgrade dependencies to Coherence Common.

Version 1.4.1 (changes since 1.4.0)

Version 1.4.1 is a patch release to upgrade dependencies to Coherence Common and to support the up and coming Coherence release.

Version 1.4.0 (changes since 1.3.4)

The most significant upgrade for this release is the adoption of Coherence 3.7. Overtime this will allow the pattern to take advantage of the lastest features, like "lite transactions" to ensure atomicity of multi-cache updates, without the overhead of "heavy" transactions.

Legend | = Fixed | = Breaking Change / Deprecation | = Improved | = New
  • Removed the separate pof and non-pof cache configurations. Cache configuration files now use the system-wide defined serializer. This dramatically simplifies serialization configuration and halves the number of cache configuration files.
  • Explicitly specifies xsds in configuration files to strictly enforce document validation.
  • Hardened support for *Extend-based TaskProcessors.

Release Version: Version 1.4.4.39174
Release Date: October 31, 2012
Coherence Version: 3.7.1+
Target Platforms: Java Standard Edition 6+
Dependencies: Coherence Common
Download: coherence-processingpattern-1.4.4.39174.jar
MD5: f6d52854cf762fdc96a3bf958a8f4496
Source and
Java Doc: 		coherence-processingpattern-1.4.4.39174-src.zip

* Previous releases are available from the attachments page

Further Documentation
Screencasts

Coherence Screencasts
     Processing Pattern Overview
     Processing Pattern New Features Summer 2010

Examples

Example source code and uses of this pattern can be found in the Coherence Incubator Examples project.

Implementation Discussion
Dispatchers and Submissions
  1. In order to process work, one or more Dispatchers need to be registered.
  2. When a dispatcher is registered, each Submission will be handed to each Dispatcher in registration order until
    one dispatcher accepts the Submission.
  3. When a dispatcher has accepted the Submission a SubmissionOutcome is returned to the client.
  4. When the dispatcher, (or in the case of Task dispatching, the TaskProcessor), has processed the Submission the SubmissionOutcome is signaled and the result is available to the client.

The following diagram illustrates the main structure of the design of the Processing Pattern.

Task dispatching and execution of ResumableTasks
  1. ResumableTasks are handled by a TaskDispatcher that hooks in to the Dispatcher chain handling Submissions with a ResumableTask as the payload.
  2. A TaskProcessor executes the ResumableTask.
  3. Tasks are mapped to TaskProcessors using a configured TaskDispatchPolicy. The default TaskDispatchPolicy matches attributes supplied as part of the SubmissionConfiguration against attributes associated with a TaskProcessor
  4. When a ResumableTask executes, a TaskExecutionEnvironment is available to support checkpointing, progress reporting and determination of resumption.
  5. A resumable task can return a special Yield object to suspend execution.

The following diagram illustrates the structure of the Task Execution.

The following diagram illustrates the states a ResumableTask can be in.

Example

The following demonstrates a simple example of the Processing Pattern.

Labels:
project project Delete
current current Delete
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.

About Oracle | | Contact Us | Site Maps | Legal Notices | Terms of Use | Your Privacy Rights