Architecture

From SharemailWiki

Basic data flow

The DRRE is designed around the concept of data flow, which is the smallest unit of work that is required for a complete implementation of the DRRE.

Transformations with Drools

The real power of the DRRE lies in its ability to pass data through Drools rulesets for complex, rule-based transformations. Drools is capable of far more than simple, if-the-else rule processors or (for exmaple) XSLT. Drools also processes rules very, very fast in comparison with simpler methods. Functions in imported Java classes can also be called in Drools rules to enable complicated lookups and data constructs to be used.

Advanced data matching

The diagrams above show the DRRE doing simple data matching to determine whether data should be used to driver a create or update operation (or deletion) in the destination data. This simple matching looks for exact matches of on or more common attributes in the data being processed and data already in existence in the destination data. Sometimes this in not enough as data is rarely 100% "clean". "Dirty" data can be the result of typos, formatting errors or incompatibilities (e.g. dates) or the passage of time (e.g. moving house). The DRRE offers more sophisticated data matching to allow for inconsistencies in the data. This uses functions available in the Postgres database and this functionality is dependent upon it.