rattail.datasync.consumers

DataSync Consumers

class rattail.datasync.consumers.DataSyncConsumer(config, key, dbkey=None, runas=None, watcher=None)

Base class for all DataSync consumers.

begin_transaction()

Called just before the consumer is asked to process changes, possibly via multiple batches.

commit_transaction()

Called just after the consumer has successfully finished processing changes, possibly via multiple batches.

post_process_changes(session, changes)

Implement any “post-processing” logic here.

Common example would be to close DB connection for external system(s).

pre_process_changes(session, changes)

Implement any “pre-processing” logic here.

Common example would be to establish DB connection to some external system(s).

In some rare cases you may also need to “sort” the changes, to ensure they are processed in a particular sequence.

Returns:

If you needed to modify the list of changes, you can return the new list. If this method returns None (which the default logic does) then the original set of changes will be processed.

process_changes(session, changes)

Process (consume) a batch of changes.

rollback_transaction()

Called when any batch of changes failed to process.

setup()

This method is called when the consumer thread is first started.

setup_importer(importer, session, **kwargs)

Perform setup for the given importer, as prep for processing a batch of changes.

class rattail.datasync.consumers.DataSyncImportConsumer(*args, **kwargs)

Base class for any DataSync Consumer which leverages an Import Handler (and ultimately its Importers) to do the heavy lifting. See Data Import / Export for more about Importers etc.

handler

Reference to the import/export handler to be leveraged by this datasync consumer.

The handler is automatically instantiated when the consumer is instantiated, usually based on handler_key.

handler_key

This should be the “key” of the import handler to be leveraged by the datasync consumer. This will be a string designating the data target/source/direction, for example 'to_rattail.from_corepos_api.import'.

The handler itself is obtained by invoking rattail.app.AppHandler.get_import_handler().

Note that for now, handler_key is not strictly required, as handler_spec will be used as a fallback. But eventually that will likely change, with handler_spec being deprecated altogether and handler_key becoming required (unless you override get_handler()).

handler_spec

This attribute should be considered deprecated; please use handler_key instead.

If this attribute is used, it should be a “spec” string referencing the import handler class from which the importers should be obtained.

default_importers_only

Flag indicating whether the consumer should only use “default” importers from the handler, vs. using “all” importers, when processing changes. This flag is True by default, meaning only “default” importers will be used.

model_map

This is a dictionary which may be used to map model names (importer “keys”) between the host and local systems. Keys of the dictionary should be the model name as it comes from the host system, in the form of rattail.db.model.DataSyncChange.payload_type. Values of the dictionary should be the model name as it is found in the import handler, i.e. in its importer_keys() return value.

skip_local_models

This is a list of model names, as they are found in the import handler.

get_handler(spec=None)

Return the appropriate import handler instance.

If spec is provided, handler will conform to that. Otherwise the “designated” handler is got according to handler_key, by invoking rattail.app.AppHandler.get_import_handler(). Or if handler_key is not set, then for now, handler_spec may be used as fallback.

An error of some sort will be raised if no handler can be got.

Returns:

The import handler instance.

get_host_object(session, change)

You must override this, to return a host object from the given DataSyncChange instance. Note that the host object need not be normalized, as that will be done by the importer. (This is effectively the only part of the processing which is not handled by the importer.)

get_importers()

Returns a dictionary, keys of which are “model” names (e.g. 'Product') and values of which are Importer instances.

Note

The keys must ultimately align with the payload_type values, coming from the host system. Or at least that’s what will make life easy for the invoke_importer() method.

get_local_object(importer, key, host_data)

Fetch the “local” (destination) object for the given key.

invoke_importer(session, change)

For the given change, invoke the default importer behavior, if one is available.

process_change(session, importer, change=None, host_object=None, host_data=None)

Invoke the importer to process the given change / host record.

process_changes(session, changes)

Process all changes.

process_changes_proper(session, changes)

Actually process the given changes, leveraging importer(s) as much as possible.

process_deletion(session, importer, change)

Attempt to invoke the importer, to delete a local record according to the change involved.

class rattail.datasync.consumers.FromRattailConsumer(*args, **kwargs)

Base class for consumers which get their data from Rattail.

class rattail.datasync.consumers.NullTestConsumer(config, key, dbkey=None, runas=None, watcher=None)

Consumer which simply does ignores changes, letting DataSync think they were processed okay.

class rattail.datasync.consumers.ErrorTestConsumer(config, key, dbkey=None, runas=None, watcher=None)

Consumer which always raises an error when processing changes. Useful for testing error handling etc.