Add Custom Rattail <-> Rattail Import/Export

If you have multiple "Poser" app nodes running then you likely will need to keep data in sync between them, to some extent. Most of the work has been done for you already; however since you probably have added some custom tables etc. to the DB, you must likewise add some custom importer logic before your custom tables are fully supported.

Importer Definitions

First you should make sure to define all custom "model" (table) importers in e.g. ~/src/poser/poser/importing/model.py. Remember, these are only responsible for defining the "target" side of the importer logic, irrespective of the data "source".

Then you must declare "proper" importers for each table desired, in e.g. ~/src/poser/poser/importing/poser.py. These will reference the "model" importers you just created, and may add any special logic needed when the data source is another Poser node (as opposed to any other system).

Import Handler Definitions

Then you must define the custom handlers, for both import and export use cases. In other words both of the following would ideally be supported:

• Poser (other) -> Poser (local) aka. "import"

• Poser (local) -> Poser (other) aka. "export"

These again will be defined in e.g. ~/src/poser/poser/importing/poser.py and again the goal is to specify only the custom parts.

The most common use case (presumably) is one where the set of tables to be synced is the same in either direction, in other words "import vs. export" is just a matter of perspective and the logic should be identical otherwise. However it is not uncommon, for certain nodes to be considered (more) "authoritative" for some tables, in which case import vs. export may in fact differ in terms of which tables should be synced. This often happens when there are different "node types" within the overall system.

In either case the approach is basically the same. You first define a "mixin" class which does nothing other than declare a set of importers which should be effective in either direction (import or export). Then you define a custom import handler, and a custom export handler; each of which will inherit from the mixin class in addition to the "true" base class.

Below is an example, where all the same tables are synced in either direction. However you can can modify the get_importers() method of either handler, to add extra importers which should be "one direction only" etc. Again, note that the mixin class should only declare those importers which should be invoked in both directions.

   1 from rattail.importing.rattail import FromRattailToRattailImport, FromRattailToRattailExport
   2 
   3 
   4 class FromPoserToPoserMixin(object):
   5     """
   6     Common "mixin" class for Poser <-> Poser import/export handlers
   7     """
   8 
   9     def add_poser_importers(self, importers):
  10 
  11         # core-extension models
  12         importers['PoserCustomer'] = PoserCustomerImporter
  13 
  14         # custom models
  15         importers['Foo'] = FooImporter
  16         importers['Bar'] = BarImporter
  17 
  18         return importers
  19 
  20 
  21 class FromPoserToPoserImport(FromRattailToRattailImport, FromPoserToPoserMixin):
  22     """
  23     Handles the Poser (other) -> Poser (local) import
  24     """
  25 
  26     def get_importers(self):
  27         importers = super(FromPoserToPoserImport, self).get_importers()
  28         self.add_poser_importers(importers)
  29         return importers
  30 
  31 
  32 class FromPoserToPoserExport(FromRattailToRattailExport, FromPoserToPoserMixin):
  33     """
  34     Handles the Poser (local) -> Poser (other) export
  35     """
  36 
  37     def get_importers(self):
  38         importers = super(FromPoserToPoserExport, self).get_importers()
  39         self.add_poser_importers(importers)
  40         return importers

Config Defaults

The last piece is to make sure Rattail knows to use your custom handlers instead of the default handlers. It is of course possible to e.g. add some values to config file to accomplish this. However the more typical approach is to define the "config defaults" directly within Poser app code.

Usually the place to do this is in ~/src/poser/poser/config.py although this assumes you already have a custom ConfigExtension defined there. Assuming that's the case, something like the following would register your custom import/export handlers:

   1 from rattail.config import ConfigExtension
   2 
   3 class PoserConfig(ConfigExtension):
   4     """
   5     Custom config extension for Poser
   6     """
   7     key = 'poser'
   8 
   9     def configure(self, config):
  10 
  11         # import handlers
  12         config.setdefault('rattail.importing', 'rattail.handler', 'poser.importing.poser:FromPoserToPoserImport')
  13         config.setdefault('rattail.exporting', 'rattail.handler', 'poser.importing.poser:FromPoserToPoserExport')

Command Line

With all the above in place, you can now run the typical commands and your extra tables should be supported:

cd /srv/envs/poser

# to import from "other" to local DB
sudo -u rattail bin/rattail -c app/quiet.conf -P import-rattail --dbkey other --dry-run

# to export from local to "other" DB
sudo -u rattail bin/rattail -c app/quiet.conf -P export-rattail --dbkey other --dry-run