wuttjamaican.batch

Batch Handlers

class wuttjamaican.batch.BatchHandler(config)

Base class and partial default implementation for batch handlers.

This handler class “works as-is” but does not actually do anything. Subclass must implement logic for various things as needed, e.g.:

• model_class

• init_batch()

• should_populate()

• populate()

• refresh_row()

add_row(batch, row)

Add the given row to the given batch.

This assumes a new row which does not yet belong to a batch, as returned by make_row().

It will add it to batch rows, call refresh_row() for it, and update the row_count.

property batch_type

Convenience property to return the batch type which the current handler is meant to process.

This is effectively an alias to batch_type.

consume_batch_id(session, as_str=False)

Fetch a new batch ID from the counter, and return it.

This may be called automatically from make_batch().

Parameters:

• session – Current db session.

• as_str – Indicates the return value should be a string instead of integer.

Returns:

Batch ID as integer, or zero-padded 8-char string.

describe_execution(batch, user=None, **kwargs)

This should return some text which briefly describes what will happen when the given batch is executed.

Note that Markdown is supported here, e.g.:

def describe_execution(self, batch, **kwargs):
    return """

This batch does some crazy things!

**you cannot possibly fathom it**

here are a few of them:

- first
- second
- third
"""

Nothing is returned by default; subclass should define.

Parameters:

• batch – The batch in question; eligible for execution.

• user – Reference to current user who might choose to execute the batch.

• **kwargs – Execution kwargs for the batch; should be similar to those for execute().

Returns:

Markdown text describing batch execution.

do_delete(batch, user, dry_run=False, progress=None, **kwargs)

Delete the given batch entirely.

This will delete the batch proper, all data rows, and any files which may be associated with it.

do_execute(batch, user, progress=None, **kwargs)

Perform the execution steps for a batch.

This first calls why_not_execute() to make sure this is even allowed.

If so, it calls execute() and then updates executed and executed_by on the batch, to reflect current time+user.

So, callers should use do_execute(), and subclass should override execute().

Parameters:

• batch – The batch to execute; instance of BatchMixin (among other classes).

• user – User who is executing the batch.

• progress – Optional progress indicator factory.

• **kwargs – Additional kwargs as needed. These are passed as-is to why_not_execute() and execute().

do_populate(batch, progress=None)

Populate the batch from initial data source(s).

This method is a convenience wrapper, which ultimately will call populate() for the implementation logic.

Therefore callers should use this do_populate() method, but subclass should override populate() instead (if needed).

See also should_populate() - you should check that before calling do_populate().

execute(batch, user=None, progress=None, **kwargs)

Execute the given batch.

Callers should use do_execute() instead, which calls this method automatically.

This does nothing by default; subclass must define logic.

Parameters:

• batch – A batch; instance of BatchMixin (among other classes).

• user – User who is executing the batch.

• progress – Optional progress indicator factory.

• **kwargs – Additional kwargs which may affect the batch execution behavior. There are none by default, but some handlers may declare/use them.

get_data_path(batch=None, filename=None, makedirs=False)

Returns a path to batch data file(s).

This can be used to return any of the following, depending on how it’s called:

• path to root data dir for handler’s batch_type

• path to data dir for specific batch

• path to specific filename, for specific batch

For instance:

# nb. assuming batch_type = 'inventory'
batch = handler.make_batch(session, created_by=user)

handler.get_data_path()
# => env/app/data/batch/inventory

handler.get_data_path(batch)
# => env/app/data/batch/inventory/03/7721fe56c811ef9223743af49773a4

handler.get_data_path(batch, 'counts.csv')
# => env/app/data/batch/inventory/03/7721fe56c811ef9223743af49773a4/counts.csv

Parameters:

• batch – Optional batch instance. If specified, will return path for this batch in particular. Otherwise will return the “generic” path for handler’s batch type.

• filename – Optional filename, in context of the batch. If set, the returned path will include this filename. Only relevant if batch is also specified.

• makedirs – Whether the folder(s) should be created, if not already present.

Returns:

Path to root data dir for handler’s batch type.

init_batch(batch, session=None, progress=None, **kwargs)

Initialize a new batch.

This is called automatically from make_batch().

Default logic does nothing; subclass should override if needed.

Note

Population of the new batch should not happen here; see instead populate().

make_batch(session, progress=None, **kwargs)

Make and return a new batch (model_class) instance.

This will create the new batch, and auto-assign its id value (unless caller specifies it) by calling consume_batch_id().

It then will call init_batch() to perform any custom initialization needed.

Therefore callers should use this make_batch() method, but subclass should override init_batch() instead (if needed).

Parameters:

• session – Current db session.

• progress – Optional progress indicator factory.

• **kwargs – Additional kwargs to pass to the batch constructor.

Returns:

New batch; instance of model_class.

make_row(**kwargs)

Make a new row for the batch. This will be an instance of __row_class__.

Note that the row will not be added to the batch; that should be done with add_row().

Returns:

A new row object, which does not yet belong to any batch.

property model_class

Reference to the batch data model class which this batch handler is meant to work with.

This is expected to be a subclass of BatchMixin (among other classes).

Subclass must define this; default is not implemented.

populate(batch, progress=None)

Populate the batch from initial data source(s).

It is assumed that the data source(s) to be used will be known by inspecting various properties of the batch itself.

Subclass should override this method to provide the implementation logic. It may populate some batches differently based on the batch attributes, or it may populate them all the same. Whatever is needed.

Callers should always use do_populate() instead of calling populate() directly.

refresh_row(row)

Update the given batch row as needed, to reflect latest data.

This method is a bit of a catch-all in that it could be used to do any of the following (etc.):

• fetch latest “live” data for comparison with batch input data

• (re-)calculate row values based on latest data

• set row status based on other row attributes

This method is called when the row is first added to the batch via add_row() - but may be called multiple times after that depending on the workflow.

should_populate(batch)

Must return true or false, indicating whether the given batch should be populated from initial data source(s).

So, true means fill the batch with data up front - by calling do_populate() - and false means the batch will start empty.

Default logic here always return false; subclass should override if needed.

why_not_execute(batch, user=None, **kwargs)

Returns text indicating the reason (if any) that a given batch should not be executed.

By default the only reason a batch cannot be executed, is if it has already been executed. But in some cases it should be more restrictive; hence this method.

A “brief but descriptive” message should be returned, which may be displayed to the user e.g. so they understand why the execute feature is not allowed for the batch. (There is no need to check if batch is already executed since other logic handles that.)

If no text is returned, the assumption will be made that this batch is safe to execute.

Parameters:

• batch – The batch in question; potentially eligible for execution.

• user – User who might choose to execute the batch.

• **kwargs – Execution kwargs for the batch, if known. Should be similar to those for execute().

Returns:

Text reason to prevent execution, or None.

The user interface should normally check this and if it returns anything, that should be shown and the user should be prevented from executing the batch.

However do_execute() will also call this method, and raise a RuntimeError if text was returned. This is done out of safety, to avoid relying on the user interface.