Data policy
The data policy of a BLISS session is defined by the session’s SCAN_SAVING object.
This object can be accessed from the BLISS shell by the global variable SCAN_SAVING that refers to current_session.scan_saving.
The SCAN_SAVING object is an instance of ParametersWardrobe which is a REDIS data structure. The object is instantiated during the setup phase of the BLISS session.
Custom data policy¶
Creating a custom data policy means deriving a class from bliss.scanning.scan_saving.BasicScanSaving:
class CustomScanSaving(BasicScanSaving):
DEFAULT_VALUES = {
# default and not removable values
"scan_number_format": "%04d",
...
# saved properties in Redis:
"_proposal": "",
...
}
# read only attributes implemented with python properties
PROPERTY_ATTRIBUTES = [
"proposal",
...
]
REDIS_SETTING_PREFIX = "custom_scan_saving"
SLOTS = ["_custom_attr"]
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self._custom_attr = None
def get(self):
try:
# calculate all parameters
except KeyError as keyname:
raise RuntimeError("Missing %s attribute in CustomScanSaving" % keyname)
A BLISS session needs to be configured to use this policy.
BasicScanSaving API¶
DEMO [1]: SCAN_SAVING
Out [1]: Parameters (default) -
.base_path = '/tmp/toto/'
.data_filename = 'data'
.user_name = 'guilloud'
.template = '{session}/'
.images_path_relative = True
.images_path_template = 'scan{scan_number}'
.images_prefix = '{img_acq_device}_'
.date_format = '%Y%m%d'
.scan_number_format = '%04d'
.session = 'cyril'
.date = '20190926'
.scan_name = 'scan name'
.scan_number = 'scan number'
.img_acq_device = '<images_* only> acquisition device name'
.writer = 'hdf5'
.creation_date = '2019-08-23-10:08'
.last_accessed = '2019-09-26-16:09'
-------------- --------- -----------------------
does not exist filename /tmp/toto/cyril/data.h5
exists root_path /tmp/toto/cyril/
-------------- --------- -----------------------
base_path corresponds to the top-level directory where scans are
stored. Then, template completes the path. It uses Python’s string
interpolation syntax to specify how to build the file path from key
values. Keys can be freely added. Key values can be numbers or
strings, or functions. In case of function key values, the function
return value is used.
SCAN_SAVING.get() performs template string interpolation and returns
a dictionary, whose key root_path is the final path to scan files.
Attributes¶
base_path: the highest level directory for the file path, e.g./datauser_name: the current Unix user namesession: current BLISS session nametemplate: defaults to{session}/.add(key, value): add a new key (string) to the SCAN_SAVING structure- value can be a scalar or a function
.get_path(): returns the build directory path where the data file will be saved..get(): evaluates template ; produces a dictionary with 5 keysroot_path:base_path+ interpolated templatedata_path: fullpath for the data file without the extension.images_path: path where image device should save (Lima)db_path_items: used to create parent node for publishing data via Rediswriter: Data file writer object.
Note
As the ScanSaving object corresponds to a persistent structure in Redis, functions as key values will be serialized. Make sure the functions are serializable.
The writer attribute¶
The .writer attribute indicates which writer to use for saving data.
BLISS currently supports
* "null": writing disabled
* "nexus": the Nexus writer
* "hdf5" (OBSOLETE): internal writer in BLISS
Template attributes¶
As an example SCAN_SAVING we will add two extra parameters
(sample and experiment) and use them to generate the final path.
DEMO [1]: # Set the base path to /data/visitor:
DEMO [2]: SCAN_SAVING.base_path = '/data/visitor'
DEMO [3]: # Add the two new parameters:
DEMO [4]: SCAN_SAVING.add('sample','lysozyme')
DEMO [5]: SCAN_SAVING.add('experiment','mx1921')
DEMO [6]: # Use them in the template:
DEMO [7]: SCAN_SAVING.template = '{experiment}/{sample}'
DEMO [8]: # result:
DEMO [8]: SCAN_SAVING
Out [8]: Parameters (default) -
.base_path = '/data/visitor''
.data_filename = 'data'
.user_name = 'guilloud'
.template = '{experiment}/{sample}'
.images_path_relative = True
.images_path_template = 'scan{scan_number}'
.images_prefix = '{img_acq_device}_'
.date_format = '%Y%m%d'
.scan_number_format = '%04d'
.experiment = 'mx1921'
.session = 'cyril'
.date = '20190926'
.scan_name = 'scan name'
.scan_number = 'scan number'
.img_acq_device = '<images_* only> acquisition device name'
.sample = 'lysozyme'
.writer = 'hdf5'
.creation_date = '2019-08-23-10:08'
.last_accessed = '2019-09-26-16:09'
-------------- --------- -------------------------------------
does not exist filename /data/visitor/mx1921/lysozyme/data.h5
exists root_path /data/visitor/mx1921/lysozyme
-------------- --------- -------------------------------------
DEMO [9]: SCAN_SAVING.get_path()
Out [9]: '/data/visitor/mx1921/lysozyme'
Dynamic attributes¶
In the previous example, the experiment can be a function instead of a string:
DEMO [10]: def get_experiment(scan_saving):
...: if scan_saving.user_name == 'seb':
...: return 'mx1921'
...: else:
...: return 'unknown'
DEMO [11]: SCAN_SAVING.add('experiment',get_experiment)
DEMO [12]: SCAN_SAVING.get_path()
Out [12]: '/data/visitor/mx1921/lysozyme'
DEMO [13]: SCAN_SAVING.user_name='toto'
DEMO [14]: SCAN_SAVING.get_path()
Out [14]: '/data/visitor/unknown/lysozyme'