Data policy
The data policy of a BLISS session is defined by the session’s SCAN_SAVING object.
This object can be accessed from the BLISS shell by the global variable SCAN_SAVING that refers to current_session.scan_saving.
The SCAN_SAVING object has fields that are stored in REDIS for presistency and others with values that are generated dynamically.
The object is instantiated during the setup phase of the BLISS session.
BasicScanSaving API¶
DEMO_SESSION [1]: SCAN_SAVING
Out [1]: Parameters (default) -
.base_path = '/tmp/toto'
.data_filename = 'data'
.template = '{session}/'
.images_path_relative = True
.images_path_template = 'scan{scan_number}'
.images_prefix = '{img_acq_device}_'
.date_format = '%Y%m%d'
.scan_number_format = '%04d'
.session = 'demo_session'
.date = '20240824'
.user_name = 'denolf'
.scan_name = '{scan_name}'
.scan_number = '{scan_number}'
.img_acq_device = '{img_acq_device}'
.data_policy = 'None'
.writer = 'hdf5'
-------------- --------- ------------------------------
does not exist filename /tmp/toto/demo_session/data.h5
exists directory /tmp/toto/demo_session
-------------- --------- ------------------------------
base_path corresponds to the top-level directory where scans are
stored. Then, template completes the path. It uses Python’s string
interpolation syntax to specify how to build the file path from key
values. Keys can be freely added. Key values can be anything that is
JSON serializable.
Attributes¶
Fields with string values are interpolated when retrieving their value.
base_path: the highest level directory for the file path, e.g./datauser_name: the current Unix user namesession: current BLISS session nametemplate: defaults to{session}/root_path: the directory path where the data file will be saveddata_path: full path of the data file without the extensionfilename: full path of the data file with the extensionimages_path: path where image devices should save (Lima)writer_object: data file writer object.add(key, value): add a new field
The writer attribute¶
The .writer attribute indicates which writer to use for saving data.
BLISS currently supports
* "null": writing disabled
* "nexus": the Nexus writer
* "hdf5" (OBSOLETE): internal writer in BLISS
Template attributes¶
As an example SCAN_SAVING we will add two extra parameters
(sample and experiment) and use them to generate the final path.
DEMO_SESSION [1]: # Set the base path to /data/visitor:
DEMO_SESSION [2]: SCAN_SAVING.base_path = '/data/visitor'
DEMO_SESSION [3]: # Add the two new parameters:
DEMO_SESSION [4]: SCAN_SAVING.add('sample','lysozyme')
DEMO_SESSION [5]: SCAN_SAVING.add('experiment','mx1921')
DEMO_SESSION [6]: # Use them in the template:
DEMO_SESSION [7]: SCAN_SAVING.template = '{experiment}/{sample}'
DEMO_SESSION [8]: # Result:
DEMO_SESSION [9]: SCAN_SAVING
Out [9]: Parameters (default) -
.base_path = '/data/visitor'
.data_filename = 'data'
.template = '{experiment}/{sample}'
.images_path_relative = True
.images_path_template = 'scan{scan_number}'
.images_prefix = '{img_acq_device}_'
.date_format = '%Y%m%d'
.scan_number_format = '%04d'
.session = 'demo_session'
.date = '20240824'
.user_name = 'denolf'
.scan_name = '{scan_name}'
.scan_number = '{scan_number}'
.img_acq_device = '{img_acq_device}'
.data_policy = 'None'
.writer = 'hdf5'
.sample = 'lysozyme'
.experiment = 'mx1921'
-------------- --------- -------------------------------------
does not exist filename /data/visitor/mx1921/lysozyme/data.h5
exists directory /data/visitor/mx1921/lysozyme
-------------- --------- -------------------------------------
DEMO_SESSION [10]: # Data file directory:
DEMO_SESSION [11]: SCAN_SAVING.root_path
Out [11]: '/data/visitor/mx1921/lysozyme'
Custom data policy¶
Creating a custom data policy means deriving a class from bliss.scanning.scan_saving.BasicScanSaving:
class CustomScanSaving(BasicScanSaving):
class _Fields:
experiment = Field()
sample = Field()
custom = Field(location=FieldLocation.attribute, mutable=False)
template = Field("{experiment}/{sample}/{custom}")
@property
def custom(self):
return "generated"
_LEGACY_WARDROBE_TEMPLATE = "scan_saving:{}"
_REDIS_KEY_TEMPLATE = "scan_saving:custom:{}"
A BLISS session needs to be configured to use this policy.
DEMO_SESSION [1]: # Set the base path to /data/visitor:
DEMO_SESSION [2]: SCAN_SAVING.base_path = '/data/visitor'
DEMO_SESSION [3]: # Set the two new parameters:
DEMO_SESSION [4]: SCAN_SAVING.sample = 'lysozyme'
DEMO_SESSION [5]: SCAN_SAVING.experiment = 'mx1921'
DEMO_SESSION [6]: # Result:
DEMO_SESSION [7]: SCAN_SAVING
Out [8]: Parameters (default) -
.base_path = '/data/visitor'
.data_filename = 'data'
.images_path_relative = True
.images_path_template = 'scan{scan_number}'
.images_prefix = '{img_acq_device}_'
.date_format = '%Y%m%d'
.scan_number_format = '%04d'
.session = 'demo_session'
.date = '20240824'
.user_name = 'denolf'
.scan_name = '{scan_name}'
.scan_number = '{scan_number}'
.img_acq_device = '{img_acq_device}'
.data_policy = 'None'
.writer = 'hdf5'
.experiment = 'mx1921'
.sample = 'lysozyme'
.custom = 'mx1921'
.template = '{experiment}/{sample}/{custom}'
-------------- --------- -------------------------------------
does not exist filename /data/visitor/mx1921/lysozyme/generated/data.h5
exists directory /data/visitor/mx1921/lysozyme/generated
-------------- --------- -------------------------------------
DEMO_SESSION [9]: # Data file directory:
DEMO_SESSION [10]: SCAN_SAVING.root_path
Out [10]: '/data/visitor/mx1921/lysozyme/generated'