Data policy and data format
Under the ESRF data policy the data directory is determined from: proposal name, sample name and dataset name. The data directory for a user experiment is:
Change the proposal name with the command
DEMO : newproposal("hg123") Proposal set to 'hg123' Data path: /data/visitor/hg123/idxy/sample/sample_0001
The root directory is determined from the proposal name (can be configured):
- user experiment: /data/visitor/proposal
- inhouse (starts with the beamline name): /data/idxy/inhouse/proposal/
- test (starts with test, temp or tmp): /data/idxy/tmp/proposal
When no proposal name is specified the default used is the beamline proposal name of the month (in this example February 2020 at beamline idxy).
DEMO : newproposal() Proposal set to 'idxy2002' Data path: /data/idxy/inhouse/idxy2002/idxy/sample/sample_0001
The default beamline proposal is also accessible through https://data.esrf.fr and offers an electronic logbook with the exception of test proposals.
At the end of an experiment you can use the function
endproposal() to ensure nobody accidentally
adds data to the proposal directory from Bliss. Just like the command
newproposal() it switches
data saving to the default beamline proposal of the month.
DEMO : endproposal() Proposal set to 'idxy2002' Data path: /data/idxy/inhouse/idxy2002/idxy/sample/sample_0001
Change sample name with the function
DEMO : newsample("treated_cells_1_4") Sample set to 'treated_cells_1_4' Data path: /data/visitor/hg123/idxy/treated_cells_1_4/treated_cells_1_4_0001
Change the dataset name with the function
DEMO : newdataset("area1") Dataset set to 'area1' Data path: /data/visitor/hg123/idxy/treated_cells_1_4/treated_cells_1_4_area1
When no dataset name is specified, datasets are numbered (unique within a sample):
DEMO : newdataset() Dataset set to '0005' Data path: /data/visitor/hg123/idxy/treated_cells_1_4/treated_cells_1_4_0005
Within the directory of a dataset you can find the main Nexus compliant HDF5 file.
All scan data belonging to the dataset are accessible through this one file, although some detectors (for example all 2D detectors) actually save their data in separate files.
A TANGO device referred to as the Nexus writer saves all data produced by BLISS. Data saving can be monitored through this device:
The ESRF Data Analysis Unit provids two main tools to visualize generic Nexus data:
- silx view
A description on how to access HDF5 files, while they are being modified by a writer, can be found here.
silx view sample_dataset.h5
Silx view can be called directly from a BLISS shell via
function. It points then on the last scan’s data file.
Do not use a silx versions older than 0.12.0
BLISS locks the dataset file sample_dataset.h5 during scans. If other applications have locked the file (which is the default behaviour when opening HDF5 files) BLISS cannot write to it and scans will be prevented from starting. Tools like silx and pymca do not lock the file and are able to visualize its content without disturbing ongoing scans.
Provides easy access to positioner and detector data for plotting.
PyMca can be launched directly from a BLISS shell via the
pymca() function. It
points then on the last scan’s data file.
The dataset file sample_dataset.h5 contains one entry for each scan. Using the Silx viewer it looks like this:
A scan entry has two subsections by default: instrument (for positioner and detector data/metadata) and measurement (links to all data under instrument). Additional subsections can appear when defining plots, scan notes, sample description and application definitions.
The instrument section contains groups of data and metadata for detectors and positioners:
The measurement group together with the positioner group under instrument has all links to all data and motor positions at the start of a scan: