Skip to content

Testing and BLISS


Doc partialy taken from pytest official doc:

Running tests on your own computer

Test setup

  • Create a conda environment (named testenv for example) dedicated to tests
make dev_env NAME=testenv
  • Test the tests with a simple one:
    source activate testenv
    cd ~/local/bliss.git/
    pytest tests/motors/ -k test_controller_from_axis

The result should be:

tests/motors/ PASSED              [100%]
====================== 1 passed, 72 deselected in 3.12s =======================

To run ALL tests

Just type pytest In BLISS root directory.

Example of run (can be long):

bliss % pytest
========================== test session starts ==============================
collected 454 items

tests/ PASSED            [  0%]
tests/ PASSED                        [  0%]
tests/ PASSED                         [  0%]


In case of strange error (like ImportError: bad magic number), try to remove old *.pyc files:

find ./ -name "*.pyc" | xargs rm

To run ONLY some tests

  • To run a single test (test_lima_ctrl_params_uploading):
    • -s: keep stdout, equivalent to --capture=no => do not capture stdout
    • -v: more verbose
    • -q: less verbose
    • --count=20: to repeat the test 20 times (pytest-repeat module must be installed)
pytest -sv --count=20  tests/controllers_sw/
  • To run a reduced set of tests

-k command line option specify an expression which implements a sub-string match on the test names instead of the exact match on markers that -m provides.

pytest tests/ -k <sub-string>

Example (in BLISS root directory):

[(bliss) guilloud@babasse]$ pytest   tests/ -k test_controller_from_axis
========================== test session starts ===================================
linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0 -- .../bliss/bin/python3.9
cachedir: .pytest_cache
rootdir: /home/guilloud/PROJECTS/bliss-master
configfile: setup.cfg
plugins: cov-4.0.0, xvfb-2.0.0, rerunfailures-11.1.2, profiling-1.7.0 ...
collected 2631 items / 2630 deselected / 1 selected

tests/motors/ PASSED                 [100%]

============================== 1 passed, 2630 deselected in 15.00s ===============

In previous example, all tests are collected before to unselect main part of them. Better specify a test file.


[(bliss)]$ pytest tests/motors/ -k test_controller_from_axis
================================ test session starts =============================
platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0 -- python3.9
cachedir: .pytest_cache
rootdir: /home/guilloud/PROJECTS/bliss-master
configfile: setup.cfg
plugins: cov-4.0.0, xvfb-2.0.0, rerunfailures-11.1.2, profiling-1.7.0, ...
collected 73 items / 72 deselected / 1 selected

tests/motors/ PASSED                [100%]

========================= 1 passed, 72 deselected in 3.12s =======================

other usages

  • Start test session device servers, like Lima camera simulators:
TANGO_HOST=localhost:20000 LimaCCDs simulator
  • Start the BLISS shell:
BEACON_HOST=localhost TANGO_HOST=localhost:20000 bliss -s test_session
  • A lot of controllers are already part of the test session. The ones depending on a Tango server are not included by default, so those devices can be accessed via the config object:
TEST_SESSION[1]: limaDev = config.get("lima_simulator")


To run Nexus writer test, use: pytest tests/nexus_writer/ --durations=30

Tests and Continuous Integration

All tests but hardware-related ones are automatically run during continuous integration on bcu-ci server.

To run tests on bcu-ci computer

Some timing problems occuring during continuous integration but not on a local computer have been observed.

To track them, it can be interesting to run tests on bcu-ci computer.

Log-in to bcu-ci (needs sudo rights) and copy/paste:

sudo docker run -it continuumio/miniconda3:latest

apt-get update && apt-get -y install xvfb libxi6 git

git clone

cd bliss

make dev_env NAME=testenv

source activate testenv

pytest tests

Tests configuration and creation


pytest.xfail() instruction

“A xfail means that you expect a test to fail for some reason. A common example is a test for a feature not yet implemented, or a bug not yet fixed. When a test passes despite being expected to fail (marked with pytest.mark.xfail), it’s an xpass and will be reported in the test summary.”


@pytest.mark.parametrize("channel_id", [1, 2])
def test_read_calc_channels(pepu, channel_id):
    cmd = "?CHVAL CALC%d" % channel_id
    with pepu.assert_command(cmd, "-1"):
        channel = pepu.calc_channels[channel_id]
        value = channel.value
    assert value in (1.5, -1.5)


Coverage indicates the percentage of lines touched by current tests suite.

Example to get a coverage report:

py.test tests/controllers_sw/   \
           --cov-report=html                              \
           --cov bliss.controllers.multiplepositions

Coverage report indicating tested lines is in: ./htmlcov/index.html


There can be some errors (lines tested but not flaged as tested) in the report.

See also:

Pytest configuration

Configuration is mainly done in setup.cfg file:

bliss %
bliss % more setup.cfg

   addopts = -v --ignore=tests/images --ignore=tests/test_configuration --ignore=tests/controllers_hw
   filterwarnings =


Tips and examples


A set of fixtures is defined in tests/ file.

Their role is to ease the definition of tests by factorizing some procedures.


  • beacon: to give access to the configuration in the test function via config
  • session: to run tests within a BLISS session
  • log_context: allows to get access to logging mechanisms
    • results are readable via caplog module
  • other examples: lima_simulator, dummy_tango_server, wago_tango_server


session fixture gives access to test_session Bliss session:

  • config
def test_SampCnt_statistics(session):
    diode = session.config.get("diode")
    diode2 = session.config.get("diode2")
  • env_dict
def test_SampCnt_mode_SINGLE(session):
    env_dict = session.env_dict

    diode2 = env_dict["diode2"]
    diode8 = env_dict["diode8"]
  • homepage_port
  • beacon_tmpdir
  • beacon_directory
  • log_directory
  • images_directory
  • ports:
    • redis_port
    • redis_data_port
    • tango_port
    • beacon_port
    • logserver_port
    • homepage_port


capsys fixture gives access to the standard output and error.


def test_bench(beacon, setup_globals, capsys):
    with bench():

    captured = capsys.readouterr()
    assert "Execution time: 1s" in captured.out

see also: capsysbinary, capfd, and capfdbinary fixtures.


To test that an exception is well reaise, pytest.raises() context manager can be used. This test will success if a ValueError exception is raised:

with pytest.raises(ValueError):
    mca.rois.set("Auguste", -63, 14)


To test equality of floats or to test 2 values with an approximation margin, pytest.approx must be used.

example to ensure that position is 6.28 more or less 0.001:

assert roby.position == pytest.approx(6.28, abs=1e-3)


rel or abs keyword arg should be used to avoid mistake.

In [38]: 111 == pytest.approx(112.0, 0.1)    # <--- !!! rel by default
Out[38]: True

In [39]: 111 == pytest.approx(112.0, abs=0.1)
Out[39]: False


assert pytest.approx(position, 2) is WRONG (compare position to nothing) use: assert position == pytest.approx(2)

see: for details.

Using a Tango device server in tests

A dummy tango device server is defined in :


- class: Dummy
  tango_name: id00/tango/dummy
  personal_name: dummy
  server: dummy_tg_server

It is used for example to test undulator object:


    class: ESRF_Undulator
    ds_name: id00/tango/dummy
            name: u23a
            attribute_position: Position
            attribute_velocity: Velocity
            attribute_acceleration: Acceleration
            steps_per_unit: 1
            velocity: 5
            acceleration: 125
            tolerance: 0.1


import pytest
def test_undulator(beacon, dummy_tango_server):
    u23a = beacon.get("u23a")

    assert u23a.position == 1.4
    assert u23a.velocity == 5
    assert u23a.acceleration == 125

Access to temporary directory

def test_session_add_del(beacon, beacon_directory):
    # beacon_directory is the temporary directory used by tests.
    # BLISS Session files are put in beacon_directory/sessions
    sess_dir = beacon_directory + '/sessions'
    setup_file = sess_dir + '/'

Hardware tests

Tests files located in bliss/tests/controllers_hw/ directory are Hardware tests. They are ignored by continuous integration but can be run manualy.


They are using the beamline database for configuration and not the test configuration.


There is a generic axis test for basic feature: position, velocity, acceleration and stop.


pytest -s --axis-name rot tests/controllers_hw/
This will do a real test on Beamline axis named rot.


This test will do real movement on the specified axis.


Pylint is a tool that checks for errors in Python code. It can recommend suggestions about how particular blocks can be refactored and can offer details about the code’s complexity.

pylint is ran automatically by the Continuous Integration pipeline before to start pytest.

For now it triggers only warnings and is not lbocking in the CI process.

pylint produces logs that can be read on gitlab pipelines page. Clic on the first orange icon in column Stages, then check_lint then at bottom of the page comments like that can be read:

68 ../ F401 'log_shell_mode' imported but unused
69 ../ F811 redefinition of 'log_shell_mode' from line 25

These comments can usually be profitably followed. Some more or less tricky bugs (typically typo on variable names) can then be avoided.

pylint false positive

In (rare) case of false positive error detection, an indication to avoid warning during CI can be added in the code.

For example, in tests/motors/, the F401 reported error is wrong, log_shell_mode is used.

To avoid warning, “# noqa: F401” comment has been added to the line 25:

from ..common.conftest import log_shell_mode  # noqa: F401

Example to avoid invalid-name message for a group of lines:

# pylint: disable=invalid-name
_float_or_countables = Union[
    numbers.Real, IterableNamespace, Counter, MeasurementGroup, CounterContainer, Tuple
_scannable_or_name = Union[Scannable, str]
_scannable_start_stop_list = Sequence[Tuple[Scannable, _float, _float]]
_scannable_start_stop_intervals_list = Sequence[Tuple[Scannable, _float, _float, _int]]
# pylint: enable=invalid-name

Example to avoid protected-access message for one line:

    with mock.patch.object(
        timer, "_sleep", wraps=timer._sleep  # pylint: disable=protected-access
    ) as timer_sleep:  # pylint: disable=not-callable
        assert timer_sleep.call_count == 1