LAVA events

lun. 23 janvier 2017 by Rémi Duraffort

For some releases now, LAVA server is publishing events every time:

  • a Job status change (Submitted, Running, ...)
  • a Device status change (Idle, Reserved, ...)

The stream of events is public and anyone can subscribe and receive updates.

A dump of the streams for the production and the staging instances is available on ReactOWeb. For instance, here is a (partial) extract of the stream:

topic: org.linaro.validation.testjob, data: {"status": "Submitted", "priority": 50, "job": 1318796, "health_check": true, "device": "aa9-01", "description": "lab-health-aa9", "submit_time": "2017-01-19T13:31:17.995716+00:00", "submitter": "lava-health", "pipeline": false, "visibility": "Publicly visible"}
topic: org.linaro.validation.device, data: {"status": "Reserved", "pipeline": false, "job": "1318429.1", "health_status": "Pass", "device_type": "x15", "device": "x15-01"}
topic: org.linaro.validation.device, data: {"status": "Reserved", "pipeline": false, "job": "1318455.0", "health_status": "Pass", "device_type": "kvm", "device": "kvm21"}

Use cases

A typical use case of the stream of event would be to know when:

  • a job is finished without polling the database
  • jobs are running on a specific board
  • a given device is going offline/online/...

For instance a user might want to build a website that aggregate all the results of the jobs on a given device type. Thanks to the event stream, it's possible to build and update such website in real time and without any polling.

Listening to the stream

The stream of event is published by a ZMQ PUB socket. In order to connect you should use the ZMQ library or a program like ReactOBus.

In Python, printing all events from the production instance is as simple as:

import zmq

ctx = zmq.Context()
sub = ctx.socket(zmq.SUB)
sub.setsockopt(zmq.SUBSCRIBE, b"")
sub.connect("tcp://validation.linaro.org:5510")

while True:
    msg = sub.recv_multipart()
    print(msg)

For the official Linaro LAVA instances the streams are available at:

Each event is an array made of:

  1. topic
  2. uuid (as generated by uuid.uuid1() in Python)
  3. datetime when the message was generated (isoformat)
  4. username of the sending process or user
  5. data as JSON

LAVA configuration

If you want to use the stream of events for your own instance of LAVA, you should add to /etc/lava-server/settings.conf:

  • EVENT_NOTIFICATION: should be true to allow to generate events
  • EVENT_TOPIC: the base topic for each events

Read the documentation for more information.


Testing LAVA with DummySys

jeu. 19 janvier 2017 by Rémi Duraffort

For more than 3 years now, I have been working on LAVA. Looking at the documentation:

LAVA is an automated validation architecture primarily aimed at testing deployments of systems based around the Linux kernel on ARM devices, specifically ARMv7 and later.

In order to test LAVA, you need:

  • a wide …
read more

A faster YAML loader

sam. 20 août 2016 by Rémi Duraffort

That's the second issue I had when playing with the LAVA log viewer.

In the new versions of LAVA, the logs are formatted in YAML:

- {"dt": "2016-08-18T14:24:01.096308", "lvl": "info", "msg": "start: 1 tftp-deploy (max 300s)"}
- {"dt": "2016-08-18T14:24:01.099413", "lvl": "debug", "msg": "start: 1.1 download_retry …
read more

Playing with CSS rules

ven. 12 août 2016 by Rémi Duraffort

While working on improving the log viewer in LAVA I ran into an issue.

The issue is really simple: how to add display: block or display: none to a list of nodes while another JS script is adding more of theses nodes to the page.

The classical approach is to …

read more

Logging for ser2net

mer. 22 juin 2016 by Rémi Duraffort

Today, I was debugging an strange issue in LAVA while trying to boot a board.

I was under the impression that the board was not receiving all the u-boot commands that LAVA was sending.

The serial connection is accessible on the network by ser2net.

I discovered that ser2net can log …

read more

Static analysis for Python

ven. 13 mai 2016 by Rémi Duraffort

I've always been a big fan of static analysis. While I was working with C and C++, I was scanning my code with:

For a longer list of static analyzers for each language, look at wikipedia

As I'm now mostly programming in …

read more

type() vs isinstance()

mar. 03 mai 2016 by Rémi Duraffort

When running pylint on your Python source code, you might encounter this message:

Using type() instead of isinstance() for a typecheck. (unidiomatic-typecheck)

This message will be raised for this kind of code:

d = dict()
if type(d) == dict:
    print("d is a dict")

The code is valid but pylint does …

read more

Migrating from Python2 to Python3

mar. 19 avril 2016 by Rémi Duraffort

Porting code from Python 2 to Python 3 is made easier by using 2to3. This application will find patterns that should be changed to keep the same behavior in Python 2 and 3.

However, 2to3 is sometime too conservative, trying to keep the exact same semantic.

For instance, dictionaries functions …

read more

@FOSDEM

mer. 08 janvier 2014 by Rémi Duraffort

We will be at FOSDEM to talk about PRoot and some other tools based on it.

Cedric will do a Lightning talk Saturday about syscall instrumentation. He will present some tools, based on syscall instrumentation, that we develop and use at STMicroelectronics:

  • PRoot, emulate chroot, bind mount and binfmt_misc for …
read more

Pretending to be root inside PRoot

jeu. 21 novembre 2013 by Rémi Duraffort

While working on PRoot to improve the extension that fake the root user, I (re)discovered an old article that I never published before. So let's focus on the fake_id0 feature and on how it allows to use package manager directly inside the root file systems.

Some root causes

In …

read more