Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Structuring multiple boards in environment config #1194

Open
nlabriet opened this issue May 31, 2023 · 10 comments · May be fixed by #1208
Open

Structuring multiple boards in environment config #1194

nlabriet opened this issue May 31, 2023 · 10 comments · May be fixed by #1208
Labels

Comments

@nlabriet
Copy link
Contributor

nlabriet commented May 31, 2023

I am setting up a lab with 8 different hardware that are similar from a LabGrid point-of-view. Beside these 8 boards, I have 8 helper hardware (1 for each board) that are exactly the same (BeagleBone Black).

My goal is to provide an environment file to my colleagues so they are able to work on any board.

The environment file looks like this:

targets:
  board1:
    resources:
      RemotePlace:
        name: board1
    drivers:
      - ModbusCoilDriver: {}
      - ModbusCoilDriver:
          name: power
          bindings:
            coil: power
      - DigitalOutputPowerDriver:
          bindings:
            output: power
      - SerialDriver: {}
      - UBootDriver:
          autoboot: Type password
          interrupt: e
          prompt: '=> '
          bootstring: Starting kernel ...
          boot_commands:
            usb: run usbbootcmd
      - ShellDriver:
          prompt: 'user@localhost:~# '
          login_prompt: '[^ ]+ login: '
          username: user
          password: password
      - SSHDriver: {}
      - UBootStrategy: {}
  board2:
    resources:
      RemotePlace:
        name: board2
    drivers:
      - ModbusCoilDriver: {}
      - ModbusCoilDriver:
          name: power
          bindings:
            coil: power
      - DigitalOutputPowerDriver:
          bindings:
            output: power
      - SerialDriver: {}
      - UBootDriver:
          autoboot: Type password
          interrupt: e
          prompt: '=> '
          bootstring: Starting kernel ...
          boot_commands:
            usb: run usbbootcmd
      - ShellDriver:
          prompt: 'user@localhost:~# '
          login_prompt: '[^ ]+ login: '
          username: user
          password: password
      - SSHDriver: {}
      - UBootStrategy: {}
# Repeat the last 28 lines for the remaining 6 boards
  board1-bbb:
    resources:
      RemotePlace:
        name: board1-bbb
    drivers:
      - ModbusCoilDriver: {}
      - DigitalOutputPowerDriver: {}
      - SSHDriver: {}
  board2-bbb:
    resources:
      RemotePlace:
        name: board2-bbb
    drivers:
      - ModbusCoilDriver: {}
      - DigitalOutputPowerDriver: {}
      - SSHDriver: {}
# Repeat the last 8 lines for the remaining 6 bbb boards

I can reduce the repetition using yaml syntax

targets:
  board1:
    resources:
      RemotePlace:
        name: board1
    drivers: &linux
    - ModbusCoilDriver: {}
    - ModbusCoilDriver:
      name: 'power'
      bindings:
        coil: 'power'
    - DigitalOutputPowerDriver:
      bindings:
        output: 'power'
    - SerialDriver: {}
    - UBootDriver:
      autoboot: "Type password"
      interrupt: 'e'
      prompt: '=> '
      bootstring: 'Starting kernel ...'
      boot_commands:
        usb: run usbbootcmd
    - ShellDriver:
      prompt: 'user@localhost:~# '
      login_prompt: '[^ ]+ login: '
      username: 'user'
      password: 'password'
    - SSHDriver: {}
    - UBootStrategy: {}
  board2:
    resources:
      RemotePlace:
        name: board2
    drivers: *linux
# Repeat the last 5 lines for the remaining 6 boards
  board1-bbb:
    resources:
      RemotePlace:
        name: board1-bbb
    drivers: &bbb
    - ModbusCoilDriver: {}
    - DigitalOutputPowerDriver: {}
    - SSHDriver: {}
  board2-bbb:
    resources:
      RemotePlace:
        name: board2-bbb
    drivers: *bbb
# Repeat the last 5 lines for the remaining 6 bbb boards

One way that seems more intuitive and reduce repetition is for RemotePlace to support a list:

targets:
  linux:
    resources:
      RemotePlace:
        name:
        - board1
        - board2
        - board3
        - board4
        - board5
        - board6
        - board7
        - board8
    drivers:
    - ModbusCoilDriver: {}
    - ModbusCoilDriver:
      name: 'power'
      bindings:
        coil: 'power'
    - DigitalOutputPowerDriver:
      bindings:
        output: 'power'
    - SerialDriver: {}
    - UBootDriver:
      autoboot: "Type password"
      interrupt: 'e'
      prompt: '=> '
      bootstring: 'Starting kernel ...'
      boot_commands:
        usb: run usbbootcmd
    - ShellDriver:
      prompt: 'user@localhost:~# '
      login_prompt: '[^ ]+ login: '
      username: 'user'
      password: 'password'
    - SSHDriver: {}
    - UBootStrategy: {}
  bbb:
    resources:
      RemotePlace:
        name:
        - board1-bbb
        - board2-bbb
        - board3-bbb
        - board4-bbb
        - board5-bbb
        - board6-bbb
        - board7-bbb
        - board8-bbb
    drivers:
    - ModbusCoilDriver: {}
    - DigitalOutputPowerDriver: {}
    - SSHDriver: {}

Another way is for the environment configuration file to use Jinja2 as a preprocessor, like the exporter configuration file:

# for idx in range (0, 8)
targets:
  board{{ 1 + idx }}:
    resources:
      RemotePlace:
        name: board{{ 1 + idx }}
    drivers: &linux
    - ModbusCoilDriver: {}
    - ModbusCoilDriver:
      name: 'power'
      bindings:
        coil: 'power'
    - DigitalOutputPowerDriver:
      bindings:
        output: 'power'
    - SerialDriver: {}
    - UBootDriver:
      autoboot: "Type password"
      interrupt: 'e'
      prompt: '=> '
      bootstring: 'Starting kernel ...'
      boot_commands:
        usb: run usbbootcmd
    - ShellDriver:
      prompt: 'user@localhost:~# '
      login_prompt: '[^ ]+ login: '
      username: 'user'
      password: 'password'
    - SSHDriver: {}
    - UBootStrategy: {}
  board{{ 1 + idx }}-bbb:
    resources:
      RemotePlace:
        name: board{{ 1 + idx }}-bbb
    drivers: &bbb
    - ModbusCoilDriver: {}
    - DigitalOutputPowerDriver: {}
    - SSHDriver: {}
# endfor

I like the 'RemotePlace list' better because it emphasis the role and make it easy to assign boards to another role, should this be needed in the future.

What do you think about this?

How does you environment file look like and how do you maintain/share it?

@Bastian-Krause
Copy link
Member

Bastian-Krause commented Jun 1, 2023

I'd suggest using templating in this case, see labgrid's environment configuration docs:

targets:
  main:
    resources:
      RemotePlace:
        name: !template $LG_PLACE
    drivers:
      - ModbusCoilDriver: {}
      - ModbusCoilDriver:
          name: power
          bindings:
            coil: power
      - DigitalOutputPowerDriver:
          bindings:
            output: power
      - SerialDriver: {}
      - UBootDriver:
          autoboot: Type password
          interrupt: e
          prompt: '=> '
          bootstring: Starting kernel ...
          boot_commands:
            usb: run usbbootcmd
      - ShellDriver:
          prompt: 'user@localhost:~# '
          login_prompt: '[^ ]+ login: '
          username: user
          password: password
      - SSHDriver: {}
      - UBootStrategy: {}
  bbb:
    resources:
      RemotePlace:
        name: !template "$LG_PLACE-bbb"
    drivers:
      - ModbusCoilDriver: {}
      - DigitalOutputPowerDriver: {}
      - SSHDriver: {}

Then select a board via..

$ export LG_PLACE=board1

@Bastian-Krause Bastian-Krause changed the title Environment file Structuring multiple boards in environment config Jun 1, 2023
@nlabriet
Copy link
Contributor Author

nlabriet commented Jun 2, 2023

I like the template based on the place. It works well for my boards even when using tokens.
And I will be able to use an environment file for each role (only one 1 for now).

But I can not access the -bbb places. I think it's because when I acquire board1-bbb, then it's considered as a board.
Since #1163 I've separated the boards and the side hardware.

Here is the error:

(labgrid-venv) labgrid$ export LG_ENV=~/board-template.yaml
(labgrid-venv) labgrid$ export LG_PLACE=board-1-bbb
(labgrid-venv) labgrid$ labgrid-client lock
Selected role linux from configuration file
acquired place board-1-bbb
(labgrid-venv) labgrid$ labgrid-client power on
Selected role linux from configuration file
Traceback (most recent call last):
  File "/home/labgrid/labgrid-venv/lib/python3.8/site-packages/labgrid/remote/client.py", line 1878, in main
    args.func(session)
  File "/home/labgrid/labgrid-venv/lib/python3.8/site-packages/labgrid/remote/client.py", line 725, in power
    target = self._get_target(place)
  File "/home/labgrid/labgrid-venv/lib/python3.8/site-packages/labgrid/remote/client.py", line 669, in _get_target
    target = self.env.get_target(self.role)
  File "/home/labgrid/labgrid-venv/lib/python3.8/site-packages/labgrid/environment.py", line 49, in get_target
    target = target_factory.make_target(role, config, env=self)
  File "/home/labgrid/labgrid-venv/lib/python3.8/site-packages/labgrid/factory.py", line 159, in make_target
    self.make_driver(target, driver, name, args)
  File "/home/labgrid/labgrid-venv/lib/python3.8/site-packages/labgrid/factory.py", line 138, in make_driver
    d = cls(target, name, **args)
  File "<attrs generated init labgrid.driver.serialdriver.SerialDriver>", line 11, in __init__
    self.__attrs_post_init__()
  File "/home/labgrid/labgrid-venv/lib/python3.8/site-packages/labgrid/driver/serialdriver.py", line 28, in __attrs_post_init__
    super().__attrs_post_init__()
  File "/home/labgrid/labgrid-venv/lib/python3.8/site-packages/labgrid/driver/consoleexpectmixin.py", line 18, in __attrs_post_init__
    super().__attrs_post_init__()
  File "/home/labgrid/labgrid-venv/lib/python3.8/site-packages/labgrid/driver/common.py", line 24, in __attrs_post_init__
    super().__attrs_post_init__()
  File "/home/labgrid/labgrid-venv/lib/python3.8/site-packages/labgrid/binding.py", line 55, in __attrs_post_init__
    target.bind(self)
  File "/home/labgrid/labgrid-venv/lib/python3.8/site-packages/labgrid/target.py", line 427, in bind
    return self.bind_driver(bindable)
  File "/home/labgrid/labgrid-venv/lib/python3.8/site-packages/labgrid/target.py", line 378, in bind_driver
    raise NoSupplierFoundError(
labgrid.exceptions.NoSupplierFoundError: no supplier matching {'NetworkSerialPort', 'SerialPort'} found in Target(name='linux', env=Environment(config_file='/home/labgrid/board-template.yaml')) (errors: [NoResourceFoundError(msg="no <class 'labgrid.resource.serialport.NetworkSerialPort'> resource found in Target(name='linux', env=Environment(config_file='/home/labgrid/board-template.yaml'))", filter=None, found=None), NoResourceFoundError(msg="no <class 'labgrid.resource.base.SerialPort'> resource found in Target(name='linux', env=Environment(config_file='/home/labgrid/board-template.yaml'))", filter=None, found=None)])

@Bastian-Krause
Copy link
Member

If you want to interact with "board1" and labgrid-client, I'd probably do:

$ export LG_ENV=~/board-template.yaml
$ export LG_PLACE=board-1
$ labgrid-client lock
acquired place board1
$ labgrid-client ssh true
Selected role main from configuration file

And with "board1-bbb" and labgrid-client:

$ labgrid-client -p board1-bbb lock
acquired place board1-bbb
$ labgrid-client -p board1-bbb ssh true
Selected role main from configuration file

@nlabriet
Copy link
Contributor Author

nlabriet commented Jun 5, 2023

Right this works when setting export LG_PLACE=board-1, but it doesn't when reserving board-1, using a token and export LG_PLACE=+

$ labgrid-client -p board-1-bbb lock
RemotePlace board-1-bbb not found in configuration file

@nlabriet
Copy link
Contributor Author

It works well for my boards even when using tokens.

I was wrong, it doesn't work for tokens. But with the following patch, it does (I will work on it and submit a PR):

diff --git a/labgrid/remote/client.py b/labgrid/remote/client.py
index 52b39f2..8116c36 100755
--- a/labgrid/remote/client.py
+++ b/labgrid/remote/client.py
@@ -1346,7 +1346,7 @@ def find_role_by_place(config, place):
         resources, _ = target_factory.normalize_config(role_config)
         remote_places = resources.get('RemotePlace', {})
         remote_place = remote_places.get(place)
-        if remote_place:
+        if remote_place or '+' in remote_places:
             return role
     return None

With this I can successfully use the following environment file:

targets:
  board1-bbb:
    resources:
      RemotePlace:
        name: board1-bbb
    drivers: &bbb
      - ModbusCoilDriver: {}
      - DigitalOutputPowerDriver: {}
      - SSHDriver: {}
  board2-bbb:
    resources:
      RemotePlace:
        name: board2-bbb
    drivers: *bbb
  main:
    resources:
      RemotePlace:
        name: !template $LG_PLACE
    drivers:
      - ModbusCoilDriver: {}
      - ModbusCoilDriver:
          name: power
          bindings:
            coil: power
      - DigitalOutputPowerDriver:
          bindings:
            output: power
      - SerialDriver: {}
      - UBootDriver:
          autoboot: Type password
          interrupt: e
          prompt: '=> '
          bootstring: Starting kernel ...
          boot_commands:
            usb: run usbbootcmd
      - ShellDriver:
          prompt: 'user@localhost:~# '
          login_prompt: '[^ ]+ login: '
          username: user
          password: password
      - SSHDriver: {}
      - UBootStrategy: {}

@nlabriet nlabriet linked a pull request Jun 13, 2023 that will close this issue
3 tasks
@Bastian-Krause
Copy link
Member

With #1208, what's left in this issue that's not solved then?

@nlabriet
Copy link
Contributor Author

This syntax is not supported by #1208:
name: !template "$LG_PLACE-bbb"

But it can be tricky (do we allow "prefix-$LG_PLACE-suffix"?) and I am happy with using yaml syntax to repeat the boardX-bbb configuration has it should not need to be changed.

@Bastian-Krause
Copy link
Member

I see. How exactly would "using yaml syntax to repeat the boardX-bbb configuration" look like?

@nlabriet
Copy link
Contributor Author

I use this for the bbb:

  board1-bbb:
    resources:
      RemotePlace:
        name: board1-bbb
    drivers: &bbb
    - ModbusCoilDriver: {}
    - DigitalOutputPowerDriver: {}
    - SSHDriver:
        connection_timeout: 180.0
  board2-bbb:
    resources:
      RemotePlace:
        name: board2-bbb
    drivers: *bbb
  board3-bbb:
    resources:
      RemotePlace:
        name: board3-bbb
    drivers: *bbb

and so on

@Bastian-Krause
Copy link
Member

Ah nice, I was not aware that this works already. If I remember correctly, yaml merge (<<) is currently not supported, but this works, too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants