Micro-services for micro-controllers

[kasperl]

Edit: The content of this post has been added to the Toit documentation: https://docs.toit.io/language/sdk/services.

We have built out the underpinnings that will allow us to decompose the functionality of an embedded device into a set of micro-services that are loosely-coupled and fine-grained.

The services can be defined by code running in one container and used from other containers, so it is a natural way of separating out complex drivers (like the ones for cellular modems), so they can run in the own address spaces.

To define a service, we start with an interface. Notice that you can find all the snippets in this document, including this first interface one, in an example in the repository:

import system.services show ServiceSelector

interface RandomService:
  static SELECTOR ::= ServiceSelector
      --uuid="dd9e5fd1-a5e9-464e-b2ef-92bf15ea02ca"
      --major=0
      --minor=1

  next limit/int -> int
  static NEXT_INDEX /int ::= 0

The RandomService.SELECTOR constant is what will bind a client of the service and the service implementation together. We typically just generate random UUIDs, because all we need is some notion of identity. The dd9e5fd1-a5e9-464e-b2ef-92bf15ea02ca constant was generated via https://www.uuidgenerator.net/.

The RandomService.SELECTOR.major and RandomService.SELECTOR.minor values represent the current version of the interface. The version is used during service discovery, because it is possible that a client will be trying to access a newer or older implementation of the service due to the fact that clients and implementations are decoupled and are likely to be updated independently of each other.

The next method is the only method in the interface. We manually assign an index called NEXT_INDEX to the method for serialization purposes, but you could imagine generating the interface definition from something like a protocol buffer service and have the indices automatically assigned. If you change the index of any existing method you should bump the major version, but if you only add new methods with previously unused indices, you can get away with just bumping the minor version.

Service clients

Once we have the service interface ready, we should make it easy to use the service. This is where a service client comes in. It is a helper class that implements the service interface through an RPC mechanism.

The helper class can be derived from the interface and for RandomService it should look like this:

import system.services show ServiceClient ServiceSelector

class RandomServiceClient extends ServiceClient implements RandomService:
  static SELECTOR ::= RandomService.SELECTOR
  constructor selector/ServiceSelector=SELECTOR:
    assert: selector.matches SELECTOR
    super selector

  next limit/int -> int:
    return invoke_ RandomService.NEXT_INDEX limit

Currently, there is no tooling available to automate the generating of the client helper classes, but it certainly might be worth looking into.

The simplest part of the helper is the implementation of the next method. It just calls invoke_ with the method index and the single limit argument. If the method had taken more arguments, we would have wrapped them in a list using [], because invoke_ always take exactly two arguments.

The constructor takes the selector as an argument, but defaults to RandomService.SELECTOR. The common way to use it is to construct the client when you need it:

main:
  client := RandomServiceClient
  client.open
  10.repeat:
    print "random = $(client.next 100)"

or to have it in a lazy-initialized constant:

service_/RandomService? ::= (RandomServiceClient).open
    --if_absent=: null

The --if_absent block is invoked when we cannot find the requested service. You can provide a timeout if you're willing to
wait a bit for the service to appear:

service_/RandomService? ::= (RandomServiceClient).open
    --timeout=(Duration --s=2)
    --if_absent=: null

Service providers

Now we are ready to provide an implementation of the RandomService interface. To do this, we introduce a service provider:

import log
import system.services show ServiceProvider ServiceHandler

class RandomServiceProvider extends ServiceProvider 
    implements ServiceHandler:
  constructor:
    super "test/random" --major=7 --minor=9
    provides RandomService.SELECTOR --handler=this

  handle index/int arguments/any --gid/int --client/int -> any:
    if index == RandomService.NEXT_INDEX: return next arguments
    unreachable
  
  next limit/int -> int:
    log.info "got request for next random" --tags={"limit": limit}
    return random limit

The provider has its own name and versioning for debugging purposes (test/random@v7.9), but the important part is that it registers that it provides an implementation of the RandomService interface through the call to provides from the constructor. The implementation of the interface methods are done through the handle method that looks at the method index and calls the right implementation method, so it isn't strictly necessary that this implements the RandomService interface. Again, this code could and probably should be generated through tooling. The pid and client arguments are useful to identify the caller, which we typically rely on if we're need to keep track of service resources owned by clients.

If you want to run the service provider in a separate process, you can combine the service client and the service provider and spawn a separate process for the provider:

main:
  spawn:: 
    service := RandomServiceProvider
    service.install
    service.uninstall --wait

  client := RandomServiceClient
  client.open
  10.repeat:
    print "random = $(client.next 100)"

Serialization of arguments

The arguments provided to service method calls are serialized to a flat sequence of bytes using a simple, but efficient serialization mechanism built into the Toit virtual machine. It handles null, numbers, booleans, strings, lists, maps, and byte arrays. So if you want to send an instance of a specific class to the other side, you have to adapt the service client code to convert the instance to one of those more primitive types. Similarly, the service provider will be handed the converted type.

Service resources and proxies

Sometimes it is useful for a service to let clients refer to resources allocated on their behalf. The resource lives with the service provider and looks something like this:

import system.services show ServiceResource ServiceProvider
 
class DieResource extends ServiceResource:
  sides_/int ::= ?

  constructor .sides_ provider/ServiceProvider client/int:
    super provider client
  
  on_closed -> none:
    // Handle cleanup.

The on_closed method is automatically called when the client closes the resource or if the client happens to go away. Instances of ServiceResource are automatically serializable, so it is possible to return them from the handle method in the service provider and the ServiceResource constructor takes care of registering them correctly, so they can be found later on future client method calls.

Now, imagine we added new create_die and roll_die methods to our service interface like this:

interface RandomService:
  ...
  create_die sides/int -> int
  static CREATE_DIE_INDEX /int ::= 1

  roll_die handle/int -> int
  static ROLL_DIE_INDEX /int ::= 2

The service provider's handle method could then be extended to handle this, but do note that at this point it might make sense to no longer mark RandomServiceProvider as implementing RandomService, because we're dealing with the calls directly from handle:

handle index/int arguments/any --gid/int --client/int -> any:
  ...
  if index == RandomService.CREATE_DIE_INDEX:
    resource := DieResource arguments this client
    return resource

The resource is automatically converted to an int when returned, so now all we need is a way to call methods on the
resource. On the client side, we will instantiate a resource proxy for the resource and let that be the facade other client
code uses:

import system.services show ServiceClient ServiceResourceProxy

class DieProxy extends ServiceResourceProxy:
  constructor client/ServiceClient handle/int:
    super client handle

and we will return instances of that from the RandomServiceClient.create_die calls:

class RandomServiceClient extends ServiceClient:
  ...
  create_die sides/int -> DieProxy:
    handle := invoke_ RandomService.CREATE_DIE_INDEX sides
    proxy := DieProxy this handle
    return proxy
  ...

Now we can extend our proxy class with a new roll method using the client_ and handle_ helpers from the ServiceResourceProxy class:

class DieProxy extends ServiceResourceProxy:
  ...
  roll -> int:
    return (client_ as RandomServiceClient).roll_die handle_
  ...

The client class needs to implement the roll_die method:

class RandomServiceClient extends ServiceClient:
  ...
  roll_die handle/int -> int:
    return invoke_ RandomService.ROLL_DIE_INDEX handle
  ...

That takes care of the client side of things. Now we need to make sure the service provider forwards the roll calls to the right resource through its handle method:

handle index/int arguments/any --gid/int --client/int -> any:
  ...
  if index == RandomService.ROLL_DIE_INDEX: 
    die := (resource client arguments) as DieResource
    return die.roll
  ...

and finally we get to implement the roll method on DieResource:

class DieResource extends ServiceResource:
  ...
  roll -> int:
    return random sides_

With all that machinery in place, we can now use create resources through our client and call methods on them:

main:
  spawn:: 
    service := RandomServiceProvider
    service.install

  client := RandomServiceClient
  client.open
  die := client.create_die 6
  10.repeat:
    print "die roll = $(die.roll)"
  die.close

Resource notifications

While most interactions with resources follow a simple request-response pattern, it can be useful to be able to asynchronously notify users of a resource of certain events. The ServiceResource and ServiceResourceProxy classes have built-in support for this through notifications. A notification is any kind of serializable object sent from the resource to the proxy. The resources that take part in this must be marked notifiable at construction time:

class DieResource extends ServiceResource:
  constructor .sides_ provider/ServiceProvider client/int:
    super provider client --notifiable
  ...

Even though you probably wouldn't expect dice to ping you periodically, we can now experiment with the behavior by adding periodic notifications like this:

class DieResource extends ServiceResource:
  sides_/int ::= ?
  task_ := null

  constructor .sides_ provider/ServiceProvider client/int:
    super provider client --notifiable
    task_ = task:: notify_periodically

  notify_periodically -> none:
    while not Task.current.is_canceled:
      sleep --ms=(random 500) + 500
      notify_ 87

  on_closed -> none:
    task_.cancel

  ...

The notifications will show up on the proxy side through calls to on_notified_:

import monitor

class DieProxy extends ServiceResourceProxy:
  ...
  pinged_ ::= monitor.Signal  

  on_notified_ notification/any -> none:
    log.info "got notified" --tags={"notification": notification}
    pinged_.raise  
  ...

You'll need to wait a bit in main for the notifications to start showing up:

main:
  ...
  die := client.create_die 6
  10.repeat:
    print "die roll = $(die.roll)"
  sleep (Duration --s=10)
  die.close

Example: Network by proxy

We have used the service framework to allow providing a full network implementation from a separate container. The core of this is the NetworkService interface and the associated NetworkServiceClient:

https://github.com/toitlang/toit/blob/master/lib/system/api/network.toit

We use them to build proxying sockets like SocketResourceProxy_ that forward reads and writes to the network service:

class SocketResourceProxy_ extends ServiceResourceProxy:
  static WRITE_DATA_SIZE_MAX_ /int ::= 2048
  ...
  write data from=0 to=data.size -> int:
    to = min to (from + WRITE_DATA_SIZE_MAX_)
    return (client_ as NetworkServiceClient).socket_write handle_ data[from..to]
  ...

You can find all the helpers in the main repository:

https://github.com/toitlang/toit/blob/master/lib/system/base/network.toit

All in all, this allows a cellular driver to provide a network to all other apps that a blissfully unaware that their data flows through a good old-fashioned sequence of AT commands.

[mikkeldamsgaard]

The version is used during service discovery, because it is possible that a client will be trying to access a newer or older implementation of the service due to the fact that clients and implementations are decoupled and are likely to be updated independently of each other.

I wouldn't say that they are likely to be updated independently, it is more that they could potentially be updated independently. In all the lib code, it would all be in one go. In our application it also is not possible for a service and its interface to be updated independently. It can happen, but it is not likely. I know this is a small issue, but it is important to notice that is more the exceptional behaviour to have them that independent and not the normal behaviour....

implements RandomService

I know it is probably a matter of coding style, but I found it to generate more confusion in our services to separate out the interface instead of just having the Client class be the interface. It somehow felt convoluted to have the Definition class implement the Service interface, so we removed that and then the interface class itself was unnecessary.

The definition has its own name and versioning for debugging purposes
We just always fell back on using the same major/minor as the client, so our definition constructors looks like:

    super "cmbs.io.service" --major=CmbsIoService.MAJOR --minor=CmbsIoService.MINOR
    provides CmbsIoService.UUID CmbsIoService.MAJOR CmbsIoService.MINOR
    install

All that repetition annoys the eyes 🙂

ServiceResourceProxy.handle_, ServiceResourceProxy.on_notified_ and ServiceResource.notify_ are expected to be used in "client" code and it seems a violation of principle that they are marked private.

[kasperl]

Updated the original discussion post to reflect the fact that you (in most cases) don't need to manually wait for the service definition to be installed and that RECURSIVE_MESSAGE_PROCESSING is no longer an issue.

Can you look through your comments, @mikkeldamsgaard, and consider if any of the remarks could use an update? Thanks! :)

[robvanlopik]

Trying to understand the services framework, I run into several problems.
The first part I can follow, except for the fact that the class definition of RandomServiceProvider the line handle pid/int client/int index/int arguments/any -> any: should be handle index/int arguments/any --gid/int --client/int-> any: , in line with the new version of ServiceHandler. I also don't understand why RandomServiceProvider implements RandomService, because the methods are actually defined through the handle method. This becomes noticeable later on, where create_die has no discernable counterpart in the provider definition. And last I don't understand the syntactical construct that associates the ..._INDEX constants with the actual client methods. Is this something special of the way interface definitions work? Anyway, I can make the example code work and have also managed to make an MQTT publish service (without the datacake complexities).

Going to Service Resources things get more complicated. In the example I miss the place where the constant CREATE_DIE_INDEX is defined, so I added it to the interface definition. In the end the line return client_.roll_die handle_ in class RandomServiceClient returns a compiler error because client_ is an instance of ServiceClient that doesn't understand roll_die. Conceptually I don't even see why you would want to have a 'resource' when you have to route calls to methods on the resource (roll) through calls to the service client (roll_die). In that case it is easier to make the resource a simple instance variable of the ServiceProvider (or rather in an instance variable that is a map containing a resource for each client)

Finally I tried the notification example. but I am afraid I don't really grasp how this is supposed to work.

All this arises because of my attempt to create a simple MQTT service that works both ways. Any clarification would be welcome

[kasperl]

Thank you for the detailed feedback, @robvanlopik! We changed the handle method signature recently and forgot to update this discussion. I've tried to remedy this.

The service resources are admittedly more complex. The resources are kept alive by clients (typically out of process) until closed, so the system keeps track of when these clients terminate. The ServiceResource/ServiceResourceProxy pair is an object and a cross-process reference to it with some reasonable cleanup semantics. There can be many resources per client and we use a randomized handle (of type int) for each of them.

I'll think about how to make the notification example easier to grok, but maybe it is best to just extend the examples/ with something you can actually run?