PythonInteractingWithTheKnowledgeBase

Python Guide Series
Architecture | Knowledge Base | Networking | Containers

The MADARA Knowledge Base

The main entry point into the MADARA middleware is the Knowledge Base. The Knowledge Base supports a variety of programming styles and interactions, and we'll discuss many of those capabilities on this page.

1. Knowledge Base Creation

There are two constructors for knowledge bases that are used most frequently by developers. The first is a default constructor which takes no arguments.

This constructor creates a knowledge base without a networking transport and is ideal for local reasoning using the Knowledge and Reasoning Language or for a knowledge base that is shared between threads for multi-threaded applications. Local variables and global variables with the default constructor will not be disseminated on a transport, but will be available to any thread with a handle to the knowledge base.

Example of default constructor

# create module alias for the knowledge_engine submodule
import madara.knowledge as engine;

# create a knowledge base with no transport
knowledge = engine.KnowledgeBase ();

# set our id to 0 and let other threads know that we are ready
knowledge.set(".id", 0);
knowledge.set("agent{.id}.ready", 1);

return 0;

The second major constructor is for creating a networked knowledge base using one of the supported transports (UDP, Multicast, Broadcast, Open Splice DDS, and RTI DDS). This constructor takes two parameters: 1) a unique identifier and a 2) QoSTransportSettings class.

If the unique identifier is an empty string (""), then a default unique identifier is created using the hostname set in the OS and an ephemeral port binding. The transport settings class contains dozens of configuration options for the transport layer. We'll discuss these options a bit more in the next wiki page.

Example of UDP transport constructor

# import the submodules and give them shorter aliases
import madara.knowledge as engine;
import madara.transport as transport;

# Create transport settings for a multicast transport
settings = transport.QoSTransportSettings();
settings.hosts.append("127.0.0.1:45000");
settings.hosts.append("127.0.0.1:45001");
settings.type = transport.TransportTypes.UDP;

# create a knowledge base with the multicast transport settings
knowledge = engine.KnowledgeBase("agent1", settings);

# set our id to 0 and let the other agent know that we are ready
knowledge.set(".id", 0);
knowledge.set("agent{.id}.ready", 1);

Example of multicast transport constructor Multicast IP range

# import the submodules and give them shorter aliases
import madara.knowledge as engine;
import madara.transport as transport;

# Create transport settings for a multicast transport
settings = transport.QoSTransportSettings();
settings.hosts.append("239.255.0.1:4150");
settings.type = transport.TransportTypes.MULTICAST;

# create a knowledge base with the multicast transport settings
knowledge = engine.KnowledgeBase("agent1", settings);

# set our id to 0 and let the other agent know that we are ready
knowledge.set(".id", 0);
knowledge.set("agent{.id}.ready", 1);

Example of broadcast transport constructor

# import the submodules and give them shorter aliases
import madara.knowledge as engine;
import madara.transport as transport;

# Create transport settings for a multicast transport
settings = transport.QoSTransportSettings();
settings.hosts.append("192.168.1.255:15000");
settings.type = transport.TransportTypes.BROADCAST;

# create a knowledge base with the multicast transport settings
knowledge = engine.KnowledgeBase("agent1", settings);

# set our id to 0 and let the other agent know that we are ready
knowledge.set(".id", 0);
knowledge.set("agent{.id}.ready", 1);

For more information on interacting with the Transport Settings class, see PythonInteractingWithTheTransport.

2. Evaluate, Get, Set, and Wait

2.a. Get, Set, and Set_Index

The most basic manner to interact with the knowledge base is as a real-time dictionary. In the following example, we show how to do basic gets and sets and also how to aggregate updates for semantic consistency.

Example of Using Get and Set

# import the submodules and give them shorter aliases
import madara.knowledge as engine;
import madara.transport as transport;
import sys;
import some_gps_module;

# Create transport settings for a multicast transport
settings = transport.QoS_TransportSettings();
settings.hosts.append("239.255.0.1:4150");
settings.type = transport.TranportTypes.MULTICAST;

# Use default eval settings, which are different than the
# defaults that are passed to set
eval_settings = engine.EvalSettings();

# use the transport constructor 
knowledge = engine.KnowledgeBase("", settings);

# set the local variable .id to the first command line argument
if len(sys.argv) >= 2:
  knowledge.set(".id", sys.argv[1]);

# set the global variable agent{.id}.ready
knowledge.set ("agent{.id}.ready", 1, eval_settings);

# delay sending modifications for next few set commands
eval_settings.delay_sending_modifieds = 1;

# use some third party library to get our current GPS position
# this is NOT a MADARA function and is used as an example of working
# with arbitrary libraries or code bases
gps_coords = some_gps_module.get_current_gps();

# now use these gps_coords to update other agents on your position
# we've set the eval settings to delay sending modifications until
# we ask it to no longer delay during a set, eval, or wait command
# or if we explicitly call send_modifieds

# here we create an array of 3 doubles called "agent{.id}. position". By setting
# the last index (2) first, we allocate the full array instead of
# growing it from 0 to 2, which would be much slower.
knowledge.set_index("agent{.id}.position", 2, gps_coords.get_altitude(), eval_settings);
knowledge.set_index("agent{.id}.position", 0, gps_coords.get_latitude(), eval_settings);
knowledge.set_index("agent{.id}.position", 1, gps_coords.get_longitude(), eval_settings);

# aggregate a nanosecond timestamp with the update of position
knowledge.set("agent{.id}.position.timestamp", madara.utility.get_time(), eval_settings);

# send all modifications in the same packet for consistency
knowledge.send_modifieds();

2.b. Evaluate and Wait

Developers are free to use get and set-based operations exclusively to minimize interaction with MADARA except for storing and transferring data between agents in a network. However, many features in MADARA are only available through the evaluate and wait calls.

Evaluate and Wait are mechanisms for evaluating Knowledge and Reasoning Language (KaRL) logic, in order to perform artificial intelligence operations, call user functions, and wait for new information to come in from the network. The main difference between evaluate and wait is that wait will block the calling application until a condition is true (non-zero).

Example showcasing the difference between evaluate and wait

# import the submodules and give them shorter aliases
import madara.knowledge as engine;
import madara.transport as transport;
import sys;
import some_gps_module;

# Create transport settings for a multicast transport
settings = transport.QoS_TransportSettings();
settings.hosts.append("239.255.0.1:4150");
settings.type = transport.TranportTypes.MULTICAST;

# use the transport constructor 
knowledge = engine.KnowledgeBase("", settings);

# set the local variable .id to the first command line argument
if len(sys.argv) >= 2:
  knowledge.set(".id", sys.argv[1]);

# assuming .id 0, then will set agent0.ready to 1
knowledge.evaluate("agent{.id}.ready = 1");

# in this advanced wait statement, we create a local counter called
# .agents_ready and we increment this counter whenever we find an
# agent ready. We use the choose right operator ";>" to tell the
# wait statement to choose the right most predicate as its truth
# condition. In this case, that condition is ".agents_ready > 5"
knowledge.wait(
  ".agents_ready = 0 ;> "
  ".i [0->10) (agent{.i}.ready => ++.agents_ready) ;> "
  "agent{.id}.ready = 1 ;> "
  ".agents_ready > 5"  # right most is the conditional because of ;>
);

# Once we get to this point in the code, we have at least 6 agents
# online and ready to conduct AI operations. Now, we create a
# state machine that each agent will do.
while knowledge.get ("terminated").is_false ():
  knowledge.evaluate (
    # call functions that sense the environment and update our location
    "sense_environment ();"
    "update_location ();"

    # choose what to do next based on a variable called 'state'
    "state 'hover' => hover ()"
    "||"
    "state 'land' => land ()"
    "||"
    "state 'takeoff' => takeoff ()"
    "||"
    "state 'move' => move () ;"

    # if battery level is low, terminate
    "battery_level < 10 => terminated = 1"
  );

2.c. Evaluate and Wait Settings

A call to evaluate, get, and set can be controlled and augmented by modifying the EvalSettings. The class WaitSettings extends the EvalSettings class by adding specific controls for the wait function.

The following example shows how to configure a wait statement with various settings available through EvalSettings and WaitSettings.

Example of Configuring Wait with the Wait Settings class

import madara.knowledge as engine;
wait_settings = engine.WaitSettings ();

# pre_print and post_print are atomic operations that occur just
# before and after, respectively, an evaluate, set, get, or wait call
wait_settings.pre_print_statement = "The value of count before eval is {count}.\n";
wait_settings.post_print_statement = "The value of count after eval is {count}.\n";

# this variable indicates that changes to global variables will
# not be disseminated over the network and will instead be treated
# as a local change
wait_settings.treat_globals_as_locals = 1;
  
# poll for truth every 0.5 seconds
wait_settings.poll_frequency = 0.5;

# wait for truth for at most 10 seconds
wait_settings.max_wait_time = 10.0;

# increase a counter and wait on the value of finished
# this is possible because of the choose right operator (;>) which
# executes the left side, then the right side, and returns the value
# of the right side
knowledge.wait("++count ;> finished", wait_settings);

For all options on these settings classes, see the documentation on WaitSettings and EvalSettings.

3. Function Calls

Developers can create their own Python functions and call them from inside of an evaluate or wait call.

Example of calling a Python function

# import the submodules and give them shorter aliases
import madara.knowledge as engine;
import some_gps_module;

def  move_to_gps (args, variables)
{
  result = madara.knowledge.KnowledgeRecord;

  if len(args) 3:
    # assuming the GPS_Library::move_to_gps function returns
    # some kind of integer, double, or string
    result.set_value(some_gps_module.move_to_gps (
      args[0].to_string (), # latitude
      args[1].to_string (), # longitude
      args[2].to_string ()  # altitude);

  return result;

# We don't need a networking transport for this example
knowledge = engine.KnowledgeBase();

# define a function call which is called "move_to_gps" inside
# of evaluate and wait functions and maps to a C++ function called
# move_to_gps
knowledge.define_function("move_to_gps", move_to_gps);

# call the mapped "move_to_gps" function with a latitude,
# longitude, and altitude.
knowledge.evaluate("move_to_gps (35.336488, -87.539062, 2000.0)");

4. System Calls

System calls in MADARA are much like function calls, but system calls are slightly faster and mapped to internal MADARA functions that provide information about internal knowledge records, time, quality, printing, utility functions, and even the ability to evaluate other logics.

Example of various MADARA system calls

# import the submodules and give them shorter aliases
import madara.knowledge as engine;
knowledge = engine.KnowledgeBase();

knowledge.evaluate (
  # get time in nanoseconds
  "cur_time = #get_time (); "

  # convert nanosecond time to a string.  Can also use #string
  "str_cur_time = #to_string (cur_time); "

  # convert a directory structure to the host directory format
  # on Unix OSes, nothing would change, but on Windows, this would
  # be changed from '/' to '\'
  "file_name = #to_host_dirs ('/file/dir/structure/file.jpg'); "

  # read the file referenced by filename
  "buffer = #read_file (file_name); "

  # save the size of the file we read into a variable called file_size
  "file_size = #size (buffer); "

  # print all system call documentation.
  "print_system_calls (); "

  # write our file buffer to a different file
  "#write_file (#to_host_dirs ('/images/file.jpg'), buffer); "

  # generate a uniform random number between 1 and 8 (inclusive)
  "some_random_int = #rand_int (1, 8); "

  # generate a uniform random double between 2 and 2000 (inclusive)
  "some_random_double = #rand_double (2.0, 2000.0); "
);

5. Checkpointing

MADARA provides powerful debugging options for saving and loading contexts with a file store. In this section, we describe how to save the full context or an incremental checkpoint, and also how to load state from these files.

5.a. Saving the Full Context

Saving everything in a KnowledgeBase to a file is done with the save_context function, as shown below. This will save both global and local variables.

Example of saving context

# import the submodules and give them shorter aliases
import madara.knowledge as engine;
knowledge = engine.KnowledgeBase();
# make changes to the context or have changes come in over the network
...

# we recommend saving to a "kkb" file -- KaRL Knowledge Base to help
# with sorting/visualization in a directory. This command saves every
# local and global variable to the context file.
knowledge.save_context("recent_context.kkb");

5.b. Saving a Checkpoint

In contrast to save_context, save_checkpoint only saves updates since the last save_context or save_checkpoint. This function is far more efficient than save_context, especially for a large Knowledge_Base.

Saving only global changes since last full save

Example of saving checkpoint after save context

# import the submodules and give them shorter aliases
import madara.knowledge as engine;
knowledge = engine.KnowledgeBase();

# make changes to the context or have changes come in over the network
...

# Save everything to a file
knowledge.save_context("recent_context.kkb");

# make additional changes to the context
...

# Append everything since save_context to the end of the file.
knowledge.save_checkpoint("recent_context.kkb");

Important note about checkpoint: Checkpointing will only save global variable changes. In order to save local variable changes, you will need to do evaluates, sets, and waits with track_local_changes enabled in the Eval Settings.

5.c. Loading a Checkpoint

MADARA provides the function load_context to read a context from a file. The function provides optional parameters to allow for either resetting the context to the contents of the file or simply adding the file's contents into the context, in addition to or overwriting whatever exists in the context.

Loading Context

# import the submodules and give them shorter aliases
import madara.knowledge as engine;
knowledge = engine.KnowledgeBase();

# Load context from a file. The second parameter is the decision of
# whether or not to load the Knowledge Base's id from the file. We
# say false to indicate we want to keep the default id generated by
# the Knowledge Base constructor, but if we had indicated true as the
# second parameter, we would use the id that the agent1_context.kkb had
# used. A third parameter is available which takes an EvalSettings
# class. The default settings here treat all updates as local updates
# (i.e., changes will not be disseminated as new updates), but this is
# tunable if you want to specify your own Eval Settings class.
knowledge.load_context("agent1_context.kkb", false);

More Information

The Python module and all submodules, classes, and functions are documented to work with the Python help system. To start browsing the inline help, try the following inside of a python interpreter:

import madara;
help (madara);
help (madara.knowledge);

If you are looking for C++ code examples and guides to supplement Python knowledge, your best bet would be to start with the tutorials (located in the tutorials directory of the MADARA root directory--see the README.txt file for descriptions). After that, there are many dozens of tests that showcase and profile the many functions, classes, and functionalities available to MADARA users.

Users may also browse the Library Documentation for all MADARA functions, classes, etc. and the Wiki pages on this website.