InteractingWithTheKnowledgeBase

The Knowledge Base

The main entry point into the MADARA middleware is the Knowledge Base. The Knowledge Base supports a variety of programming styles and interactions, and we'll discuss many of those capabilities on this page.

1. Knowledge Base Creation

There are two constructors for knowledge bases that are used most frequently by developers. The first is a default constructor which takes no arguments.

This constructor creates a knowledge base without a networking transport and is ideal for local reasoning using the Knowledge and Reasoning Language or for a knowledge base that is shared between threads for multi-threaded applications. Local variables and global variables with the default constructor will not be disseminated on a transport, but will be available to any thread with a handle to the knowledge base.

Example of default constructor

#include "madara/knowledge/KnowledgeBase.h"

int main (int argc, char ** argv)
{
  madara::knowledge::KnowledgeBase knowledge;

  // set our id to 0 and let other threads know that we are ready
  knowledge.set (".id", madara::knowledge::KnowledgeRecord::Integer (0));
  knowledge.set ("agent{.id}.ready");

  return 0;
}

The second major constructor is for creating a networked knowledge base using one of the supported transports (UDP, Multicast, Broadcast, Open Splice DDS, and RTI DDS). This constructor takes two parameters: 1) a unique identifier and a 2) QoSTransportSettings class.

If the unique identifier is an empty string (""), then a default unique identifier is created using the hostname set in the OS and an ephemeral port binding. The transport settings class contains dozens of configuration options for the transport layer. We'll discuss these options a bit more in the next wiki page.

Example of UDP transport constructor

#include "madara/knowledge/KnowledgeBase.h"

int main (int argc, char ** argv)
{
  // setup a UDP transport with ourself at port 45000 and a
  // second agent at port 45001 on localhost (127.0.0.1)
  madara::transport::QoSTransportSettings settings;
  settings.hosts.push_back ("127.0.0.1:45000");
  settings.hosts.push_back ("127.0.0.1:45001");
  settings.type = Madara::transport::UDP;

  // use the transport constructor 
  madara::knowledge::KnowledgeBase knowledge ("", settings);

  // set our id to 0 and let the other agent know that we are ready
  knowledge.set (".id", madara::knowledge::KnowledgeRecord::Integer (0));
  knowledge.set ("agent{.id}.ready");

  return 0;
}

Example of multicast transport constructor Multicast IP range

#include "madara/knowledge/KnowledgeBase.h"

int main (int argc, char ** argv)
{
  // setup a Multicast transport at 239.255.0.1:4150
  madara::transport::QoSTransportSettings settings;
  settings.hosts.push_back ("239.255.0.1:4150");
  settings.type = madara::transport::MULTICAST;

  // use the transport constructor 
  madara::knowledge::KnowledgeBase knowledge ("", settings);

  // set our id to 0 and let the other agent know that we are ready
  knowledge.set (".id", madara::knowledge::KnowledgeRecord::Integer (0));
  knowledge.set ("agent{.id}.ready");

  return 0;
}

Example of broadcast transport constructor

#include "madara/knowledge/KnowledgeBase.h"

int main (int argc, char ** argv)
{
  // setup a Broadcast transport at "192.168.1.255:15000"
  // The broadcast ip needs to be appropriate for your IP subnet mask
  madara::transport::QoSTransportSettings settings;
  settings.hosts.push_back ("192.168.1.255:15000");
  settings.type = madara::transport::BROADCAST;

  // use the transport constructor 
  madara::knowledge::KnowledgeBase knowledge ("", settings);

  // set our id to 0 and let the other agent know that we are ready
  knowledge.set (".id", madara::knowledge::KnowledgeRecord::Integer (0));
  knowledge.set ("agent{.id}.ready");

  return 0;
}

In order to the use the Open Splice transport, you have to install Open Splice DDS. There are instructions on how to do this on the InstallationFromSource wiki page.

Example of Open Splice DDS transport constructor

#include "madara/knowledge/KnowledgeBase.h"

int main (int argc, char ** argv)
{
  // setup a reliable Open Splice DDS transport
  madara::transport::QoSTransportSettings settings;
  settings.type = madara::transport::SPLICE;
  settings.reliability = madara::transport::RELIABLE;

  // use the transport constructor 
  madara::knowledge::KnowledgeBase knowledge ("", settings);

  // set our id to 0 and let the other agent know that we are ready
  knowledge.set (".id", madara::knowledge::KnowledgeRecord::Integer (0));
  knowledge.set ("agent{.id}.ready");

  return 0;
}

For more information on interacting with the Transport Settings class, see InteractingWithTheTransport.

2. Evaluate, Get, Set, and Wait

2.a. Get, Set, and Set_Index

The most basic manner to interact with the knowledge base is as a real-time dictionary. In the following example, we show how to do basic gets and sets and also how to aggregate updates for semantic consistency. The knowledge base natively supports storing integers, doubles, vectors of the same, strings, and blobs. It can also store custom user-defined types.

Example of Using Get and Set

#include "madara/knowledge/KnowledgeBase.h"
#include "madara/utility/Utility.h"
#include "Some_GPS_Library.h"

int main (int argc, char ** argv)
{
  // setup a Multicast transport at 239.255.0.1:4150
  madara::transport::QoSTransportSettings settings;
  settings.hosts.push_back ("239.255.0.1:4150");
  settings.type = Madara::Transport::MULTICAST;

  // Use default eval settings, which are different than the
  // defaults that are passed to set
  madara::knowledge::EvalSettings eval_settings;

  // use the transport constructor 
  madara::knowledge::KnowledgeBase knowledge ("", settings);

  // set the local variable .id to the first command line argument
  if (argc ># 2)
    knowledge.set (".id", argv[1]);

  // set the global variable agent{.id}.ready
  knowledge.set ("agent{.id}.ready", eval_settings);

  // delay sending modifications for next few set commands
  eval_settings.delay_sending_modifieds = true;

  // use some third party library to get our current GPS position
  // this is NOT a MADARA function and is used as an example of working
  // with arbitrary libraries or code bases
  Some_GPS_Library::GPS gps_coords = Some_GPS_Library::get_current_gps ();

  // now use these gps_coords to update other agents on your position
  // we've set the eval settings to delay sending modifications until
  // we ask it to no longer delay during a set, eval, or wait command
  // or if we explicitly call send_modifieds

  // here we create an array of 3 doubles called "agent{.id}. position". By setting
  // the last index (2) first, we allocate the full array instead of
  // growing it from 0 to 2, which would be much slower.
  knowledge.set_index ("agent{.id}.position", 2, gps_coords.get_altitude (), eval_settings);
  knowledge.set_index ("agent{.id}.position", 0, gps_coords.get_latitude (), eval_settings);
  knowledge.set_index ("agent{.id}.position", 1, gps_coords.get_longitude (), eval_settings);

  // aggregate a nanosecond timestamp with the update of position
  knowledge.set ("agent{.id}.position.timestamp", madara::utility::get_time (), eval_settings);

  // send all modifications in the same packet for consistency
  knowledge.send_modifieds ();
}

2.b. Share

See KnowledgeBase Data Sharing for full explanation.

The KnowledgeBase is capable of storing large objects: strings, integer arrays, double arrays, and binary files (i.e., blobs). The KnowledgeBase avoids copying these large values as much as possible, by storing them behind std::shared_ptrs. Methods such as to_string and to_integers will create a copy of the large value, returning it as the underlying type (std::string or std::vector<...>). To gain a copy of the std::shared_ptr itself, and avoid copying those large values, use the methods of the form share_*, such as share_string and share_integers. These methods can be called on the KnowledgeBase directly, taking a std::string key or VariableReference, or on the KnowledgeRecord returned by KnowledgeBase::get.

#include "madara/knowledge/KnowledgeBase.h"
using namespace madara::knowledge;

...

KnowledgeBase kb;

std::string str = "This is a string that might be much longer and be expensive to copy.";

 // Creates a vector with 4000 entries, all equal to 42, without any copying,
 // by calling the std::vector<int64_t> constructor in-place within the record
KnowledgeRecord ints(tags::integers, 4000, 42);

std::unique_ptr<std::vector<double>> unique_dptr (
  new std::vector<double> (iptr->begin(), iptr->end()));
KnowledgeRecord dbls(tags::shared(tags::doubles), std::move(unique_dptr));

kb.set(".my_string", std::move(big_str)); // std::move avoids copying the string data
kb.set(".my_array", ints);                // std::move not needed here to avoid copying the data,
kb.set(".my_doubles", std::move(dbls));   // but would be slightly more efficient as it would
                                          // avoid touching ref counts

// shared with .my_string still in kb
std::shared_ptr<const std::string> str_out = kb.share_string(".my_string");

// shared with .my_array still in kb
std::shared_ptr<const std::vector<int64_t>> ints_out = kb.share_integers(".my_array");

kb.set_index(".my_array", 0, 47); // Causes a copy to be made within the KnowledgeBase,
assert(ints_out->at(0) == 42);    // so we can modify within kb without changing ints_out

2.c. Evaluate and Wait

Developers are free to use get and set-based operations exclusively to minimize interaction with MADARA except for storing and transferring data between agents in a network. However, many features in MADARA are only available through the evaluate and wait calls.

Evaluate and Wait are mechanisms for evaluating Knowledge and Reasoning Language (KaRL) logic, in order to perform artificial intelligence operations, call user functions, and wait for new information to come in from the network. The main difference between evaluate and wait is that wait will block the calling application until a condition is true (non-zero).

Example showcasing the difference between evaluate and wait

#include "madara/knowledge/KnowledgeBase.h"
#include "madara/utility/Utility.h"
#include "Some_GPS_Library.h"

int main (int argc, char ** argv)
{
  // setup a Multicast transport at 239.255.0.1:4150
  madara::transport::QoSTransportSettings settings;
  settings.hosts.push_back ("239.255.0.1:4150");
  settings.type = madara::transport::MULTICAST;

  // use the transport constructor 
  madara::knowledge::KnowledgeBase knowledge ("", settings);

  // set the local variable .id to the first command line argument
  if (argc >= 2)
    knowledge.set (".id", argv[1]);

  // assuming .id 0, then will set agent0.ready to 1
  knowledge.evaluate ("agent{.id}.ready = 1");

  // in this advanced wait statement, we create a local counter called
  // .agents_ready and we increment this counter whenever we find an
  // agent ready. We use the choose right operator ";>" to tell the
  // wait statement to choose the right most predicate as its truth
  // condition. In this case, that condition is ".agents_ready > 5"
  knowledge.wait (
    ".agents_ready = 0 ;> "
    ".i [0->10) (agent{.i}.ready => ++.agents_ready) ;> "
    "agent{.id}.ready = 1 ;> "
    ".agents_ready > 5"  // right most is the conditional because of ;>
  );

  // Once we get to this point in the code, we have at least 6 agents
  // online and ready to conduct AI operations. Now, we create a
  // state machine that each agent will do.

  while (knowledge.get ("terminated").to_integer () == 0)
  {
    knowledge.evaluate (

      // call functions that sense the environment and update our location
      "sense_environment ();"
      "update_location ();"

      // choose what to do next based on a variable called 'state'
      "state 'hover' => hover ()"
      "||"
      "state 'land' => land ()"
      "||"
      "state 'takeoff' => takeoff ()"
      "||"
      "state 'move' => move () ;"

      // if battery level is low, terminate
      "battery_level < 10 => terminated = 1"
  }

2.d. Evaluate and Wait Settings

A call to evaluate, get, and set can be controlled and augmented by modifying the EvalSettings. The class WaitSettings extends the Eval_Settings class by adding specific controls for the wait function.

The following example shows how to configure a wait statement with various settings available through Eval_Settings and Wait_Settings.

Example of Configuring Wait with the Wait Settings class

  madara::knowledge::WaitSettings wait_settings;

  // pre_print and post_print are atomic operations that occur just
  // before and after, respectively, an evaluate, set, get, or wait call
  wait_settings.pre_print_statement = "The value of count before eval is {count}.\n";
  wait_settings.post_print_statement = "The value of count after eval is {count}.\n";

  // this variable indicates that changes to global variables will
  // not be disseminated over the network and will instead be treated
  // as a local change
  wait_settings.treat_globals_as_locals = true;
  
  // poll for truth every 0.5 seconds
  wait_settings.poll_frequency = 0.5;

  // wait for truth for at most 10 seconds
  wait_settings.max_wait_time = 10.0;

  // increase a counter and wait on the value of finished
  // this is possible because of the choose right operator (;>) which
  // executes the left side, then the right side, and returns the value
  // of the right side
  knowledge.wait ("++count ;> finished", wait_settings);

For all options on these settings classes, see the documentation on WaitSettings and EvalSettings.

3. Function Calls

Developers can create their own C++ or Java functions and call them from inside of an evaluate or wait call.

Example of calling a C++ user function

#include "madara/knowledge/KnowledgeBase.h"
#include "Some_GPS_Library.h"

madara::knowledge::Knowledge_Record
  move_to_gps (madara::knowledge::Function_Arguments & args,
            madara::knowledge::Variables & variables)
{
  madara::knowledge::Knowledge_Record result;

  if (args.size () == 3)
  {
    // assuming the GPS_Library::move_to_gps function returns
    // some kind of integer, double, or string
    result.set_value (GPS_Library::move_to_gps (
      args[0].to_string (), // latitude
      args[1].to_string (), // longitude
      args[2].to_string ()  // altitude);
  }

  return result;
}

int main (int argc, char ** argv)
{
  // We don't need a networking transport for this example
  madara::knowledge::Knowledge_Base knowledge;

  // define a function call which is called "move_to_gps" inside
  // of evaluate and wait functions and maps to a C++ function called
  // move_to_gps
  knowledge.define_function ("move_to_gps", move_to_gps);

  // call the mapped "move_to_gps" function with a latitude,
  // longitude, and altitude.
  knowledge.evaluate ("move_to_gps (35.336488, -87.539062, 2000.0)");

  return 0;
}

4. System Calls

System calls in MADARA are much like function calls, but system calls are slightly faster and mapped to internal MADARA functions that provide information about internal knowledge records, time, quality, printing, utility functions, and even the ability to evaluate other logics.

Example of various MADARA system calls

#include "madara/knowledge/KnowledgeBase.h"

int main (int argc, char ** argv)
{
  madara::knowledge::Knowledge_Base knowledge;

  knowledge.evaluate (
    // get time in nanoseconds
    "cur_time = #get_time (); "

    // convert nanosecond time to a string.  Can also use #string
    "str_cur_time = #to_string (cur_time); "

    // convert a directory structure to the host directory format
    // on Unix OSes, nothing would change, but on Windows, this would
    // be changed from '/' to '\'
    "file_name = #to_host_dirs ('/file/dir/structure/file.jpg'); "

    // read the file referenced by filename
    "buffer = #read_file (file_name); "

    // save the size of the file we read into a variable called file_size
    "file_size = #size (buffer); "

    // print all system call documentation.
    "print_system_calls (); "

    // write our file buffer to a different file
    "#write_file (#to_host_dirs ('/images/file.jpg'), buffer); "

    // generate a uniform random number between 1 and 8 (inclusive)
    "some_random_int = #rand_int (1, 8); "

    // generate a uniform random double between 2 and 2000 (inclusive)
    "some_random_double = #rand_double (2.0, 2000.0); "
  );

  return 0;
}

5. Checkpointing

MADARA provides powerful debugging options for saving and loading contexts with a file store. In this section, we describe how to save the full context or an incremental checkpoint, and also how to load state from these files.

5.a. Saving the Full Context

Saving everything in a Knowledge_Base to a file is done with the save_context function, as shown below. This will save both global and local variables.

Example of saving context

madara::knowledge::Knowledge_Base knowledge ("", settings);

// make changes to the context or have changes come in over the network
...

// we recommend saving to a "kkb" file -- KaRL Knowledge Base to help
// with sorting/visualization in a directory. This command saves every
// local and global variable to the context file.
knowledge.save_context ("recent_context.kkb");

5.b. Saving a Checkpoint

In contrast to save_context, save_checkpoint only saves updates since the last save_context or save_checkpoint. This function is far more efficient than save_context, especially for a large Knowledge_Base.

Saving only global changes since last full save

Example of saving checkpoint after save context

madara::knowledge::KnowledgeBase knowledge ("", settings);

// make changes to the context or have changes come in over the network
...

// Save everything to a file
knowledge.save_context ("recent_context.kkb");

// make additional changes to the context
...

// Append everything since save_context to the end of the file.
knowledge.save_checkpoint ("recent_context.kkb");

Important note about checkpoint: Checkpointing will only save global variable changes. In order to save local variable changes, you will need to do evaluates, sets, and waits with track_local_changes enabled in the Eval Settings.

5.c. Loading a Checkpoint

MADARA provides the function load_context to read a context from a file. The function provides optional parameters to allow for either resetting the context to the contents of the file or simply adding the file's contents into the context, in addition to or overwriting whatever exists in the context.

Loading Context

madara::knowledge::KnowledgeBase knowledge ("", settings);

// Load context from a file. The second parameter is the decision of
// whether or not to load the Knowledge Base's id from the file. We
// say false to indicate we want to keep the default id generated by
// the Knowledge Base constructor, but if we had indicated true as the
// second parameter, we would use the id that the agent1_context.kkb had
// used. A third parameter is available which takes an Eval_Settings
// class. The default settings here treat all updates as local updates
// (i.e., changes will not be disseminated as new updates), but this is
// tunable if you want to specify your own Eval Settings class.
knowledge.load_context ("agent1_context.kkb", false);

More Information

If you are looking for code examples and guides, your best bet would be to start with the tutorials (located in the tutorials directory of the MADARA root directory--see the README.txt file for descriptions). After that, there are many dozens of tests that showcase and profile the many functions, classes, and functionalities available to MADARA users.

Users may also browse the Library Documentation for all MADARA functions, classes, etc. and the Wiki pages on this website.

Video Tutorials

First Steps in MADARA: Covers installation and a Hello World program.
Intro to Networking: Covers creating a multicast transport and coordination and image sharing between multiple agents.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly