Skip to content

Java Custom KnowledgeRecord Types

Jump to bottom Edit New page
James Edmondson edited this page Apr 3, 2020 · 5 revisions

FEATURE DEPRECATED

As for 3.3.0, this feature is deprecated. We specifically removed this due to complications with Boost long term support and the requirement of boost filesystem, which was causing issues with moving to UE4 as a simulation environment for GAMS. If you are interested in this feature, let us know, and we will try to prioritize its reinclusion. However, there is no planned resurrection for this tool at the moment.

This feature lives in v3.2.3, at the latest.

Madara's KnowledgeBase and KnowledgeRecord support several useful types natively: integers, doubles, vectors of both, strings, and blobs. Where appropriate, it is best to use those types. They will be accessible by any KnowledgeBase, and a variety of useful functions will work with them, including overloaded math operators and indexing operations. For types which cannot easily be represented by those native types, KnowledgeRecord, and thus KnowledgeBase, can store an Any type, provided by Madara, which can in turn store custom types directly.

Table of Contents

Supported Types

There are two main kinds of objects supported by Any: native C++ types, and Cap'n Proto messages. Supported native C++ types must be default constructible, copy constructible, and support serialization via either the Cereal Library, or via Madara's translation system to a compatible Cap'n Proto message struct. Note that this translation system is an alternative to using Cap'n Proto generated types and schemas directly.

Types must be registered with Madara using a string "tag" which identifies them. This tag should be used on all systems and platforms to refer to the same kind of data, but each might register different types, as long as they have the same serialized format.

To be used from Java, a native C++ custom type must be registered within C++ code, or with the predefined Any.register* functions provided in the Java port for various commonly used types.

Cap'n Proto generated types can be registered within Java code, and thus do not rely on C++ registration code. Madara uses the capnproto-java Java port of Cap'n Proto. Be sure to familiarize yourself with its documentatation. Most usage of these types will be driven by that library.

Loading Native C++ External Types

Custom types may only be defined in C++, but can be made available to Java via a shared library. To load a shared library, use the System.LoadLibrary() call. To ensure successful linking, ensure libMADARA.so is loaded first. For example:

public static void main (...)
{
  System.loadLibrary("MADARA"); // note that "lib" and ".so" are omitted
  System.loadLibrary("datatypes"); // replace with your library's name

  ...
}

Registering Cap'n Proto Generated Types

To store and load Cap'n Proto messages to and from Any objects in Java, they must be registered with the Any.registerClass() static method. It takes two arguments. The first is is a string "tag" which will be used to portably identify the type across all systems using it. The second is the Cap'n Proto generated Factory instance. For example, using the generated classes for the schema in $MADARA_ROOT/tests/capnfiles/Point.capn:

import org.capnproto.MessageBuilder;
import ai.madara.tests.capnp.Geo;

// In main, or a function called from it:
Any.registerClass("Point", Geo.Point.factory);

Note that you must register the Java factory even if the tag is registered in C++ or Python already. You will not be able to use any Cap'n Proto message in Java with Madara that hasn't been registered in Java.

The Any Class

The Java port provides a version of the C++ implementation of Any.

To create an Any holding a Cap'n Proto message, construct it with a capnproto-java MessageBuilder, and pass the type's factor, and MessageBuilder into the constructor (or emplace method of an existing Any):

MessageBuilder msg = new MessageBuilder();
Geo.Point.Builder builder = msg.initRoot(Geo.Point.factory);
builder.setX(12);
builder.setY(32);
builder.setZ(47);
Any any = new Any(Geo.Point.factory, msg);

To create an Any holding a native C++ type, construct it with the name of a registered class:

/**** C++ code ****/
struct Point {
  double x, y, z;
};
// Some details omitted, see C++ version documentation for more.

// Register the above
Any::register_type<Point>("point");
/**** Java code ****/
// Register C++ type std::map<std::string, std::string> with tag "smap"
Any.registerStringToStringMap("smap");

Any point = new Any("point");
Any map = new Any("smap");

These Any objects now hold ownership of a default constructed C++ object. When you are finished with an Any, call its free() method to destruct that object, and free its memory. A Java Any object will call free() automatically from its finalize() method when garbage collected, but this should not be relied upon. GC in Java is non-deterministic, and might not manage the Any object properly since it will only be aware of the relatively small Java object itself, not the arbitrarily large C++ object it owns.

Accessing Cap'n Proto message Any Objects

The Any object provides a reader() method which will return a capnproto-java reader object for the held object. If the held object is not a registered Cap'n Proto message, an exception will be thrown. For example:

Geo.Point.Reader reader = any.reader(Geo.Point.factory);
assert reader.getX() == 12;

Manipulating Native C++ Any Objects

To access fields and contained elements of the held object, Any provides several methods. To access a field, use the ref() method, with the name of the field as a string:

AnyRef xref = point.ref("x");

The AnyRef acts much like Any itself, and has most of the same methods, but doesn't own the object it points to. It also does not extend the lifetime of that object. It acts like a C pointer. If the Any it was constructed from is destroyed, it will be left dangling, so be careful keeping these objects long-term.

You can modify the data held by the C++ object pointed to by an AnyRef using the assign() method:

xref.assign(2);
point.ref("y").assign(3.5);

An AnyRef (or Any itself) can be converted to various types, the ones supported natively by KnowledgeRecord. Typically, you'll use to_integer(), to_double(), or to_string():

double x = xref.to_double(); // == 2
double y = point.ref("y").to_double(); // == 3.5

You can use the at() method to access elements of the held type, if it supports indexing by integer or string:

AnyRef fooRef = map.at("foo"); // Creates a new entry in the map
map.at("bar").assign("world");
fooRef.assign("hello");
String foo = fooRef.to_string(); // == "Hello"
String bar = map.at("bar").to_string(); // == "World"

You can also call size() to call that method on the held object, if it supports it, and list_fields() to get a list of field names supported by ref().

Any with KnowledgeRecord and KnowledgeBase

You can store Any within KnowledgeRecord, and in turn into KnowledgeBase. To store an Any in a KnowledgeRecord, simply construct the record with the Any object:

KnowledgeRecord pointRecord = new KnowledgeRecord(point);

If you have a KnowledgeRecord, you can get its contents as an Any using the to_any() method:

Any p = pointRecord.to_any();

This will work on any KnowledgeRecord. If it contains an Any, you will get a copy of it. If it doesn't, you will get a copy of the stored data held within a new Any. To access that data, be sure to register the appropriate types using the Any.register* static methods.

You can store an Any into a KnowledgeBase using the set() method:

KnowledgeBase kb = new KnowledgeBase;
kb.set("pointKey", point);

To get an Any from the KnowledgeBase, use the get() method to retrieve a KnowledgeRecord as usual, then call to_any() on it.

Add a custom footer
Add a custom sidebar
Clone this wiki locally