Custom KnowledgeRecord Types

Madara's KnowledgeBase and KnowledgeRecord support several useful types natively: integers, doubles, vectors of both, strings, and blobs. Where appropriate, it is best to use those types. They will be accessible by any KnowledgeBase, and a variety of useful functions will work with them, including overloaded math operators and indexing operations. For types which cannot easily be represented by those native types, KnowledgeRecord, and thus KnowledgeBase, can store an Any type, provided by Madara, which can in turn store custom types directly.

Supported Types

To be supported by Any a type must be default constructible, copy constructible, and support serialization via the Cereal Library. This includes all STL container types and primitive types. Note for Shield AI developers: any type which supports the internal "BufferedStore" class will also support Any.

Storing Custom Types in KnowledgeBase

For many use cases, you need not use the Any type directly. To store a supported type, call the set_any or emplace_any methods. The set_any method takes a std::string key or VariableReference as first argument, and an object to store as second. This object will be stored by value, but will be copied or moved depending on whether it is given as an lvalue or rvalue reference. For example:

using namespace madara; using namespace knowledge;
KnowledgeBase kb;
using strvec = std::vector<std::string>;
kb.set_any("foo", strvec{"a", "b", "c", "d"}); // Constructs vector, and moves into "foo"
kb.emplace_any<strvec>("bar", mk_init({"e", "f", "g"}); // Use mk_init helper to pass initializer lists

Reading Custom Types from the KnowledgeBase

To get back the stored data, use get() as normal to get the KnowledgeRecord, which provides a to_any method. This method accepts a template type parameter, and returns a copy of the stored data, provided the type exactly matches what was stored originally. The type cannot simply be convertible from the stored type, or be base type of it, it must be the exact same type, or a BadAnyAccess exception will be thrown. Continuing the above example:

auto foo = kb.get("foo").to_any<strvec>();
assert(foo.size() == 4);
assert(foo[1] == "b");
kb.set_any("baz", 10); // Stores an int, since that's the default type of an integer literal
assert(kb.get("baz").to_any<int>() == 10);
kb.get("baz").to_any<short>(); // Throws BadAnyAccess; short does not exactly match int
kb.emplace_any<int>("baz2", 10); // This version explicitly names the type, to avoid confusion

KnowledgeBase also provides share_any and take_any methods to access the internal shared_ptr that KnowledgeRecord holds. For more details on these methods, see their documentation section.

In addition, KnowledgeRecord provides get_any_ref and get_any_cref to obtain a reference to the stored value directly. This is less safe than using share_any, but more efficient. You should ensure that the KnowledgeBase is either locked, or that you hold a copy of the KnowledgeRecord as long as you are using the reference, and only modify the object through reference while holding the KnowledgeBase.

Lazy Deserialization

When the KnowledgeBase serializes a KnowledgeRecord holding an Any, whether to disk or network, the held type is serialized using the Cereal library. When the KnowledgeBase receives or loads that KnowledgeRecord, it does not know what type is stored within it (and the current process might not even have the type available). The raw serialized data will be stored within the Any for lazy deserialization, upon the first attempt to access its contents. Currently, this operation is not fully checked. While access via the wrong type should be memory safe, it may result in garbled data in the deserialized object, rather than a thrown exception. This downside will be addressed in the future.

Creating a Custom Type

A type supported by Any must be default constructible, copy constructible, and support serialization via the Cereal Library (a dependency used by Madara). To satisfy the last requirement, you can either provide the functions expected by Cereal, or define a free function forEachField in the same namespace as your type. The function should have the signature, replacing YourType with your type:

template<typename Fun, typename T>
auto forEachField(Fun &&fun, T &&val) -> madara::enable_if_same_decayed<T, YourType>;

This function will be called with a functor (fun) and the a reference (const or not) value of your type. Your forEachField should call fun for each field in your type which you want serialized with a const char * argument holding the field's name (used for JSON serialization) and a reference to the field within the given val. For example:

struct Example
{
  int a;
  double b;
  std::string c;
};
template<typename Fun, typename T>
auto forEachField(Fun &&fun, T &&val) -> madara::enable_if_same_decayed<T, Example>
{
  fun("a", val.a);
  fun("b", val.b);
  fun("c", val.c);
}