Introduction
Permazen makes persistence simpler and more natural for Java programmers.
Permazen's goal is to make Java persistence as simple as possible, doing so in a Java-centric manner, while remaining strictly type safe.
Permazen does this without sacrificing flexibility or scalability. In fact, Permazen provides a more powerful persistence solution than traditional ORM solutions like Hibernate and JPA.
Ironically, it does this by relegating the database to the simplest role possible - storing data as key/value pairs - and reimplementing the other supporting features, such as indexes, queries, schema management, command line interface, etc., in a simpler Java-centric manner.
In other words, Permazen is not trying to re-invent existing algorithms. Rather, it reimplements many of the same, well-tested ideas and algorithms used in traditional databases, but in a way that is much simpler and more natural for Java programmers to access. So simplicity and maintainability of your code are primary benefits, but as it happens, this approach also enables new solutions to several problems that traditional databases don't solve for you.
The paper Permazen: Language-Driven Persistence for Java describes the issues that are inherent to persistence programming and how Permazen addresses them.
Permazen is a Java Persistence Layer: it provides a Java-centric view of a transactional key/value database. It is also as a general purpose serialization framework with built-in schema management, and an object indexing, reference tracking, and field change notification system.
Actually, Permazen is broken into three layers, each dependent on the one below it; you can use any or all of them:
- Java Layer: Let's you interact with persistent data naturally using pure Java concepts: type-safety, collections and streams, annotations, JSR 303 validation, etc.
- Core API Layer: Implements the actual encoding of data, indexing, and schema definition/verification
- Key/Value Layer: A very simple API exposing a transactional, sorted binary key/value datastore
Because of the simplicity of the key/value layer API, Permazen can be easily ported to any transactional, sorted key/value store. Permazen includes key/value adapters for several popular databases, such as Google Spanner, RocksDB, LevelDB, Oracle's Berkeley DB, and any SQL database.
Permazen also includes RaftKVDatabase, an ACID-compliant distributed key/value database based on the Raft consensus algorithm, as well as in-memory, read optimized and XML flat file key/value stores for testing and prototyping.
Permazen tries to minimize the distinction between "normal Java" and "persistence Java".
For example, Permazen does not have or need a "query language". All data access is done through normal Java code, using objects and method calls.
Your "database schema" is defined by your Java classes and their fields. These implicitly define your one-to-one, many-to-one, and many-to-many relationships. Permazen provides tightly controlled and well-defined schema versioning support so that you can safely refactor your code freely.
Permazen also avoids design choices that would limit scalability. Some Permazen design choices that support scalability are:
- Permazen runs on top of any database that can look like a transactional key/value store
- Permazen avoids the use of "bottleneck" keys (e.g., auto-increment counters) that can cause high contention between transactions
- Permazen provides support not only for indexing fields, but also tools to support building your own custom indexes, so that virtually anything you frequently query can be indexed
- Permazen supports incremental schema migration for rolling, no-downtime software updates that involve running different versions of your code (with different schemas) on different nodes at the same time
- Permazen does not include or require any operations that affect the entire database
Most importantly, Permazen does all of this while remaining completely type safe.
Permazen itself is configured using just a few Java annotations. For example:
@PermazenType
public interface Person extends PermazenObject {
// My age
int getAge();
void setAge(int age);
// My friends
Set<Person> getFriends();
}Of note:
- We are defining a database object type
Personthat has two fields:- An
intfield namedageof typeint - A
Setfield namedfriendswith element typePerson
- An
- There is no setter method for the
friendsset- Collection fields are initially empty but always exist;
getFriends()will never returnnull
- Collection fields are initially empty but always exist;
- The model class can be a normal POJO, or an
abstractclass or aninterface- Permazen will generate a concrete subclass that overrides/implements the declared bean property methods at runtime
-
Personis also declared to implement thePermazenObjectinterface- This makes life easier, but there is no requirement that the model class extend or implement anything
- Permazen ensures generated subclasses always implement
PermazenObject
Sometimes a source file is worth a thousand words, so...
public interface HasName {
@PermazenField(indexed = true) // this field is indexed
String getName();
void setName(String name);
/**
* Find all objects of the specified type having the specified name.
*
* @param type type to search for
* @param name name to search for
* @return all objects of type {@code type} having that name
*/
public static <T extends HasName> NavigableSet<T> findByName(Class<T> type, String name) {
// This is how you view an indexed field
final NavigableMap<String, NavigableSet<T>> idx = PermazenTransaction.getCurrent()
.queryIndex(type, "name", String.class).asMap();
// Query the index for objects with the given name, if any
final NavigableSet<T> objs = idx.get(name);
// Return an empty set if none exist
return objs != null ? objs : Collections.emptyNavigableSet();
}
}
@PermazenType
public abstract class Account implements HasName {
public abstract float getBalance();
public abstract setBalance(float balance);
public boolean isSuspended() {
return this.getBalance() <= 0.0f;
}
/**
* Get all users associated with this account.
*/
public NavigableSet<User> getUsers() {
final NavigableSet<User> users = this.getTransaction().queryIndex(
User.class, "account", Account.class).asMap().get(this);
return users != null ? users : Collections.emptyNavigableSet();
}
/**
* Get the total balance of all accounts.
*/
public static double getTotalBalance() {
return this.getTransaction().getAll().stream()
.mapToDouble(Account::getBalance).sum().orElse(0);
}
/**
* Find account(s) by name.
*/
public NavigableSet<Account> findByName(String accountName) {
return HasName.findByName(Account.class, accountName);
}
}
@PermazenType
public abstract class User implements HasName {
@NotNull
@Pattern("[a-z][a-z0-9]+")
@PermazenField(indexed = true, unique = true)
public abstract String getUsername();
public abstract void setUsername(String username);
@NotNull
@PermazenField(onDelete = DeleteAction.DELETE) // if account is deleted, delete users too
public abstract Account getAccount();
public abstract void setAccount(Account account);
/**
* Find user by username (exact match).
*
* @param username user's username
* @return corresponding user, or null if not found
*/
public static User getByUsername(String username) {
try {
return PermazenTransaction.getCurrent().queryIndex(User.class,
"username", String.class).asMap().get(username).first();
} catch (NoSuchElementException e) {
return null; // not found
}
}
/**
* Find users by username prefix and return in descending order by username.
*
* @param prefix username prefix
* @return all users whose usernames start with {@code prefix}
* sorted in reverse order by username
*/
public static Stream<User> getByUsernamePrefix(String prefix) {
return PermazenTransaction.getCurrent().queryIndex(User.class, "username",
String.class).asMap().subMap(prefix, true, prefix + '\u0000', false)
.descendingMap().values().stream().flatMap(NavigableSet::stream);
}
/**
* Find some user matching the specified predicate (brute force search).
*
* @param predicate matching predicate
* @return some user matching {@code predicate}, or null if none match
*/
public static User findUser(Predicate<? super User> predicate) {
return PermazenTransaction.getCurrent().getAll(User.class)
.stream().filter(predicate).findAny().orElse(null);
}
/**
* Create new user.
*
* @param username new user's username
* @param name new user's name
* @return the new user
*/
@Transactional
public static User createUser(Account account, String username, String name) {
final User user = PermazenTransaction.getCurrent().create(User.class);
user.setAccount(account);
user.setUsername(username);
user.setName(name);
return user;
}
}Permazen supports three kinds of fields: simple atomic fields, complex collection fields, and lock-free counter fields.
A simple field is any field that can be considered as a single atomic value and encoded into binary form. This inclues primitive types and primitive wrapper types, String, other usual suspects such as Date, UUID, etc., and any Enum type.
All simple fields have a well-defined sort order, which is also reflected in their binary encodings in the key/value store (as unsigned byte[] arrays).
Simple fields with array type up to 255 dimensions are also supported. Array types are read and written atomically by value. Array elements can have any simple type.
Simple fields also include reference fields, which are fields that refer to other database model objects.
You can also define your own simple type by providing a class annotated with @JFieldType that implements the FieldType interface.
The supported complex field types are Set (actually NavigableSet), List, and Map (actually NavigableMap). Permazen includes explicit support for these types, including indexing, change notification, etc.
The complex field types support any simple field type for their sub-field(s), i.e., the Set and List element, and Map key and value.
In the case of complex fields with primitive sub-field type, both true primitive and primitive wrapper types are supported. In the former case, null values are not allowed:
@PermazenType
public abstract class Person implements PermazenObject {
public abstract Set<Integer> wrapperSet(); // element type is java.lang.Integer
@PermazenSetField(element = @PermazenField(type = "int"))
public abstract Set<Integer> primitiveSet(); // element type is int
public void test() {
this.wrapperSet().add(null); // OK
this.primitiveSet().add(null); // IllegalArgumentException!
}
}Counter fields are a special type optimized for lock-free addition/subtraction, allowing a high degree of concurrency. Counter fields contain a 64-bit counter value.
Once you have defined your classes, programming with Permazen is all normal Java programming. There is no "query language".
A PermazenTransaction represents a Permazen transaction. In typical usage, a PermazenTransaction is associated with the current thread and made available via the static method PermazenTransaction.getCurrent(). For example, when using Spring Framework and Permazen's PermazenTransactionManager implementation.
You can also manage transactions yourself, creating them via Permazen.createTransaction(), associating them with the current thread via PermazenTransaction.setCurrent(), and always closing them via PermazenTransaction.commit() or PermazenTransaction.rollback().
All state associated with Permazen objects lives in the PermazenTransaction; the model objects themselves are essentially stateless. When you access a field of a model object, you are implicitly pulling that field's value from the PermazenTransaction associated with the object. Each transaction has its own pool of representative Java objects.
If you try to access a Person object in a transaction after that Person has been deleted, you get a DeletedObjectException. You can check for this condition via PermazenObject.exists(), and (if need be) recreate the object (with all fields reset) via PermazenObject.recreate().
There are three ways to access objects when starting from nothing: you can get the set of all objects of some type, you can get a specific object, or you can query an index.
To get all Persons:
// Get all people
public static Set<Person> getAll() {
return PermazenTransaction.getCurrent().getAll(Person.class);
}The parameter to PermazenTransaction.getAll() can be any Java type, including interface types, e.g.:
// Get all objects that implement the HasName interface
public static Set<HasName> getAllNamed() {
return PermazenTransaction.getCurrent().getAll(HasName.class);
}All objects have an internal 64-bit object ID which is represented by the ObjId class. Typically you would not access objects by object ID but you can if, for example, you want to get the representative object from a new or different transaction:
public static Person find(ObjId id) {
return PermazenTransaction.getCurrent().getJObject(id, Person.class);
}The getJObject() method simply returns a Person object with the specified object ID without checking whether the Person actually exists in the transaction; use exists() for that.
To query by index, use PermazenTransaction.queryIndex(), specifying the type of value you're looking for (this may be narrower that what is actually indexed), and the name and type of the indexed field:
@PermazenField(indexed = true)
public abstract String getLastName();
public abstract void setLastName(String lastName);
public static NavigableSet<Person> findByLastName(String lastName) {
final NavigableSet<Person> peopleWithName
= PermazenTransaction.queryIndex(Person.class, "lastName", String.class).asMap().get(lastName);
return peopleWithName != null ? peopleWithName : Collections.emptyNavigableSet();
}For primitive field types, use the wrapper type:
@PermazenField(indexed = true)
public abstract int getAge();
public abstract void setAge(int age);
public static Stream<Person> findOlderThan(int age) {
return PermazenTransaction.queryIndex(Person.class, "age", Integer.class).asMap()
.tailMap(age, false).values().stream().flatMap(NavigableSet::stream);
}For composite indexes, PermazenTransaction.queryCompositeIndex() performs the lookup.
To create a Person, invoke create() on a PermazenTransaction object, e.g.:
// Create a Person
public static Person create() {
return PermazenTransaction.getCurrent().create(Person.class);
}To delete a Person:
person.delete();The exists(), delete(), and recreate() methods are part of the PermazenObject interface, which all generated model subclasses implement. Your classes don't have to to be declared to implement PermazenObject, but life is a easier if they are.
In regular Java, you can't explicitly delete objects, you can only unreference them. Since they are unreferenced, there is by definition no issue with other objects still referring to them.
However, all persistence layers that support references between "objects" must decide with what to do when deleting an object which is either referenced by, or itself references, other objects that still exist.
First we consider the situation where an object is deleted, but that object still has one or more other objects referencing it. In SQL we have ON DELETE ... for this situation.
In Permazen, every reference field is configured with a DeleteAction to be taken when the referred-to object is deleted. There are four choices:
-
EXCEPTION- Disallow deletion of such an object; instead, throwReferencedObjectException. This is the default value. -
UNREFERENCE- Set the reference to null (in the case of simple reference fields) or remove the corresponding collection element (in the case of sets, lists, and maps). -
DELETE- Delete the referring object, repeating recursively as necessary. Cycles in the reference graph are handled correctly. -
NOTHING- Do nothing, leaving a dangling reference; attempts to dereference the deleted object will throw aDeletedObjectException.
Note the subtle difference between SQL's ON DELETE SET NULL and DeleteAction.UNREFERENCE. For simple fields they work the same, but for collection fields, the reference is removed from the collection rather than being set to null.
To be be guaranteed to never see any DeletedObjectExceptions, you can do the following:
- Avoid configuring reference fields with
DeleteAction.NOTHING - Always check
exists()before using Java model objects acquired directly by object ID
The second requirement is inherent in the nature of persistence programming, because there is always the possibility of an object disappearing between transactions due to some other thread deleting it.
Permazen also allows object deletion to cascade to other objects referenced by the deleted object. If so configured, these other objects will also be deleted, and the cascade will proceed recursively if needed (cycles in the graph of references are handled correctly).
See @PermazenField for details on configuring the delete behavior for reference fields.
Given the above Person class with no indexes configured, if you wanted to calculate whether any Person having a specific age existed, you would have to write this code:
public boolean existsPersonOfAge(int age) {
for (Person person : Person.getAll()) {
if (person.getAge() == age)
return true;
}
return false;
}Having to write that code is a good thing: Permazen is exposing the underlying performance reality and forcing the developer to confront the fact that the cost of calculating this function is an iteration over every Person in the database. With other persistence layers such as JPA, this cost would be hidden - the number of database rows visited is decided elsewhere and completely non-obvious from looking at Java code (or generated SQL).
With a large database, such a whole database iteration could be impractically slow. At least with Permazen, it will be slow for an obvious reason, not mysteriously so. An explicit goal of Permazen is that it not be possible to have queries be horribly slow unless you write them that way yourself!
Of course this is the perfect situation for an index on the age field. Here we create one by adding indexed = true to the @PermazenField annotation:
// My age - now indexed!
@PermazenField(indexed = true)
public abstract int getAge();
public abstract void setAge(int age);
public boolean existsPersonOfAge(int age) {
return PermazenTransaction.getCurrent().queryIndex(
Person.class, "age", Integer.class).asMap().containsKey(age);
}Now it's obvious by inspection that this query will run efficiently.
A database index on a field is just a sorted, seekable mapping from field value to the set of all objects having that value in the field.
Permazen simple indexes implement the Index interface, which allows viewing the index as either a map from value to the set of objects with that value in the field:
public static NavigableMap<Integer, NavigableSet<Person>> queryPersonAges() {
return PermazenTransaction.getCurrent().queryIndex(Person.class,
"age", Integer.class).asMap();
}...or as a set of (value, object) pairs:
public static NavigableSet<Tuple2<Integer, Person>> queryPersonAges() {
return PermazenTransaction.getCurrent().queryIndex(Person.class,
"age", Integer.class).asSet();
}The Person.class and Integer.class parameters accomplish two goals: first, they provide compile-time type safety (Permazen also verifies the types at runtime). Secondly, they allow you to narrow or widen the type to suit your needs.
For example, if HappyPerson extends Person and you only want to find happy people of a certain age, you can do this:
public static NavigableMap<Integer, NavigableSet<HappyPerson>> queryHappyPersonAges() {
return PermazenTransaction.getCurrent().queryIndex(HappyPerson.class,
"age", Integer.class).asMap();
}Or, suppose your schema has evolved over time and the old superclass Mammal which Person used to extend no longer exists, but you still have some leftover non-human Mammal's in your database and you want to include them in the index query. Then you can do this:
public static NavigableMap<Integer, NavigableSet<PermazenObject>> queryObjectAges() {
return PermazenTransaction.getCurrent().queryIndex(PermazenObject.class,
"age", Integer.class).asMap();
}Since your old Mammal class is long gone, the old Mammal objects will appear as UntypedPermazenObject's. You can still access their fields by directly invoking the corresponding PermazenTransaction introspection methods.
In all cases, Permazen guarantees type safety, even in the face of arbitrary refactoring of your code over time.
Since index maps are NavigableMaps, you can efficiently find the minimum or maximum value, create a restricted range of values, iterate values in forward or reverse order, etc.
public static int getMaximumAge() {
NavigableMap<Integer, NavigableSet<Person>> index = Person.queryPersonAges();
return !index.isEmpty() ? index.lastKey() : -1;
}Since we have a NavigableSet<Person>, you may be wondering what sort order applies to Person. Permazen sorts database objects by their unique object ID (the object ID is available via PermazenObject.getObjId()). Object IDs are opaque 64-bit values. Newly created objects get a randomly generated value which avoids distributed database contention that would otherwise occur with an auto-increment counter.
Object ID's include an encoding of the object's type (i.e., Java model class) as a prefix. Therefore, objects of the same type sort together.
One final note: reference fields are always indexed; you don't need to specify this explicitly.
You can also index complex fields, by indexing the element sub-field of a Set or List, or the key or value sub-field of a Map. For example:
@PermazenType
public abstract class Student implements PermazenObject {
// Database Fields
// The classes this student is attending
@PermazenSetField(element = @PermazenField(indexed = true))
public abstract Set<LectureClass> getLectureClasses();
// This student's ranking of his/her teachers
@PermazenListField(element = @PermazenField(indexed = true))
public abstract List<Teacher> getTeacherRankings();
// This student's test scores
@PermazenMapField(key = @PermazenField(indexed = true), value = @PermazenField(indexed = true))
public abstract Map<Test, Float> getTestScores();
// Index Query Methods
// Map classes to students in the class
public static NavigableMap<LectureClass, NavigableSet<Student>> queryStudentsByLectureClass() {
return PermazenTransaction.getCurrent().queryIndex(
Student.class, "lectureClasses.element", LectureClass.class).asMap();
}
// Map teacher to students that rank the teacher
public static NavigableMap<Teacher, Student> queryStudentsRankings() {
return PermazenTransaction.getCurrent().queryIndex(
Student.class, "teacherRankings.element", Teacher.class).asMap();
}
// Map tests to students who have taken the test
public static NavigableMap<Test, NavigableSet<Student>> queryStudentsByTest() {
return PermazenTransaction.getCurrent().queryIndex(
Student.class, "testScores.key", Test.class).asMap();
}
// Map tests scores to the students who got that score on some test
public static NavigableMap<Float, NavigableSet<Student>> queryStudentsByTestScore() {
return PermazenTransaction.getCurrent().queryIndex(
Student.class, "testScores.value", Float.class).asMap();
}
}In Permazen, reference fields are always indexed (analogous to SQL, where an index is required for foreign key constraints). So in the above example, @PermazenField annotations are actually unnecessary except for testScores.value.
Indexes are a perfect way to provide Java access to both sides of a one-to-many, many-to-one, or many-to-many relationship without there having to actually be any redundant information: you just query the index for the "inverse" side of the relationship:
@PermazenType
public abstract class User implements PermazenObject {
// Get this user's account
public abstract Account getAccount();
public abstract void setAccount(Account account);
}
@PermazenType
public abstract class Account implements PermazenObject {
// Get all users with this account
public NavigableSet<User> getUsers() {
return this.getTransaction().queryIndex(
User.class, "account", Account.class).asMap().get(this);
}
}Contrast with JPA, where it's possible for e.g., parent.getChildren() and child.getParent() to get out of sync.
In Permazen, indexes are always up-to-date, reflecting the current transaction state.
Permazen also supports composite indexes. A composite index is an index on more than one field. Composite indexes are mainly useful when you need to efficiently sort on multiple fields at once:
@PermazenCompositeIndex(name = "byName", fields = { "lastName", "firstName" })
@PermazenType
public abstract class Person implements PermazenObject {
public abstract String getLastName();
public abstract void setLastName(String lastName);
public abstract String getFirstName();
public abstract void setFirstName(String firstName);
// Get Person's sorted by last name, then first name
public static Index2<String, String, Person> queryByLastNameFirstName() {
return PermazenTransaction.getCurrent().queryCompositeIndex(Person.class, "byName", String.class, String.class);
}
}A composite index on two fields is returned as an Index2, a composite index on three fields is returned as an Index3, etc.
Any higher-order index can be viewed as a lower-ordered index on any prefix of its indexed fields, e.g.:
// Maps each last name to the set of all associated first names
public static NavigableMap<String, NavigableSet<String>> queryNamesLastFirst() {
return Person.queryByLastNameFirstName().asIndex().asMap();
}The List element and Map value complex sub-fields are special, because their values are not unique to the field. In other words, the same value can appear more than once in a List or as a Map value.
Therefore, indexes on these two complex sub-fields can also be viewed as a type of composite index, where the "extra" field is the distinguishing value, i.e., the list index or map key.
For example:
@PermazenType
public abstract class Student implements PermazenObject {
// This student's ranking of his/her teachers
public abstract List<Teacher> getTeacherRankings();
// This student's test scores
@PermazenMapField(value = @PermazenField(indexed = true))
public abstract Map<Test, Float> getTestScores();
// Map teacher to students that rank the teacher AND the coresponding rank(s)
public static Index2<Teacher, Student, Integer> queryStudentsRankings() {
return PermazenTransaction.getCurrent().queryListElementIndex( // instead of querySimpleField()
Student.class, "teacherRankings.element", Teacher.class);
}
// Map tests scores to the students who got that score on some test AND the test(s)
public static Index2<Float, Student, Test> queryStudentsByTestScore() {
return PermazenTransaction.getCurrent().queryMapValueIndex( // instead of querySimpleField()
Student.class, "testScores.value", Float.class, Test.class);
}
}Back to our indexed age field, now suppose the question you need to frequently answer is not whether any person exists of a specific age, but whether a specifc Person has any friend of a specific age.
We just need to somehow intersect the set of that Person's friends with the set of people of a specified age. This is the equivalent of an SQL INNER JOIN. Permazen provides a clean way to do this.
First we need to make sure friends is declared as a NavigableSet (which is what it really is):
// My friends
public abstract NavigableSet<Person> getFriends();Then we just do the "join" via set intersection:
// Get all of my friends who are the specified age
public NavigableSet<Person> getFriendOfAge(int age) {
// Get all Person's of age 'age', whoever they are
final NavigableSet<Person> peopleOfAge = this.queryPersonAges().get(age);
if (peopleOfAge == null)
return NavigableSets.<Person>empty();
// Get all of my friends
final NavigableSet<Person> myFriends = this.getFriends();
// Return the intersection
return NavigableSets.intersection(peopleOfAge, myFriends);
}Reasoning about sets and operations like intersection, union, and difference is straightforward. The NavigableSets utility class provides efficient methods for set intersection, union, and difference. The sets must have comparable elements and a consistent sort order for those elements (this property is what allows the intersection operation to be efficient). Permazen provides this consistent ordering for you in all of its returned sets.
In Permazen reference fields, including sub-fields of a collection field, are always indexed. Therefore, unlike in normal Java programming where you can only follow a reference in the forward direction, we can efficiently go in either direction. Use normal Java dereferencing for the forward direction, and queryIndex() for the inverse direction:
// Who considers me a friend?
public Set<Person> whoConsidersMeAFriend() {
return this.getTransaction().queryIndex(
Person.class, "friends.element", Person.class).get(this);
}Permazen defines a reference path as a sequence of steps through reference fields from some set of starting objects to a set of target objects; see ReferencePath. Permazen also provides PermazenTransaction.followReferencePath() and PermazenTransaction.invertReferencePath() convenience methods:
// Who considers any of my friends a friend-of-a-friend?
public Set<Person> whoConsidersAnyOfMyFriendsAFriendOfAFriend() {
final PermazenTransaction jtx = this.getTransaction();
final ReferencePath path = jtx.getPermazen().parseReferencePath(
Person.class, "friend.element.friend.element", false);
return jtx.invertReferencePath(path, this.getFriends());
}When traversing reference paths, Permazen eliminates duplicates (caused by multiple routes to the same destination) as soon as possible. However, there's no magic surrounding how reference paths are followed. If you're not careful, you can create a combinatorial explosion when there is a high degree of fan-in.
Reference paths are the basis of the @OnChange annotation, described below.
Permazen transactions can be thought of as containers for the state of objects. As with any other transactional database, when the transaction closes that state is no longer available.
In the case of JPA, some state may be available after a transaction (detached objects), but exactly which state is often difficult to control or determine.
To allow you to keep a "snapshot" of some portion of the transaction state after that transaction has closed, and be able to specify exactly how much information to "snapshot", Permazen provides detached transactions. Detached transactions are independent, in-memory transactions that persist as long as you keep a reference to them.
Detached transactions are initially empty; you can copy objects between any two transactions (of either kind) using the PermazenObject methods copyIn() and copyOut(). These methods allow you to specify precisely how much information you want to copy, either as a cascade name, or by explicitly providing an Iterable containing the objects to copy. When copying data between transactions, the copies are performed efficiently at the key/value level, and cycles in the graph of object references are handled correctly.
Unlike in JPA, where detached objects are just plain objects, a detached transaction is a fully functional transaction and supports all Permazen features such as index queries, reference inversion, validation, change notification, etc. As a result, your code works the same either way.
Detached transactions make a perfect foundation for user interface presentation models. You can create multiple independent detached transactions if needed and keep them around as long as you like. The only thing you can't do with a detached transaction is commit() it.
As mentioned above, the PermazenObject.copyIn() and PermazenObject.copyOut() methods can be used to copy individual objects between transactions. However, you often need to copy multiple objects related to each other by references.
The easy way to do that is using reference cascades. In short, you can configure any reference to be part of one or more defined cascades. The methods PermazenObject.copyIn() and PermazenObject.copyOut() can then be used to copy not only the target object, but all objects reachable through the specified cascade. Cascades can flow through reference fields in either the forward or reverse direction, and reference cycles are handled properly.
cascades are described in detail in the @PermazenField Javadoc.
Detached transactions are also easy to serialize, send to a remote host, and deserialize - they are just a bunch of key/value pairs. When decoded, the recipient gets an object-level view of the data, can query indexes normally, etc. And if the sender and recipient are running different versions of the application, the normal Permazen schema update semantics automatically handle any required schema migrations. Use Permazen.createDetachedTransaction() to create a detached transaction around decoded key/value data.
The KeyListEncoder class has readPairs() and writePairs() methods for (de)serializing a key/value store. Also see the io.permazen.spring package for various Spring MessageConverter implementations.
Underneath the covers, every Permazen model class, field, and composite index has a unique storage ID. Internally, database identity and overall schema structure are defined by storage ID's, not names. The storage ID's, in encoded form, are used to prefix keys in the underlying key/value store.
In other words, storage IDs provide a level of indirection between your Java code and database identity, allowing your Java code to change more freely while also providing a flexible, controlled and well-defined way to update database contents (if necessary) when the schema does change. See Schema Management for more info on schema updates.
Unless you specify them explicitly, Permazen will auto-generate storage ID's for you by hashing the name of the class, field, or composite index. So you normally do not need to bother with storage ID's.
For classes and fields, the Permazen name defaults to the class's simple name or the field's Java bean property name. Therefore, by default, changing a class or getter method name will result in a schema change.
The way that storage ID's are encoded requires fewer bytes for smaller values. Values up to 250 only require one byte; values up to 506 only require two bytes, values up to 65,786 only require three bytes, etc. The default auto-generation of storage ID's results in values that encode in three bytes, i.e., in the range 507-65786. To save a few bytes, you can assign storage ID's explicitly if you want to:
@PermazenType(storageId = 100)
public abstract class Person implements PermazenObject {
// My age
@PermazenField(storageId = 101)
public abstract int getAge();
public abstract void setAge(int age);
// My friends
@PermazenSetField(storageId = 102, element = @PermazenField(storageId = 103))
public abstract Set<Person> getFriends();
}A database index is just a special case of derived information that is kept up to date automatically for you by the database.
The key benefits of using database indexes are:
- Much better performance - usually constant time queries of the indexed information
- The derived information is kept up-to-date automatically
- The database hides all the implementation details, keeping your application logic clean
With Permazen you can easily build your own arbitrary indexes, based on any derived information, that also satisfy these criteria.
Traditionally, in general this is hard part because it means tracking down all the places in your code that could possibly modify the information from which your index is derived and adding hooks in all those places. Permazen makes solving this problem much easier.
With traditional databases, sometimes you can make it work when the information is all contained in one object: then you can add intercept/update code into the setter methods of all the fields that affect the derived information. This extra code effectively serves as a trap for change notification, where it then updates the derived information.
Or if you're lucky, your derived information matches an SQL built-in aggregate function like MAX() for which SQL databases often have built-in functions. Often these built-in functions rely on "secret" internal indexes that make the function fast. But even if so, that hack only works when your query runs MAX() over the entire table.
Thinking more generally, what if your "index" is derived for many objects, possibly ones that are far away from each other? Suppose for example you need to be able to efficiently calculate the average age of all the friends of any Person. There's no way in SQL to tell the database to keep an index of the average age of each Person's friends, and an SQL AVG() query will need to iterate through all the friends to calculate the average. With today's huge datasets, iterating through every element of a collection may not be an option. So you have to construct and maintain this index yourself.
This is an important subtlety that affects scaling applications written using SQL databases: if what you want to index can't be indexed by a capability built-in to the database, but your dataset is too large to not index the information you need, then suddenly you have "index" logic spamming your Java codebase.
In our present example, you would have to track down any code that either (a) alters the age of a Person (and then figure out who that Person is a friend of), or (b) changes any Person's set of friends. But clearly the logic for maintaining a database index, which is nothing but derived information, belongs at the data layer, not the service or business layer.
With Permazen, you can index virtually anything, based on any reachable information, using the @OnChange annotation. The @OnChange annotation allows you to monitor arbitrarily distant changes by specifying the reference path to the change you want to monitor.
To maintain an index in Permazen:
- Detect relevant changes via
@OnChange-annotated methods - Update the derived information when a change occurs
Most importantly for maintainability, the code that implements #1 and #2 above can all live in one Java class.
In the example below, we monitor for changes in our set of friends and their ages, and update the average age automatically. This works no matter how the age is modified or where else those changes might be made in your codebase. We keep all the code to maintain our custom "friends' average age" index in one place where it belongs - in the data object.
// Public methods
// Get my friends' average age - value is always up to date!
public double getFriendsAverageAge() {
return (double)this.getFriendsAgeSum() / this.getNumFriends().get();
}
// Internal methods
// Keep track of how many friends I have (without locking!)
protected abstract Counter getNumFriends();
// Keep track of the sum of my friends' ages (could also use a Counter here...)
protected abstract long getFriendsAgeSum();
protected abstract void setFriendsAgeSum(long ageSum);
// Notify me when any of my friend's ages changes
@OnChange("friends.element.age")
private boolean onFriendAgeChange(SimpleFieldChange<Person, Integer> change) {
this.setFriendsAgeSum(this.getFriendsAgeSum() - change.getOldValue() + change.getNewValue());
}
// Notify me when a friend is added
@OnChange("friends")
private boolean onFriendsAdd(SetFieldAdd<Person, Person> change) {
this.setFriendsAgeSum(this.getFriendsAgeSum() + change.getElement().getAge());
this.getNumFriends().adjust(1);
}
// Notify me when a friend is removed
@OnChange("friends")
private boolean onFriendsRemove(SetFieldRemove<Person, Person> change) {
this.setFriendsAgeSum(this.getFriendsAgeSum() - change.getElement().getAge());
this.getNumFriends().adjust(-1);
}
// Notify me when friends is cleared
@OnChange("friends")
private boolean onFriendsClear(SetFieldClear<Person> change) {
this.getNumFriends().set(0);
this.setFriendsAgeSum(0);
}This index is maintained as efficiently as possible: we are notified only when a meaningful change occurs, and the update is incremental (constant time) and immediate.
The average age is a simple example, but the indexed information can be arbitrary, e.g., the sum of your friends ages modulo 23, or whatever.
By the way, Permazen does not notify for "changes" that don't actually change anything, such as person.setAge(21) when the age equals 21, or person.getFriends().add(friend) when friend is already in the set. Also, each object is only notified once about any particular change, even if the change is visible through multiple different routes through the reference path (this would necessarily involve a collection field somewhere on the path).
Of course, @OnChange notifications are handy for lots of other purposes besides custom indexes. For example, you might want to monitor some condition involving several related objects and generate an alert, etc. @OnChange notifications are also handy for validation scenarios involving distant dependencies (see Validation).
Notice in the example we are tracking the size of the friends set manually. Why not just use this.getFriends().size()?
In Permazen invoking size() on a Set or Map requires an O(n) time interation through the collection, which for very large collections can be a slow operation (invoking size() on a list is constant time, because list elements are explicitly indexed; lists have performance characteristics similar to ArrayList).
This may seem dumb - why not just keep track of the size? There is an important reason: allowing for reduced contention in distributed databases.
Imagine a database with many distributed nodes on the network, and on each node an application is rapidly adding new objects to some Set. If Permazen maintained a hidden size field with each collection, then that field would have to be updated with each insert, and therefore would be highly contended (i.e., causing a bottleneck), causing the database to be slow. Without it, however, the set elements can be added concurrently without conflict.
Permazen leaves it up to you whether the size of a collection should be explicitly maintained. After all, the size of a collection is just another type of index (derived information).
To support such usage, Permazen provides the Counter field type, which holds a long value and can be mutated via addition/subtraction without actually reading the value; on many key/value stores, this operation can be performed entirely without locking.
Permazen provides notifications whenever a database object is created or deleted via the @OnCreate and @OnDelete annotations:
@OnCreate
private void gotCreated() {
System.out.println("Hello world");
}
@OnDelete
private void gotDeleted() {
System.out.println("Goodbye world");
}Note that @OnCreate is not the same event as object construction. For example, a deleted object can be recreate()d, in which case @OnCreate will be invoked again but no Java construtor will be invoked because the representative Java object already exists.
Traditional SQL databases provide no explicit support for managing schema changes. This used to be OK when you had a single application server running on a single database, and it was small enough that running a few ALTER TABLE statements after a restart was relatively quick.
Today, however, not only are the data sets too large for schema changes that lock an entire table, it's often not acceptable to bring down all of the application servers at the same time, even if you could upgrade them all quickly. Instead, a rolling upgrade is required, where during the upgrade period different application servers will be running different versions of the software. Therefore, at some point you are going to have two different versions of your application running on two different nodes, both reading from and writing to the same database, but expecting to see and use different schemas. This situation presents obvious challenges and few databases provide any real solution.
Some NoSQL databases duck this question by being "schemaless", which really means "the schema is your problem, not mine". Other databases support the notion of an object version, but leave the rest to the application developer.
An important goal of Permazen is to provide a first class solution to this problem by providing explicit support to the application for schema maintenance and migration. In addition, the code for handling schema migration should live only in the data layer and not pollute the rest of the application. And finally, as always Java type safety should never be violated.
This support has several aspects.
Permazen databases have an explicit notion of a schema as well as schema version numbers, and you must explicitly tell Permazen what schema version number you are using before you can do anything with a database. If the schema you are intending to use conflicts with what's recorded in the database, you will get an error. Permazen simply won't let you inadvertently read or write data in an incompatible way.
A schema is defined as a set of object types, the fields in those object types, and any associated composite indexes. Each object type, field, and composite index is identified by its unique storage ID. Therefore, changes only to the names of things do not require a schema change.
Internally, schemas are represented by SchemaModel objects. This object type has an XML representation which is used to serialize the schema into the database. Normally, you don't need to mess with SchemaModels - they are generated for you from your annotated classes and handled automatically. But methods exist to allow you to access and introspect the current and all previously recorded database schemas if necessary.
Permazen allows arbitrary changes between schema versions, with one exception: a field cannot change types between two schema versions and also be indexed in both of those schema versions.
Permazen allows field storage ID's to be reused in different object types. This facilitates moving fields around in the Java type hierarchy. For example, if a schema change refactors the Vehicle class, replacing it with Car and Truck classes, then an existing licensePlate field could be copied into both Car and Truck without change -- or simply put into a common Vehicle abstract superclass. Then, for example, an index on the licensePlate field would contain both Cars and Trucks.
Within each Permazen database is recorded the version and associated SchemaModel of every schema version used by any object in the database. When a Permazen database is initialized, a schema and schema vesion number is generated from your annotated classes, and the first thing that occurs in each new transaction is a (quick) verification that the generated schema matches what's recorded under that version number in the database.
If there is a mismatch, an InvalidSchemaException exception is thrown. If the database has no record of that schema version, and you have configured Permazen to allow recording new schemas, it will write the new schema into the database and proceed normally. If Permazen is configured to not allow recording new schemas, an exception is thrown (more on when to enable this setting below).
In addition to tracking the schemas, Permazen also records with each object the version of the schema according to which the object was written. This tells us what fields are part of that particular object.
So now imagine two different application servers talking to the same database, but using two different schema versions. If they are both creating objects, then the database might contain objects of the same object type but having different versions and therefore containing different fields.
Here is what happens when an object with version X is read by the application server using schema version Y:
- The object is automatically upgraded (or downgraded) from schema version X to schema version Y:
- Newly removed fields are removed (pending step #2)
- Newly added fields are added and initialized to their default values
- Any associated indexes are updated as necessary
- For simple fields that have changed type, the configured
@PermazenField.upgradeConversion()policies are applied. - Any
@OnVersionChange-annotated methods in the affected object are invoked, with some or all of these parameters:- The old schema version
- The new schema version
- Either a
Map<String, Object>containing all of the old field values indexed by name, or aMap<Integer, Object>containing all of the old field values indexed by storage ID (depending on the method's declared parameters)
So the @OnVersionChange notification allows the object to handle the upgrade in a controlled manner. For example, suppose your application is at release 1.0 and uses schema version #1 which has this field containing a Person's name in the form Last, First:
public abstract String getName();
public abstract void setName(String name);But in application release 2.0 you decided you needed separate last and first names, so you create schema version #2, replacing the name field with lastName and firstName fields. Then your release 2.0 code, with a new method to handle the schema change, might look like this:
public abstract String getLastName();
public abstract void setLastName(String lastName);
public abstract String getFirstName();
public abstract void setFirstName(String firstName);
@OnVersionChange(oldVersion = 1, newVersion = 2)
private void splitNameField(Map<String, Object> oldValues) {
final String name = (String)oldValues.get("name");
final int comma = name.indexOf(',');
this.setLastName(name.substring(0, comma).trim());
this.setFirstName(name.substring(comma + 1).trim());
}The splitNameField() method will be invoked when the Person object is upgraded, which happens just prior to the first field access. It ensures that the field is forward-migrated properly, and none of your other release 2.0 code will ever need to know anything about prior schema versions.
In practice, it's usually simplest to completely ignore schema version numbers - by default, Permazen will auto-generate them for you by hashing the schema - and in your schema migration logic, just use the presence or absence of fields in oldValues to determine what migration work needs to be done:
@OnVersionChange
private void applySchemaChanges(Map<String, Object> oldValues) {
if (!oldValues.containsKey("balance")) // at some point we added a new field "balance"
this.setBalance(DEFAULT_BALANCE);
if (oldValues.containsKey("fullName")) { // we replaced "fullName" with "lastName" & "firstName"
final String fullName = (String)oldValues.get("fullName");
if (fullName != null) {
final int comma = fullName.indexOf(',');
this.setLastName(comma == -1 ? null : fullName.substring(0, comma));
this.setFirstName(fullName.substring(comma + 1).trim());
}
}
// ...etc
}Object versions are themselves indexed; you can access this index via PermazenTransaction.queryVersion(). This makes it easy to implement a one time process that proactively visits every object that needs to be upgraded.
The astue reader may notice that in a "no downtime" world, even adding the above @OnVersionChange handler is insufficient. That's because backward migration may also occur, for example, when you have upgraded some (but not all) of your servers to release 2.0, and a yet-to-be-upgraded release 1.0 server needs to read an object written by an already-upgraded release 2.0 server. If the forward migration destroys data, then version 1.0 servers can have a problem.
The combination of multiple nodes, no application downtime, and incremental upgrades involving incompatible schemas makes things tricky indeed.
The answer is a multi-step migration process. First, create an intermediate software release 1.5 containing the downward migration @OnVersionChange handler:
public abstract String getName();
public abstract void setName(String name);
@OnVersionChange(oldVersion = 2, newVersion = 1)
private void joinNameField(Map<String, Object> oldValues) {
this.setName(oldValues.get("lastName") + ", " + oldValues.get("firstName"));
}This new intermediate release 1.5 of your software still uses the original schema version #1, but it is now prepared to handle a schema version #2 object if it encounters one. Again, this handling is contained entirely in the data object.
Now your upgrade rolls out in two phases:
- Upgrade all machines from version 1.0 to 1.5
- Upgrade all machines from version 1.5 to 2.0
During phase 1, all objects stay at schema version #1. During phase 2, objects are upgraded and downgraded as necessary as different servers access them. Once phase 2 is complete, all future object accesses will use schema version #2, upgrading old version #1 objects on demand over time. Or, you can run a one-time scan that upgrades objects using PermazenObject.upgrade().
Voilà - an incremental, multi-node rolling application upgrade across releases using incompatible schema versions and with no downtime.
Note that release 2.0 of your software will need to configure Permazen to allow the addition of new schemas. Once phase 2 is complete, this setting can be turned back off.
The Permazen CLI utility allows you to inspect all schema versions recorded in a database. You can also remove any obsolete schemas when there are no more remaining objects with that version.
Permazen supports automatic incremental verification of JSR 303 validation constraints. By "incremental" we mean only those objects containing a field that has actually changed are (re)validated. Each time an object field is changed, the object is added to an internal validation queue (if not already on it). The validation queue is processed automatically on commit(), and if validation fails, a ValidationException is thrown.
What is described above is the same as supplied by e.g., JPA. However, Permazen also adds a few additional useful features.
- You can manually add any object to the validation queue by invoking
revalidate()on that object. - You can trigger processing of the validation queue at any time via
PermazenTransaction.validate(), or empty the queue viaPermazenTransaction.resetValidationQueue() - You can supply custom validation logic by annotating a method with
@Validate. All such methods will be invoked when the object is validated. - Combining
@Validatewith@OnChangeallows you to validate complex, multi-object constraints (see below)
Note that Permazen validates on a per-object basis. Therefore, the @Valid constraint, which causes validation to recurse through reference(s) to other objects, is redundant and rarely needed. Instead, ensure each individual object is added to the validation queue as necessary when changed.
When creating a new Permazen transaction, you specify the ValidationMode for the transaction. The default is AUTOMATIC which gives the behavior described above. The available options are:
-
DISABLED- no validation will be performed, even if you invokePermazenTransaction.validate()explicitly -
MANUAL- validation is only performed when you invokePermazenTransaction.validate()explicitly; changes to fields with JSR 303 annotations do not enqueue the object for validation -
AUTOMATIC- validation is performed when you invokePermazenTransaction.validate()explicitly, and automatically oncommit(); changes to fields with JSR 303 annotations automatically enqueue the object for validation
Although javax.validation.constraints provides many handy validation constraints, there are often cases that require validation of more complex constraints that depend on more than one field, or even multiple objects at the same time.
For example, suppose each Person may have both friends and enemies and you have a constraint that says nobody can be both a friend and an enemy. This kind of constraint is difficult to handle efficiently with traditional ORM solutions.
With Permazen, this is easy to do by following these steps:
- Add a
@Validate-annotated method that checks the constraint - Detect relevant changes via
@OnChangeand enqueue for validation
Here's an example:
@OnChange("friends")
private boolean onFriendsChange(SetFieldChange<Person> change) {
this.revalidate();
}
@OnChange("enemies")
private boolean onEnemiesChange(SetFieldChange<Person> change) {
this.revalidate();
}
@Validate
private void checkForFrenemies() {
if (!NavigableSets.intersection(this.getFriends(), this.getEnemies()).isEmpty())
throw new ValidationException(this, "we don't allow frenemies");
}An important point here is that the validation constraint is only checked when required, i.e., when there is a change to friends or enemies.
In addition to JSR 303 validation and the @Validate annnotation, Permazen also supports validation of uniqueness constraints on simple fields. These constraints verify that each object's value in some field is unique among all objects containing that field. Fields with uniqueness constraints must be indexed.
For example, this code would ensure all usernames were unique:
@PermazenType
public abstract class User implements PermazenObject {
@PermazenField(indexed = true, unique = true)
public abstract String getUsername();
public abstract void setUsername(String username);
}Often there is a special value or two that you want to exclude from a uniqueness constraint, for example null values, or a value of zero. Permazen gives you complete flexibility using the uniqueExclude() property.
For example:
@PermazenField(indexed = true, unique = true, uniqueExclude = { "NaN", "Infinity", "-Infinity" })
public abstract float getPriority();
public abstract void setPriority(float priority);Permazen includes support for use with the Spring Framework, including a PlatformTransactionManager that integrates with Spring's @Transactional annotation.
See the API Javadocs for more info.
Permazen includes a command line interface (CLI) program just like most databases do. However, Permazen's CLI can parse Java expressions, allowing you to work in your data's natural language.
The Permazen CLI has these features:
- Database maintenance commands
- Schema inspection and management
- XML import/export
- Include your own custom CLI commands, defined by annotated classes on the classpath
- Command line history searching and editing (provided via JLine)
- Pervasive tab-completion, supported by all internal parsers
- Java expression parser/evaluator
- Use regular Java expressions for database queries and changes
- Supports Java 8 lambdas and method references
- Built-in parse customizations, such as object ID literals
- Extended expression syntax with built-in functions such as
concat(), etc. - Include your own custom functions, defined by annotated classes on the classpath
Here's a few examples of using the CLI to query into the provided demo database (you can view the model classes here). Lines are wrapped for clarity.
First, we count how many planets there are and then show them sorted by mass:
permazen: automatically configuring demo database using the following flags:
--xml demo-database.xml --classpath demo-classes --model-pkg io.permazen.demo
Welcome to Permazen. You are in PERMAZEN mode. Type `help' for help.
Permazen> import io.permazen.demo.* # works the same as Java's import statement
Permazen> import java.util.*
Permazen> eval all(Planet.class).size() # count how many planets there are
9
Permazen> eval queryIndex( # show all planets sorted by mass
Planet.class, "mass", Float.class).asMap()
.values().stream().flatMap(Collection::stream)
.forEach(p -> print(String.format("Name: %-10s Mass: %.0f", p.name, p.mass)))
Name: Pluto Mass: 13099999664401168000000
Name: Mercury Mass: 330000001687677400000000
Name: Mars Mass: 642000016384680500000000
Name: Venus Mass: 4869999810916809000000000
Name: Earth Mass: 5969999912619192000000000
Name: Uranus Mass: 86800001317335690000000000
Name: Neptune Mass: 101999998530235880000000000
Name: Saturn Mass: 568000005560064000000000000
Name: Jupiter Mass: 1897999959991329600000000000
You can see the use of the built-in top-level functions all(), queryIndex(), and print(). You can add your own custom functions using @Function-annotated classes from the classpath.
Now let's set a few variables for some planets we are interested in:
Permazen> eval $earth = all(Planet.class).stream()
.filter(p -> p.name.equals("Earth")).findAny().get()
io.permazen.demo.Planet$$Permazen@364ee37d
Permazen> eval $earth.mass
5.97E24
Permazen> eval $jupiter = all(Planet.class).stream()
.filter(p -> p.name.equals("Jupiter")).findAny().get()
io.permazen.demo.Planet$$Permazen@4cd452db
Note Java bean properties can be accessed using dot-property notation. Let's show Jupiter's moons:
Permazen> eval $jupiter.satellites
[io.permazen.demo.Moon$$Permazen@3963a1b8, io.permazen.demo.Moon$$Permazen@3c0e5477, ...
Instead of their toString() values, let's show their names:
Permazen> import java.util.stream.*
Permazen> eval $jupiter.satellites.stream().map(Planet::getName).collect(Collectors.toList())
[Io, Europa, Ganymede, Callisto]
You can use the built-in command transform for the same effect:
Permazen> eval transform($jupiter.satellites, $s, $s.name)
[Io, Europa, Ganymede, Callisto]
The CLI extends regular Java operators with additional smarts. For example, the & operator, when applied to two Sets, will intersect them. For example, to find all moons of Jupiter above a certain mass:
Permazen> eval $jupiter_moons = $jupiter.satellites
.stream().map(Moon::getName).collect(Collectors.toSet())
[Io, Europa, Callisto, Ganymede]
Permazen> eval $heavy_objects = queryIndex(Object.class, "mass", Float.class)
.asMap().tailMap(1e23f).values()
.stream().flatMap(Collection::stream)
.map(Body::getName).collect(Collectors.toSet())
[Earth, Mars, Neptune, Jupiter, Saturn, Venus, Uranus, Callisto, Titan, Mercury, Sun, Ganymede]
Permazen> eval $jupiter_moons & $heavy_objects
[Callisto, Ganymede]
Objects can be referred to using object ID literals, which have the form @ followed by 16 hex digits:
Permazen> eval $jupiter.objId
fc21bf0000000005
Permazen> eval @fc21bf0000000005.name
Jupiter
Permazen also includes a Vaadin-based graphical user interface (GUI). It provides a way to view and edit objects in a Permazen database, as well as perform queries using the same Java syntax supported by the CLI.
The GUI is entirely auto-generated from your Java model classes.
Permazen is implemented in two distinct layers: the upper "Permazen" layer, which is the layer you normally work with, and the "core API" layer, which is independent of the Java language. Think of the Permazen layer as the "object" layer and the core API layer as the "data" layer. This separation is critical for maintaining rigorous data integrity in the database itself, while also allowing complete flexibility at the Java level.
This section gives an overview of the core API layer. Normally the core API can be ignored; instead, you should only ever need to think in Java. However, an understanding of the core API layer can elucidate how Permazen works. See the core API Javadoc for implementation details.
The core API layer has its own notion of schema, objects, data types, references, etc., which mostly models Java but has a few important differences. In particular, core API "objects" are really just "structures", and references and enum types are handled differently.
The core database layer has the following notions:
- Schema
- Has a version number (positive integer)
- Includes a collection of object types
- Object Type
- Identified by a unique storage ID
- Includes a collection of fields
- Objects
- Objects have an associated object type
- Objects have a schema version number
- Fields
- Is either simple, complex, or counter
- Has a unique storage ID in the object type
- Simple Fields
- Holds an atomic, sortable value like
intorDate - Primitive types are supported
- Non-primitive types may be null
- May optionally be indexed.
- Reference Fields
- A special type of simple field that holds a reference to an object
- Always indexed
- Complex Fields
- Sets, Lists, and Maps
- Complex fields have one or more sub-fields, which are always simple fields
- A complex field may not be itself indexed, but any of its simple sub-field(s) may be
- Sub-fields may have primitive type; if so, nulls are not allowed and
IllegalArgumentExceptionis thrown if you try to add one - Counter Fields
- Optimized for lock-free add/subtract updates
- Does not support indexing or
@OnChange - Composite Indexes
- An index on two or more simple fields
All objects and fields have a unique storage ID. However, the same field may be contained in multiple object types (this is how Java sub-types are handled).
Use of storage ID's can change arbitrarily across schema versions, subject to these restrictions:
- The storage ID of an object type cannot also be used for an indexed field.
- The same storage ID cannot be used for two indexed fields having different types.
Permazen validates all of this for you, and will throw an exception if you try to do anything invalid.