BestPractices
This documents some "best practices" and useful tricks when using Permazen.
Instead of having a DAO (data access objects) layer, just put your query methods directly into your data model classes.
For example,
@PermazenType
public abstract class User implements PermazenObject {
// Fields
// Get this user's username
@PermazenField(indexed = true, unique = true)
@NotNull
public abstract String getUsername();
public abstract void setUsername(String username);
// "DAO" methods
// Get all users
public static NavigableSet<User> getAll() {
return PermazenTransaction.getCurrent().getAll(User.class);
}
// Create new user
public static User create() {
return PermazenTransaction.getCurrent().create(User.class);
}
// Find user by username
public static User getByUsername(String username) {
final NavigableSet<User> users = PermazenTransaction.getCurrent().queryIndex(
String.class, "username", User.class).get(username);
return users != null ? users.first() : null;
}
}Often a singleton database instance is needed.
This is easy; here's an example:
@PermazenType
public abstract class Config {
/**
* Get the singleton instance.
*/
public static Config getInstance() {
final PermazenTransaction jtx = PermazenTransaction.getCurrent();
try {
return jtx.getAll(Config.class).first();
} catch (NoSuchElementException e) {
return jtx.create(Config.class);
}
}
...
}Applications often need access to database objects after a transaction closes. Unlike JPA, where you can keep using "detached" database objects after a transaction closes (sort-of), Permazen database objects become inaccessible once a transaction closes. Therefore, if you need to access object data after closing a transaction, you must explicitly "copy out" the database objects into a separate, in-memory transaction while the database transaction is still open.
Although this adds an extra step, it has two benefits: it gives you more precise control over exactly what data gets held in memory after a transaction closes, and because the in-memory transaction behaves just like any other transaction, you can query your "detached" data using index queries, query for objects by type, etc.
Copying out individual objects is easy using PermazenObject.copyOut():
// Note: the invoker of this method may be outside of any transaction
@Transactional
public User findUser(Predicate<? super User> predicate) {
final User user = PermazenTransaction.getCurrent().getAll(User.class)
.stream().filter(predicate).findAny().orElse(null);
return user != null ? user.copyOut() : null;
}It's often the case that a database object has some related objects that should go with it when doing such a copy. For example, you probably want to display each User's account name in some table of users in the GUI, etc.. So each User has its Account as a "related object", and you always want an object's "related objects" to also be copied out with it.
Permazen supports the notion of reference cascades, which are useful for accessing an entire graph of related objects at once, for example, using the methods PermazenObject.copyIn() and PermazenObject.copyOut(). These methods take a cascade name parameter which determines which reference fields to traverse during the copy.
Copy cascades are similar to JPA's cascades for persist, merge, etc. but are more flexible in several ways.
First, you can define as many different types of cascades as you want: in some situations you may want to cascade to one set of related objects, and in other situations you may want to cascade to another set of related objects. The cascade names are user-defined, and you can create as many as you like.
Secondly, copy cascades can flow independently in either/both of the forward and inverse directions, and may form cycles, which will be properly handled during the copy operation.
Finally, you can remap object ID's during the copy, effecting a "deep clone" operation.
Here's an example showing a User and Account classes:
public interface HasAccount {
@PermazenField(cascades = "load")
@NotNull
Account getAccount();
void setAccount(Account account);
}
public interface User extends HasAccount<User> {
String getUsername();
void setUsername(String username);
}Here's a more complex example:
public interface Node extends PermazenObject {
/**
* Get the parent of this node, or null if node is the root.
*/
@PermazenField(cascades = { "tree", "ancestors" }, inverseCascades = { "tree", "descendants" })
Node getParent();
void setParent(Node parent);
/**
* Get the children of this node.
*/
default NavigableSet<Node> getChildren() {
final NavigableSet<Node> children = this.getTransaction().queryIndex(
Node.class, "parent", Node.class).get(this);
return children != null ? children : NavigableSets.empty();
}
default Node copySubtreeTo(PermazenTransaction dest) {
return (Node)this.copyTo(dest, -1, new CopyState(), "descendants");
}
default Node copyWithAnscestorsTo(PermazenTransaction dest) {
return (Node)this.copyTo(dest, -1, new CopyState(), "ancestors");
}
default Node copyEntireTreeTo(PermazenTransaction dest) {
return (Node)this.copyTo(dest, -1, new CopyState(), "tree");
}
}A common operation in many applications is when you want to reacquaint yourself with a database object. That is, you have a detached object from a prior transaction, and you want to find the current version of that object in the current transaction. Adding a getDatabaseInstance() method makes this easy:
public interface HasDatabaseInstance<T extends HasDatabaseInstance<T>> extends PermazenObject {
/**
* Get the instance in the current transaction corresponding to
* this instance, wherever it came from.
*/
@SuppressWarnings("unchecked")
public T getDatabaseInstance() {
return (T)PermazenTransaction.getCurrent().getJObject(this);
}
}
public abstract class User extends HasDatabaseInstance<User> {
// Now User.getDatabaseInstance() returns the User in the current transaction
}Of course, instead of using interfaces like HasDatabaseInstance, you have the option of putting that functionality into a common superclass from which your database model classes extend.
Permazen key/value stores are permitted to throw RetryTransactionExceptions in cases of conflicts, optimistic lock failure, deadlock avoidance, etc.
In fact, any database that supports simultaneous transactions and guarantees a basic level of transaction consistency will have such an exception.
For some key/value stores, a RetryTransactionException can mean that the transaction may have actually succeeded. In fact, this scenario is inevitable with any distributed database.
Therefore, application code must be prepared to retry. One easy way to do this is with the @RetryTranasaction annotation from dellroad-stuff (or another similar option), e.g.:
@RetryTranasaction
@Transactional
public void doWhatever() {
...
}Moreover, because a retried exception may have actually succeeded the first time, transactions must be written so as to be idempotent. Often this is already the case (e.g., creating a new user where the username must be unique), but in cases where it's not, the java.util.UUID class can be used make things idempotent:
@PermazenType
public abstract class AccountCharge {
public abstract String getUsername();
public abstract void setUsername(String username);
public float getAmount();
public float setAmount(float amount);
// Added to ensure idempotency
public UUID getUUID();
public UUID setUUID(UUID amount);
/**
* Charge user the given amount.
*/
public static void charge(String username, float amount) {
this.charge(username, amount, UUID.randomUUID());
}
@RetryTranasaction
@Transactional
private static void charge(String username, float amount, UUID uuid) {
// Ensure we don't add the same charge twice
if (JTransction.getCurrent().getAll(AccountCharge.class)
.stream().anyMatch(charge -> charge.getUUID().equals(uuid)))
return; // we already added it, bail out!
// Add new charge
final AccountCharge charge = JTransction.getCurrent().create(AccountCharge.class);
charge.setUsername(username);
charge.setAmount(amount);
charge.setUUID(uuid);
}
...
}Permazen supports a lock-free Counter data type. This means that multiple transactions can adjust the counter value without creating conflicts.
Aside from the obvious use case of counting things, Counter fields are also useful when combined with key watches as a way to implement automated notifications, especially across a distributed database.
Key watches are an optional feature supported at the key/value layer. Creating a key watch generates a Future that fires when the value associated with the specified key is changed by some committed transaction. If you watch the key associated with a Counter field, you can then increment the Counter field to trigger a notification to all listeners watching the key.
When used with distributed key/value stores like RaftKVDatabase this can function as an efficient inter-node notification mechanism. For example, by creating an "inbox" and "outbox" queue for each node in the cluster, you get efficient exactly-once message passing semantics.
The power of notifications is enhanced when combined with @OnChange methods. Notifications can be generated based on arbitrarily complex database state changes.
For example, suppose a manager wants to be notified whenever the average salary of all of his direct reports exceeds his own salary so he can ask for a raise:
@PermazenType
public class Employee implements JObjecgt {
// Fields
// My salary
public abstract int getSalary();
public abstract void setSalary(int salary);
// My direct reports
public abstract Set<Employee> getReports();
// Counter to be watched for raise needed notifications
public abstract Counter getRaiseNeededNotification();
// Salary monitoring
// Get a Future to watch for when I need a raise
public Future<Void> getRaiseNeededFuture() {
final PermazenTransaction jtx = this.getTransaction();
final byte[] key = jtx.getKey(this, "raiseNeededNotification");
return jtx.getTransaction().getKVTransaction().watchKey(key);
}
// Internal salary monitoring logic
// Invoked when my or any report's salary change
@OnChange({ "salary", "reports.element.salary" })
private void onReportSalaryChange() {
if (this.isRaiseNeeded())
this.getRaiseNeededNotification().increment();
}
// Compare my salary vs. my direct reports' average salary
@OnValidate
private boolean isRaiseNeeded() {
int avgReportSalary = this.getReports()
.stream()
.mapToInt(Employee::getSalary)
.average()
.orElse(0);
return this.getSalary() > avgReportSalary;
}
}User interfaces are normally built using the model-view-controller (MVC) pattern. In a typical pattern, a read-only model is constructed by querying the database for the latest relevant information. By "relevant" we mean the information that is to be displayed. The view builds the actual display based on this model. The controller consists of various buttons and widgets that allow the user to control what is supposed to be viewed.
The view may therefore need to change based on a change in viewing parameters (e.g., the controller's "next page" button pressed), or because the information shown has changed in the database and therefore the view has become out-of-date. Either case may be handled simply by rebuilding the model from a new query, then updating the view; this is a "refresh" operation. Often the user interface has a "Refresh" button for doing this.
Automating this update is easy when the update originates from a button or other controller entity, however, automating updates based on changes in the underlying database is harder, because this requires some sort of agent who listens for changes and then notifies the user interface to perform the refresh. The simpler forms of such a listening agent are often too inefficient: for example, it is usually not practical to update the user interface after any database change. Ideally, this listener only triggers when the part of the database on which the model is based changes.
When such an agent exists, it may also be possible for the notification to include information about exactly what changes occurred, so that the model does not have to be entirely reconstructed but instead can be "patched" by applying these changes. However, this optimization is rarely worth the complexity, because a human can only view so much information at a time, and therefore either reconstructing the model should be a fast operation or else you have a larger design problem to fix.
If your key/value store supports KVTransaction.readOnlySnapshot(), then the model is (re)created simply by taking a snapshot, a constant time operation. Be sure to close() the snapshot when you're done.
If the key/value store does not support KVTransaction.readOnlySnapshot(), then perform a query and copyOut() the objects you need into an in-memory transaction; the copied objects then serve as the model.
If your key/value store supports KVTransaction.watchKey(), efficient automated notification of database changes are possible using Counters and key watches as described above:
- For each view (or class of views), create a database object containing a
Counterto be incremented any time the view needs to be refreshed - Add
@OnCreate,@OnDelete, and/or@OnChangemethod(s) as needed to monitor for the relevant changes in the database - Within the annotated method(s), increment the
Counter
At this point we have a Counter that increments every time the view needs to refresh. Now we finish up as follows:
- Have each view monitor a
Futurereturned byKVTransaction.watchKey(), where the key to watch corresponds to theCounterfield (usePermazenTransaction.getKey()to determine which key to watch) - When the
Futuretriggers, refresh the model and view, get a newFuture, and repeat
If you're using RaftKVDatabase, it is safe to load the model in a read-only transaction with EVENTUAL_COMMITTED consistency, which means no network traffic will be required.
Now you don't need a "Refresh" button :)
The NavigableSetPager utility class is useful for paging through a NavigableSet of query results.
In particular, use it when implementing a paging table view with Previous and Next buttons.
Permazen verifies schema compatibility at the start of each transaction, and the schema is derived automatically from your Java classes, so it's important to know when changes to your Java classes are going to result in a new, incompatible schema. While this safety guarantee is good, it's nice to know about any problems at build time rather than at run time.
If you build using Maven, you can use the Permazen Maven Plugin to check. For ant users, the Permazen ant task works similarly.
Permazen records information about each schema in use in a special meta-data area of the key/value database. As the current schema changes over time, obsolete schema version meta-data accumulates. Although it doesn't take up much room, you can choose to "garbage collect" these obsolete schema versions. There is a CLI command for doing this, or you can do it programmatically. For example, in simple setups (e.g., only a single node and therefore no rolling upgrades), you may want to garbage collect obsolete schemas programmatically.
The example below does two things: (a) upgrade all objects to the current schema version, then (b) discards meta-data regarding any obsolete schema versions. Note that deleteSchemaVersion() will throw an exception if any objects having that version still exist (for safety), so in the code below (a) must occur prior to (b). Alternatively, you could perform step (b) without upgrading everything in step (a) if you only delete schema versions having no remaining objects; this additional condition is shown in the commented out line.
import io.permazen.PermazenObject;
import io.permazen.PermazenTransaction;
import io.permazen.core.Transaction;
import java.util.ArrayList;
import javax.annotation.PostConstruct;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.transaction.annotation.Transactional;
/**
* Bean that automatically upgrades all database objects to the current
* schema version and discards meta-data regarding obsolete schema versions.
*/
public class SchemaScrubber {
private final Logger log = LoggerFactory.getLogger(this.getClass());
@PostConstruct
@Transactional
public void start() {
// Upgrade all database objects to the current schema version
final PermazenTransaction jtx = PermazenTransaction.getCurrent();
jtx.getAll(PermazenObject.class).forEach(PermazenObject::upgrade);
// Delete any obsolete schema version meta-data
final Transaction tx = jtx.getTransaction();
final int currentVersion = tx.getSchema().getVersionNumber();
this.log.info("scrubbing obsolete schema versions; current version is " + currentVersion);
new ArrayList<Integer>(tx.getSchemas().getVersions().keySet()).stream()
.filter(v -> v != currentVersion)
// .filter(v -> !tx.queryVersion().containsKey(v))
.forEach(v -> {
this.log.info("deleting meta-data for obsolete schema version " + v);
tx.deleteSchemaVersion(v);
});
}
}Note you can also do this via the CLI:
-
eval all().forEach(PermazenObject::upgrade)will upgrade the entire database -
jsck -gcgarbage collects unused schema versions