NextTypes is a standards based information storage, processing and transmission system that integrates the characteristics of other systems such as databases, programming languages, communication protocols, file systems, document managers, operating systems, frameworks, file formats and hardware in a single tightly integrated system using a common data types system.
The reference implementation is programmed in Java 17 and uses the PostgreSQL 15 database manager. It is distributed under the Apache 2.0 license, in an easily installable WAR file. To test the software interface (in read-only mode), without having to install it, you can explore the project website, which is developed using said software. In the menu on the left, in the "Control panel" section and in the buttons on the top/right of the page, you will find the available actions.
NextTypes integrates the primitive PostgreSQL data types such as numbers, text, binary, dates, JSON or XML, although giving them different names to simplify nomenclature and facilitate integration with other systems. For example, instead of using "smallint", "integer" and "bigint" the root "int" followed by the number of bits (int16, int32 and int64) is used or instead of "character varying" or "varchar" it is used "string" as in some programming languages.
It also adds other types of data such as HTML or URL, and using PostgreSQL ability to create composite data types for columns it includes data types for files, documents, images, audios and videos. The binary content of these fields can be scanned with the ClamAV antivirus. Complex data types such as HTML or images are made up of elements that can be accessed individually and can be applied restrictions to them such as tags or formats allowed.
The storage system is primarily SQL based but is a relational/network/objects/files hybrid. Each table is a data type and each row in a table is an object. Each row has a column with an identifier (id) that is the primary key and several columns with metadata similar to those of the files: creation date (cdate), update date (udate) and backup made (backup). The rest of the columns are the data fields of the object. The date of creation (cdate) and alteration (adate) of the data types is also stored. Dates use the UTC standard and include hours, minutes, seconds and microseconds. The ISO 8601 standard is used for its representation in text.
The fields of the objects can be of a primitive type or point to other objects using the identifier as a foreign key, thus forming a network database. The identifier is a text string with a unique index and by default it receives a UUID. This simplification and uniformity in the primary and foreign keys facilitates automation and integration with other systems.
The correspondence with the entity-relationship model is similar to that of the relational model. Each entity is an object, attributes are fields, and relationships and multiple attributes are made with fields that refer to other objects.
Object identifier and fields can be part of non-unique, unique, and full-text search indexes. The definition of data types and objects can be exported and imported to/from JSON, allowing backup copies and transfer of information between systems.
{
"name" : "article_language",
"cdate" : "2015-04-01T14:30:16Z",
"adate" : "2018-09-26T14:59:35.53Z",
"fields" : {
"title" : {
"type" : "string",
"length" : 254,
"not_null" : true
},
"language" : {
"type" : "language",
"length" : 100,
"not_null" : true
},
"text" : {
"type" : "html",
"not_null" : true
},
"article" : {
"type" : "article",
"length" : 100,
"not_null" : true
}
},
"indexes" : {
"al_ft_search_index" : {
"mode" : "fulltext",
"fields" : [ "title", "text" ]
}
},
"actions" : { }
}
The alteration and update dates allow the implementation of an optimistic concurrency control system. If when altering a type or updating an object the last known date of alteration or update is indicated, the system checks that this date is the same as the current date of the type or object to rule out that it has been modified by another user at a later date.
The "backup" metadata is used to create an incremental backup system of updated objects with full copies each a specified number of incremental copies. Every time an object is updated the metadata "backup" is changed to "false" to indicate that there is no copy of that object and it should be copied in the next backup.
Through the use of PostgreSQL features such as transactional DDL and deferrable constraints, the system allows the creation or modification of various data types and objects in the same transaction while the system is in use.
To complement the full-text search system, the extraction of metadata and text from PDF, OpenDocument and Office Open XML documents, Microsoft Office binary formats and other formats is added using Tika.
NextTypes is a mainly relational system with some object-oriented features that facilitate its use and automation from programming languages. Unlike object-relational systems, it does not use an object-class correspondence, instead it uses a series of generic classes and methods for all types and objects that allow performing the basic actions (CRUD) of creation, reading, updating and deletion.
For complex actions such as queries that use several tables it provides a system with a higher level of abstraction than JDBC that facilitates the use of SQL and integrates with the data types of the storage. This system allows parameterized queries with the execution of a single method and uses table and column names as parameters, which offers protection against SQL injection. It also allows you to use an array as a parameter.
Basic actions can be intercepted similar to SQL triggers to modify types or objects or perform additional actions. In addition, new actions can be defined and made available to the system.
In the Java implementation some primitive data types in storage, such as numbers, have a direct correspondence with classes in the Java Runtime Enviroment. For other types such as HTML, documents, images or videos, the system has classes that integrate with the storage types and allow the creation and modification of data.
The main protocol used is HTTP, although adapters can be created for any other protocol. The system includes an SMTP adapter that allows the reception of emails and their conversion into objects. On top of HTTP, a REST interface is built with authentication through X.509 certificates and passwords (encrypted with bcrypt), HTTP Basic Auth, protection against DoS attacks and management of the robots file and web sitemap. This interface allows access to all types of data, objects, fields, and elements with one URL.
The storage, processing and transmission systems form an MVC system that allows access to the data through different views or formats: HTML, WebDAV, CalDAV, JSON, XML, RSS, iCalendar or other systems. Each of these views can be modified for all types of data or objects or some of them to adapt it to the user's needs. UTF-8 encoding is used throughout the system to allow the use of text in different languages.
The HTML view provides a responsive and multi-language graphical interface that allows to perform all the actions of the storage/processing system. It is programmed in HTML5/CSS3 with SVG resizable icons, WCAG compliance, text editors with syntax highlighting (Codemirror) or WYSIWYG (TinyMCE) and protection against CSRF and XSS. This view integrates the HTML5 data types with those of NextTypes, adding the necessary components to the interface to complement the controls provided by HTML5. In the "example" data type you can see the controls for all primitive data types and object references. You can adapt the HTML view by modifying the DOM.
Each object has an identifying QR code with the name of the server, the type of data and the identifier of the object. This code can be used to label real objects or read it from a mobile application. There is also the possibility to export some data in JSON-LD format.
Other views allow remote access to data by any system. Through the use of Javascript and JSON or XML views it is possible to access all the data and carry out information processing in the browser. The WebDAV view allows you to access data as a file system in which the data types and objects are folders and the fields are files. This allows you to open and modify any field of an object with an external application. For example, you can edit a document field with a word processing application.
The CalDAV view is a modification of the WebDAV view that allows you to expose some objects as calendar events. Those events can also be accessed using the iCalendar view. The RSS view allows access to object listings with any client of that format.