Skip to content

Home

Jump to bottom Edit New page
Simon Y. Blackwell edited this page Aug 1, 2018 · 59 revisions

Summary

**NOTE: In 2017 JOQULAR as a code base was replaced by https://github.com/anywhichway/reasondb. JOQULAR will not be just a query specification.

NOTE: This Wiki is for Version 1 of JOQULAR. An ALPHA of Version 2 of JOQULAR was released on January 10th, 2016. It is available at this branch https://github.com/anywhichway/joqular/tree/v2. It is recommended you not start using Version 1 for anything other than exploratory investigation and you may simply be better off with the ALPHA. See the README: https://github.com/anywhichway/joqular/blob/v2/README.md.

JOQULAR (JavaScript Object Query Language Representation)

  1. Serializable pattern and SQL like object matching for JavaScript, including joins!.
  2. More built-in predicates/operators than most other other data query mechanisms, currently 44. Includes built in random and statistical sampling capability.
  3. Extensible with just one line of code per predicate/query operator. Puts the intelligence in what you store, not the database engine.
  4. Just-in-time, fully indexed in-memory database, 2x to 10x faster than linear search, faster than IndexedDB and PouchDB for insert and search.
  5. Indexes represent the live state of JavaScript objects. Results are also live state or POJO projections, your choice.
  6. Extensive Date and Time comparisons with precision at the year, month, day, hour, second, millisecond
  7. Optional zero configuration indexing and persistence.
  8. 3-Way Single Function You Choose API ... callbacks, Promises, return values.

Coming In July

  1. Enhanced persistence with SQL like Insert, Update, and Delete for both client and server
  2. Language syntax changes, all functions will need to be preceded by $ sign. This is consistent with other providers approaches and also gives our query compiler optimization hints. We apologize for the lack of backward compatibility this will introduce.

Examples

// everyone named Joe
{name: {eq: 'Joe'}} 
// all adult women
{age: {gte: 18}, gender: 'female'}} 
// adult women Bainbridge Island and downtown Seattle
{age: {gte: 18}, gender: 'female', address: {zipcode: {in: [98110,98101]}}}} 
// all grandsons named the same as their grandfather
{father: {father: {name: {'/': 'name'}}}} 
// all partners who are partnered with the same gender
{partner1: {gender: {'..partner2': 'gender'}}}
// all partners who are not partnered with the same gender
{partner1: {gender: {neq: {'..partner2': 'gender'}}}} 
// Joe's children, if any are sick
{name: 'Joe', {children: {some: function(child) { return child.isSick; }}}} 
// Joe's children, if all are sick
{name: 'Joe', {children: {every: function(child) { return child.isSick; }}}} 
// [], unless all females are named Jo
{gender: 'female', {forall: function(object) { return object.name==='Jo'; }}} 
// all females if any are named Jo
{gender: 'female', {exists: function(object) { return object.name==='Jo'; }}} 
// anyone named Joe or Jo
{name: {soundex: 'Joe'}} 
// anyone with a name starting in Jo
{name: {match: /Jo*/}}
// anyone who is female and authorized based on the value, i.e. 21 or over 
function authorized(value) { return value>=21; }.predicate=true;
{gender: 'female', {age: {$: authorized}}} 
// anyone who is female and authorized based on the object, i.e. a volunteer 
function authorized() { return this.volunteer }.predicate=true;
{gender: 'female', $$: authorized}
// general query
select().from({p1: Person}).where(<any of the above patterns>);
// all Person's ordered by zipcode, then name
select().from({p1: Person}).orderBy({'p1.address.zipcode': 'asc', 'p1.name':'asc'});
 // all combinations of people
select().from({p1: Person,p2: Person}).where({p2: {neq: p1}});
// first 10 Person's matching a query
select().first(10).from({p1: Person}).where(<any of the above patterns>);
// last 10 Person's matching a query
select().last(10).from({p1: Person}).where(<any of the above patterns>);
// random sample at 95% confidence +/- 3
select().sample(.95,.03).from({p1: Person});

Documentation

The following sections of documentation are provided:

  1. An Introduction, History, Design Philosophy, Compatibility and License on this page.
  2. An overview of JOQULAR Patterns
  3. [JOQULAR Select & Joins]https://github.com/anywhichway/joqular/wiki/JOQULAR-Select-&-Joins)
  4. API and How Develop using joqularMatch, joqularFind, and their supporting functions like createIndex and joqularIndex.
  5. Example Code
  6. Extending JOQULAR
  7. Benchmarks.
  8. A description of the internals.
  9. Comparisons to other client side JavaScript databases, although like most, JOQULAR can also be used on the server in node.js.

Introduction

The standard JOQULAR release provides:

  1. A means for expressing serializable data queries in JSON, this includes expressing predicate function calls like has, intersects or isNaN. All keys in JSON objects become either references to values, references to nested objects, or references to predicates. By declaring functions on prototypes as predicates, the language is almost infinitely extendable, e.g. using {has: } is legal for Sets by default but becomes legal for Arrays with just one line of code, e.g. 
(Array.protoype.has=function(value){return this.indexOf(value)>=0;}).predicate=true;
  1. A library that implements the ability to match in memory JavaScript objects, e.g.
this.joqularMatch({name:{match:/S*/g,age:{gt:24}}) // true if name starts with S and age > 24
  1. A library that implements a high speed, indexed, in-memory database of JavaScript objects that is always up to date with the current state of the objects, e.g.
var o1={pairs:[[1,2],[3,4],[5,6]]};
var o2={pairs:[[3,4],[5,6],[1,2]]};
var o3={pairs:[[3,4],[5,6],[7,8]]};
Object.find({pairs:{contains:[1,2]}}) // returns [o1,o2]
o1.pairs[0][0]=[7,8];
Object.find({pairs:{contains:[1,2]}}) // returns [o2]

Depending on the disbursement and ordering of data, querying this database can be 5x to 10x faster than linear searches. Because JOQULAR defers 75% of the indexing effort until the first query requiring the index, the up front cost of indexing is marginal. It also turns out that the just-in-time indexing has a negligible impact on the first query. What is avoided and passed onto the user as time savings is lots of thrashing during object inserts and updates. Subsequent queries use a cached index until it is invalidated by an update or delete. See benchmark page and Internals for details.

Adding instances of any class to the database is far easier than creating and maintaining your own indexing mechanism or manually designing indexes for a database. It is done with one call that results in all properties of class instances, including potential method calls, to be indexed when future instances are created, e.g.

MyClass = JOQULAR.createIndex(MyClass);

History

The basic code from which JOQULAR is derived is at the core of a multi-database query engine used by AnyWhichWay on client projects. This engine has the ability to query and update Google Cloud Datastore, Quickbase, Elastic Search, and some Amazon AWS services using the same expression language. The engine provides most SQL operations like JOIN, UNION, TOP, as well as summary routines like SUM, AVG, TOTAL, STDEV, etc. We did not feel the entire engine was at a quality level it could be released; so we extracted the best parts and are making them available separately. JOQULAR is the first and most broad reaching. The second is RIJS, pronounced "Regis", (Relational Referential Integrity For JavaScript) which we plan to release in June.

Design Philosophy

The intent is to keep the library modular, light weight and approachable. Hence:

  1. Standard Javascript syntax and semantics are leveraged and a specialized parser is not required. All expressions are represented as JSON with a few additional reserved words for keys, e.g. gt, gte, etc.
  2. Memory and storage are cheap, time is not. Save developer and user time by indexing everything, so long as it does not slow down inserts and queries.
  3. The libraries have no external dependencies.
  4. Extending JOQULAR does not require knowledge of or access to the source code.

Compatibility

JOQULAR will work in the latest versions of Chrome and node.js without pollyfills. Firefox and IE require polyfills for Object.Observe. IE requires a polyfill for Promise. And, despite the use of a Set polyfill, IE will not work with the Set functionality of JOQULAR, you will have to stick with Arrays.

Users have found success using: https://github.com/MaxArt2501/object-observe/blob/master/dist/object-observe.js for Object.observe.

License

The language expression, to the degree that it can be copyrighted (if at all), is provided under the Creative Commons License.

The library code is provided under the GPL 3 License.

Copyright (and left) 2015 AnyWhichWay, LLC and Simon Y. Blackwell

Add a custom footer
Add a custom sidebar
Clone this wiki locally