Multi-threading primitives that use a fixed thread pool

[matko]

This pull request provides some predicates that can be used to do multi-threading in terminusdb. They are implemented using a thread pool that is initialized once at startup with an amount of threads equal to the amount of hardware threads available to terminusdb. Its purpose is to run cpu-bound work, that is, work that mostly (if not only) does calculation, and no I/O.
For truly cpu-bound workloads there is no point in having more than the amount of hardware threads, and in fact this hurts performance. So we don't want every request to potentially spawn its own number of threads causing thread count to fluctuate widely. Instead, parts of the code that need to do cpu-bound multi-threading can all go through this common thread pool.

Currently implemented:

• cpu_concurrent_findall/4 takes a template, a generator goal and an action goal, generates results on the calling thread using the generator, and schedules the action goal to be run on the thread pool, collecting the results. cpu_concurrent_findall/4 is stable in its result order, including error results. Use case is schema checking where we want all witnesses.
• cpu_concurrent_forall/2 takes a generator and an action goal where the generator goal generates results on the caslling thread, and the action goal is scheduled to run on the thread pool. This is a drop-in replacement for forall/2. Use case is document elaboration and insertion.
• cpu_concurrent_findfirst/4 is like cpu_concurrent_findall/4, but stops at the first result and returns that, rather than collecting all results. Again, this is stable, always returning the same first result, rather than the result which happened to complete first. Use case is schema checking where we are interested in the first witness.

Planned:

• cpu_generate/5: Like cpu_concurrent_findall/4 but backtracking over results rather than collecting a list. Extra argument is to specify how far ahead results should be generated. Use case is document retrieval (generate documents to return in the background while the main thread writes them to the cgi stream).
• cpu_grouped_findall/5, cpu_grouped_findfirst/5 and cpu_grouped_forall/3 where an extra numerical argument is given. This argument N specifies how many elements should be generated before being handed off to a background thread, instead of processing each generated item on a background thread individually. This will likely improve performance due to less messaging overhead. However, it's a tradeoff, as this will also keep the cpu-bound threadpool busy longer, preventing other requests from getting work done, possibly leading to starvation issues.

This is a draft for now while those planned primitives are still under development.

[matko]

This draft has evolved a bit from its original form. I'm now implementing a task framework. The reason for doing this is that it can often happen that a task running on the threadpool itself will want to fork off other tasks, and these too need to run on the thread pool. The original implementation would not support this, as submitting work on the thread pool would block if no workers were available.
Such a task mechanism will come in handy especially for recursive algorithms, such as our document elaboration, where enough work can be done to warrant the overhead of multithreading. This is the case for example for enormous documents that consists of large lists of subdocuments.

Currently running into some weird deadlock behavior in task.pl with multiple uses of thread_wait/2, which I think is a problem in the core library. I'll be rewriting this to using a message queue to wake up the main scheduler instead.