Multithread implementation of the search

[savvas-paragkamian]

How to implement multithread option of the searches?

See the awk scripts that do text mining with awk

[savvas-paragkamian]

He makes use of MPI language called Swift in order to call AWK script concurrently. Each thread of the HPC handles a specific file, this means that for the Batch partition up to 200 files can be handled simultaneously.

Batch is composed by 10 nodes each one with 20 cores and 128 GB RAM. A user can assign parallel jobs (either in a single node or across several nodes) to the batch partition.

[savvas-paragkamian]

In an HPC environment I can use Job Arrays to call a process in each core. Independent processing jobs, see here for more info.