OpenCV / Surveyor pattern / Nanomsg

[roscopecoltran]

Hi guys,

Hope you are all well !

I was checking out your cool library for a project I am working on based on OpenCV and Dlib.

In a nutshell, I would like to create a graph of distributed pre and post processing tasks on a stream web-camera image on iOS . In a nutshell, I would like to use RaftLib as in your example with openCV, and add a layer the surveyor protocol from the nanomsg library, to execute a flow like the following:

Events flow example:

1. Skip frame, execute chain of processes every 3 frames
2. Resize frame
3. Filter frame, eg: if the image is blurry, stop the chain if not valid quality
4. Rotate frame
5. Mirror frame
6. Distributed Processing frame
   • dlib's faciallandmark detection process (timeout 1s)
   • inverted visual dictionary (simple bag of words based on Features2d module) (timeout 1s)
7. Processing locks
   • if face detected stop the pre-processing steps, and block request for marker detection.
   • if a sub-process return no results, return to the default processing flow.

Questions:

1. Would it be possible to do such processing flow with RaftLib ?
2. Is it possible to couple Raftlib with puffin-buffer and puffin-stream to aggregate results ?
3. As I saw that Raftlib is really performance oriented, what would be the performances bottlenecks in such flow based processing ?

Refs:

• http://www.learnopencv.com/speeding-up-dlib-facial-landmark-detector/
• https://github.com/nanomsg/nanomsg
• https://github.com/nanomsg/nng (Incoming new version of nanomsg)
• https://github.com/daniel-j-h/DistributedSearch (Distributed Search based on the nanomsg suveyor pattern)
• https://github.com/designerror/puffin-buffer#example
• https://github.com/designerror/puffin-stream#example

Have a great day !

Cheers,
Richard

[jonathan-beard]

Hi. Thanks for checkout out the library. I'll answer in order, only partially (need to read the links you provided a bit)

• Would it be possible to do such processing flow with RaftLib?

Yes

-Is it possible to couple Raftlib with puffin-buffer and puffin-stream to aggregate results ?

Likely, I need to read through the provided links a bit more. Seems puffin-buffer uses fstream which should work just fine. I'll see if I can put together an example if you haven't already. Any chance puffin has documentation somewhere? Will need to dig through their code to make sure it's thread safe (the file freed/fseek/etc.) inside wouldn't be thread safe by themselves typically.

-As I saw that Raftlib is really performance oriented, what would be the performances bottlenecks in such flow based processing?

That depends. I built RaftLib while working on stream processing applications for bioinformatics and genomics. The bottleneck varies greatly by application. Typically stream processing makes parallelizing applications easier, and also makes the analysis easier to do statically (even dynamic reasoning becomes easier). The next major concern is minimizing data movement and locality. After that, you have the issue of queueing. To get the maximum performance you have to enable mis-matched rates between actors. Picture two exponential random number generators, they usually don't put out the exact same numbers if started with different seeds. Actors are like that, sometimes they'll dump a lot of data all at once, despite having the exact same mean rate for producer and consumer. You want to minimize blocking, and maximize the work. My goal with RaftLib was to make all the above easier to reason about without having to write thousands of lines of C code to get it done.

I'll read up on the other links.

[roscopecoltran]

Hi,

Hope you are all well !

Sorry for the delay in my reply ^^ vacation time :-)

I could not compile Raftlib for iPhoneos using some toolchains provided by Polly because of Scotch dependency.

After a quick look to its sources, I seems that it does not compile for armv7 or arm64. Am I correct/wrong ^^ ?

Would you mind to create a 3rparty folder with all the Raftlib dependencies sources ?

More globally, is Raftlib supposed to be working on mobile devices ?

Context:
I just added Boost dependency with hunter (link) but all others dependencies are not hunterized.

Command used (with polly) at root project dir:

$ build.py --toolchain=ios-10-2 --config Release --reconfig --clear --jobs 4

Thanks in advance and have an awesome week !

Cheers,
Richard

[jonathan-beard]

I hadn't originally intended this for mobile, but it's a direction I've been thinking about going in. It will build for raspberrypi arm64, softiron 1000, and for zynq-7020 (without scotch). It'll build just fine without the partitioner. I'll have time over the weekend to check on the iOS build chain using Polly. There should a be a few #defines that might have to be updated to include iOS but that should be about it.