Likeliness between faces

[RealHarshThakur]

First off, just want to appreciate and thank you for this cool project you're working on.
I was wondering if you could add the feature to compare two faces and describe how likely they are.

[esimov]

Theoretically it's possible by using a features detection algorithm, like SURF or SIFT, but the implementation requires more preparation. Anyway I will ad it into the TODO list.

[RealHarshThakur]

Yes. I have seen other frameworks in Python which do it that way. Once they create the face encodings for the faces, then they calculate distance using Euclidean or some other formula.

[esimov]

I've developed another library (https://github.com/esimov/forensic) which detect copy move forgery in images, this at some degree resembles the same theory, but not really sure if I can adapt it to this problem. Maybe i'll evaluate it.

[RealHarshThakur]

Just took a look at it. That's really interesting. The underlying solution seems to be pretty much the same . Please, do let me know your findings. Also, I just realized if you do add this feature, this would turn into facial recognition library.

[olekukonko]

@harshthakur9030 you might want to see #23

[RealHarshThakur]

Thanks @olekukonko . Preciously the points I had in mind. I ought to find some time to contribute to this in my free time. I would be glad if you or @esimov could point me how to go about it .

[esimov]

I made a little research on the possibilities and I opted to implement the SURF (Speeded Up Robust Features) method - http://people.ee.ethz.ch/~surf/eccv06.pdf. I assume the integration should be possible without major roadblocks and should be also fast.

[olekukonko]

@esimov

I was doing a little research and I noticed dlib decent level of accuracy and speed can be attributed to using HOG - source & Support Vector Machine - SVM

Looking at research - A comparative study of CFs, LBP, HOG, SIFT, SURF, and BRIEF for security and face recognition

When reading a ROC curve, the points farther away from the diagonal line (y = x) and closer to closer to the point of coordinates (0, 1) are better, as that indicate lower FPR (fewer incorrect classifications) and higher TPR (more correct classifications). The Figure 3 shows ROC curves for the six methods, the VLC and LBP achieve better performance than HOG, SIFT, SURF, and BRIEF. Also, the face recognition times for different methods as shown in Table 1. The VLC which is based on the fast Fourier transform (FFT) and HOG which is calculate histograms by using the angle orientation of the edges are the less complex methods. Note that we have implemented VLC on GPU in [20].

The results clearly show that VLC and LBP outperformed the other studied methods on all the subsets of PHPID dataset. Finally, we conclude in this work that VLC and LBP are the best methods in terms of implementation and application in real-time in future work.

On the Issue of also fast.

HOC, VLC, and LPB seems to also be faster

Question

• Is there any particular reason you chose SURF?
• Does it mean pigo's Facial landmark points detection would work better with SURF
• Is there a way to have pluggable methods?

[esimov]

HOG might be faster in computation than SURF, but as far as I know it's mostly used in image classification (aka to feed up the feature vectors into an image classification algorithm like SVM) so i'm afraid it wouldn't be too useful for a simpler task like image features detection and matching. Please correct me if i'm wrong.

Right now the library is more a CV than a ML library. So I foreseen the next step in the library evolution of being capable in recognizing image similarities, in other words to detect identical faces. Later on, if everything is going well it can be evolved into a ML direction, in terms of being capable of image classification and training.

Is there a way to have pluggable methods?

That's a good idea. This is how I conceive the integration of feature detectors: pluggable methods like in OpenCV.

[olekukonko]

HOG might be faster in computation than SURF, but as far as I know it's mostly used in image classification (aka to feed up the feature vectors into an image classification algorithm like SVM)

Absolutely, however, SURF can also be used to feed SVM see Face Recognition using SURF Features and SVM Classifier

So I foreseen the next step in the library evolution of being capable in recognizing image similarities.

Makes a whole lot of sense.

Later on, if everything is going well it can be evolved into a ML direction, in terms of being capable of image classification and training.

Too bad Edward Walker port of libsvm has not been maintained for 6 years ( but can still be savaged anyways). After a SURF Implementation to pigo feeding it into a reliable SVM implementation would have been a breeze

You can be sure you have my full support. Nice one.

[dilshat]

Awesome. Looking forward to this feature!

[andrewwormald]

Wondering if there’s been any time put in on the above conversation? Looks very interesting and would love to see the SURF implementation.

[esimov]

I didn't had too much time lately focusing on this feature, effectively nothing has been done yet, only research, but it's not abandoned at all. I really hope that the next big thing on the project development will be this feature.

[gedw99]

photoprism is using pigo to do the same thing. Check their code.
https://github.com/photoprism/photoprism/tree/develop/internal/face

they use clustering to avoid needing kmeans. Its also faster then kmeans

[esimov]

@gedw99 in case you know, can you point out where exactly this is implemented? I was searching through the code, but couldn't find exactly where is done.

[gedw99]

I found an issue where they discussed it a while ago. I searched for it again before i wrote, but could not find it again.

[gedw99]

Hey @esimov they use pigo core here, but i did not really look into it to much.. But it might be good for you and others here to see how they use it.

https://github.com/photoprism/photoprism/blob/develop/internal/face/detector.go#L13

[gedw99]

I didn't had too much time lately focusing on this feature, effectively nothing has been done yet, only research, but it's not abandoned at all. I really hope that the next big thing on the project development will be this feature.

Maybe do a c to golang transpilation for some of the opencv aspects. I know someone else inthe GIo community working on it.

https://github.com/gotranspile/cxgo has some examples and is highly active.

POTrace has been done this way: https://github.com/dennwc/gotrace, and i saw you using the c version here: https://github.com/esimov/colidr#requirements. It's quite cool how it supports the pure go as well as the c version. Make it easier to do porting. BTW @dennwc this is really inspiring !!

[esimov]

Maybe do a c to golang transpilation for some of the opencv.

There is no OpenCV involved in this project. Pigo is a standalone face detection application and I don't want at all to involve OpenCV exactly for that reason, that it requires clang to communicate with the C backend, and as you probably know this operation is not cost effective. So in order to support the features detection I opted to implement the SURF detection method as I mentioned earlier in this thread.

[Deng-Xian-Sheng]

I look your code, very good!
if this project next, please @ me!
I no money, but me like this project.