New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Test it on VOC2007 and get 60% mAP #147
Comments
This is a nice work for tensorflow developer, and cloud you fix the code of darkflow according to project https://github.com/leetenki/YOLOv2 and https://github.com/hizhangp/yolo_tensorflow or https://github.com/gliese581gg/YOLO_tensorflow |
@ShawnDing1994 cloud you contribute your code for computing mAP of darkflow? |
When building the repo, I see minor numerical differences happens after each operation (e.g. convolution in TF produces result different from Darknet within 1e-4 or so), they get amplified after each layer and eventually results in (catastrophically) inferior mAP in this Tensorflow port. Pretty sure the high-level implementation in this repo is correct (reasonable detection), but lower level (opKernels in Tensorflow and C implementation in Darknet) may have technical mismatch. It is important to understand how severe it is even when numerical difference is small (< 1e-7). Each trained layer in Darknet expects a different result than what it obtains from Tensorflow, thus produce an even worse result for the next layer. This error accumulates as the net gets deeper. So, fine-tuning from Darknet weights is a cheap and direct solution to what I described above. Hope this helps. |
@ShawnDing1994 how did you implement mAP for darkflow, could you contribute your code so I can use it? or could you guide me on how to implement it? |
Hi,
Nice work!
But I wrote some codes to test it on VOC2007 test set with configuration '--model cfg/yolo-voc.cfg --load models/yolo-voc.weights' and got 60% mAP, while the YOLO 9000 paper reported 78.6%.
Then I changed input resolution in yolo-voc.cfg from 416416 to 544544 and improved mAP by 2%.
I also tested on a few images and compared them with outputs from darknet and saw significant differences.
I used these testing codes a few times before and am sure they are bug-free.
So I think there must be some bugs...
Thanks
The text was updated successfully, but these errors were encountered: