Supports yolov5 and yolov8, with models directly uploaded for AI prediction

[kkive]

I am using makesense.ai for data inference work,
Running AI locally for inference is a great feature, but for me as a beginner, it's a bit difficult to convert [. pt] files to [tensorflow. js], and I don't quite understand how to obtain [model. json] files.
Therefore, it would be great if makesense could support uploading [. pt] files for YOLOv5 and YOLOv8 for instance segmentation reasoning

[github-actions]

👋 Hello @kkive, thank you for your interest in make-sense - free to use online tool for labelling photos! 🏷️

🐞 Bug reports

If you noticed that make-sense is not working properly, please provide us with as much information as possible. To make your life easier, we have prepared a bug report template containing all the relevant details. We know, we ask for a lot... However, please believe that knowing all that extra information - like the type of browser you use or the version of node you have installed - really helps us to solve your problems faster and more efficiently. 😉

💬 Get in touch

If you've been trying to contact us but for some reason we haven't responded to your issue yet, don't hesitate to get back to us on Gitter or Twitter.

💻 Local setup

# clone repository
git clone https://github.com/SkalskiP/make-sense.git

# navigate to main dir
cd make-sense

# install dependencies
npm install

# serve with hot reload at localhost:3000
npm start

To ensure proper functionality of the application locally, an npm 8.x.x and node.js v16.x.x versions are required. More information about this problem is available in the #16 issue.

[Babyhamsta]

You can only run YOLOv5 models through MakeSense currently. You can easily export them using YOLOv5's export CLI.
python export.py --weights best.pt --include tfjs (Just adjust to your PT model)

I would like to see the ability to run YOLOv8 web models, I recently upgraded to YOLOv8 and noted that the Web Export appears to be different than the YOLOv5 ones.

YOLOv5

YOLOv8

There is also a difference in the model.json output:

YOLOv5 (snippet):
{"format": "graph-model", "generatedBy": "2.11.1", "convertedBy": "TensorFlow.js Converter v3.21.0", "signature": {"outputs": {"Identity": {"name": "Identity"}, "Identity_1": {"name": "Identity_1"}, "Identity_2": {"name": "Identity_2"}, "Identity_3": {"name": "Identity_3"}}}, "modelTopology": {"node": [{"name": "model/agnostic_nms_1/agnostic_nms/TensorArrayV2_3/element_shape", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/agnostic_nms_1/agnostic_nms/TensorArrayV2_3/num_elements", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/agnostic_nms_1/agnostic_nms/TensorArrayV2Stack/TensorListStack/element_shape", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {"dim": [{"size": "2"}]}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/agnostic_nms_1/agnostic_nms/TensorArrayV2_4/element_shape", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/agnostic_nms_1/agnostic_nms/TensorArrayV2_4/num_elements", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/agnostic_nms_1/agnostic_nms/TensorArrayV2Stack_1/TensorListStack/element_shape", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {"dim": [{"size": "1"}]}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/agnostic_nms_1/agnostic_nms/TensorArrayV2_5/element_shape", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/agnostic_nms_1/agnostic_nms/TensorArrayV2_5/num_elements", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/agnostic_nms_1/agnostic_nms/TensorArrayUnstack_1/TensorListFromTensor/element_shape", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {"dim": [{"size": "2"}]}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/agnostic_nms_1/agnostic_nms/TensorArrayV2Stack_2/TensorListStack/element_shape", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {"dim": [{"size": "1"}]}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/agnostic_nms_1/agnostic_nms/TensorArrayV2_6/element_shape", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/agnostic_nms_1/agnostic_nms/TensorArrayV2_6/num_elements", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "ConstantFolding/model/tf.math.truediv/truediv_recip", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {}}}, "dtype": {"type": "DT_FLOAT"}}}, {"name": "ConstantFolding/model/tf.math.truediv_1/truediv_recip", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {}}}, "dtype": {"type": "DT_FLOAT"}}}, {"name": "ConstantFolding/model/tf.math.truediv_2/truediv_recip", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {}}}, "dtype": {"type": "DT_FLOAT"}}}, {"name": "model/tf.split/split/split_dim", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/tf.__operators__.getitem/strided_slice/stack", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {"dim": [{"size": "2"}]}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/tf.__operators__.getitem/strided_slice/stack_1", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {"dim": [{"size": "2"}]}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/tf.__operators__.getitem/strided_slice/stack_2", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {"dim": [{"size": "2"}]}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "ConstantFolding/model/tf.math.truediv_3/truediv_recip", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {}}}, "dtype": {"type": "DT_FLOAT"}}}, {"name": "model/tf.concat/concat/axis", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/agnostic_nms_1/agnostic_nms/TensorArrayUnstack/TensorListFromTensor/element_shape", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {"dim": [{"size": "2"}]}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/tf.__operators__.getitem_1/strided_slice/stack", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {"dim": [{"size": "3"}]}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/tf.__operators__.getitem_1/strided_slice/stack_1", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {"dim": [{"size": "3"}]}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/tf.__operators__.getitem_1/strided_slice/stack_2", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {"dim": [{"size": "3"}]}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/tf_detect/strided_slice_3/stack", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {"dim": [{"size": "2"}]}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/tf_detect/strided_slice_3/stack_1", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {"dim": [{"size": "2"}]}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/tf_detect/strided_slice_3/stack_2", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {"dim": [{"size": "2"}]}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "model/tf_detect/mul_2/y", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {}}}, "dtype": {"type": "DT_FLOAT"}}}, {"name": "model/tf_detect/sub", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {"dim": [{"size": "1"}, {"size": "6400"}, {"size": "1"}, {"size": "2"}]}}}

YOLOv8 (snippet):
{"format": "graph-model", "generatedBy": "2.11.1", "convertedBy": "TensorFlow.js Converter v3.21.0", "signature": {"outputs": {"Identity:0": {"name": "Identity:0"}}}, "modelTopology": {"node": [{"name": "ConstantFolding/PartitionedCall/model_4/tf.math.divide/truediv_recip", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {"dim": [{"size": "1"}, {"size": "1"}, {"size": "1"}]}}}, "dtype": {"type": "DT_FLOAT"}}}, {"name": "1328", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {"dim": [{"size": "1"}, {"size": "2"}, {"size": "8400"}]}}}, "dtype": {"type": "DT_FLOAT"}}}, {"name": "1320", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT64", "tensorShape": {"dim": [{"size": "3"}]}}}, "dtype": {"type": "DT_INT64"}}}, {"name": "1322", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT64", "tensorShape": {"dim": [{"size": "3"}]}}}, "dtype": {"type": "DT_INT64"}}}, {"name": "PartitionedCall/model_4/tf.strided_slice_19/ones_like", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT64", "tensorShape": {"dim": [{"size": "3"}]}}}, "dtype": {"type": "DT_INT64"}}}, {"name": "1330", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {"dim": [{"size": "1"}, {"size": "2"}, {"size": "8400"}]}}}, "dtype": {"type": "DT_FLOAT"}}}, {"name": "1314", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT64", "tensorShape": {"dim": [{"size": "3"}]}}}, "dtype": {"type": "DT_INT64"}}}, {"name": "1316", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT64", "tensorShape": {"dim": [{"size": "3"}]}}}, "dtype": {"type": "DT_INT64"}}}, {"name": "PartitionedCall/model_4/tf.strided_slice_16/StridedSlice/strides", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT64", "tensorShape": {"dim": [{"size": "3"}]}}}, "dtype": {"type": "DT_INT64"}}}, {"name": "PartitionedCall/model_4/tf.reshape_3/Reshape/shape", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {"dim": [{"size": "4"}]}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "PartitionedCall/model_4/tf.compat.v1.transpose_4/transpose/perm", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {"dim": [{"size": "4"}]}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "PartitionedCall/model_4/tf.compat.v1.transpose_5/transpose/perm", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {"dim": [{"size": "4"}]}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "1318", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {"dim": [{"size": "1"}, {"size": "1"}, {"size": "16"}, {"size": "1"}]}}}, "dtype": {"type": "DT_FLOAT"}}}, {"name": "PartitionedCall/model_4/tf.compat.v1.transpose_8/transpose/perm", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {"dim": [{"size": "4"}]}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "PartitionedCall/model_4/tf.reshape_4/Reshape/shape", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {"dim": [{"size": "3"}]}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "1324", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT64", "tensorShape": {"dim": [{"size": "3"}]}}}, "dtype": {"type": "DT_INT64"}}}, {"name": "1326", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT64", "tensorShape": {"dim": [{"size": "3"}]}}}, "dtype": {"type": "DT_INT64"}}}, {"name": "PartitionedCall/model_4/tf.strided_slice_18/ones_like", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT64", "tensorShape": {"dim": [{"size": "3"}]}}}, "dtype": {"type": "DT_INT64"}}}, {"name": "PartitionedCall/model_4/tf.concat_17/concat/axis", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_INT32", "tensorShape": {}}}, "dtype": {"type": "DT_INT32"}}}, {"name": "PartitionedCall/model_4/tf.math.multiply_252/Mul/y", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {"dim": [{"size": "1"}, {"size": "1"}, {"size": "8400"}]}}}, "dtype": {"type": "DT_FLOAT"}}}, {"name": "1252", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {"dim": [{"size": "3"}, {"size": "3"}, {"size": "64"}, {"size": "64"}]}}}, "dtype": {"type": "DT_FLOAT"}}}, {"name": "1264", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {"dim": [{"size": "64"}]}}}, "dtype": {"type": "DT_FLOAT"}}}, {"name": "1276", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {"dim": [{"size": "3"}, {"size": "3"}, {"size": "64"}, {"size": "64"}]}}}, "dtype": {"type": "DT_FLOAT"}}}, {"name": "1288", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {"dim": [{"size": "64"}]}}}, "dtype": {"type": "DT_FLOAT"}}}, {"name": "1300", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {"dim": [{"size": "1"}, {"size": "1"}, {"size": "64"}, {"size": "64"}]}}}, "dtype": {"type": "DT_FLOAT"}}}, {"name": "1310", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {"dim": [{"size": "64"}]}}}, "dtype": {"type": "DT_FLOAT"}}}, {"name": "1250", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {"dim": [{"size": "3"}, {"size": "3"}, {"size": "64"}, {"size": "64"}]}}}, "dtype": {"type": "DT_FLOAT"}}}, {"name": "1262", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {"dim": [{"size": "64"}]}}}, "dtype": {"type": "DT_FLOAT"}}}, {"name": "1274", "op": "Const", "attr": {"value": {"tensor": {"dtype": "DT_FLOAT", "tensorShape": {"dim": [{"size": "3"}, {"size": "3"}, {"size": "64"}, {"size": "64"}]}}}, "dtype": {"type": "DT_FLOAT"}}}