Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filter] Allow dynamic framerate changes #4078

Open
wants to merge 53 commits into
base: main
Choose a base branch
from

Conversation

tschulz
Copy link
Contributor

@tschulz tschulz commented May 11, 2023

One might want to dynamically change the framerate at any time while the pipeline is running. This is prevented by code in tensor_converter and tensor_filter that, upon reconfigure events, require the new caps to be 'compatible' with the original caps on the sink pads. Requiring the caps to have matching framerates seems unnecessary and too restrictive.

This patch removes that restriction from both tensor_converter and tensor_filter elements.

A new unit test for this case has been added.

tschulz and others added 25 commits February 9, 2023 16:11
…on found for /usr/local/lib/libtensorflow.so.2 '
…on found for /usr/local/lib/libtensorflow.so.2 '
** (process:615338): CRITICAL **: 07:37:24.833: Exception while loading the model: Legacy model format is not supported on mobile.
… == true.

This fixes: "RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!"
Allow tensor_filter to be configured with smaller number of output tensors than PyTorch returns natively.

Signed-off-by: Torsten Schulz <torsten@eyepop.ai>
  * Non-maintainer upload.
  * Allow PyTorch filter to be configured with less output tensors than PyTorch natively returns
One might want to dynamically change the framerate at any time while the
pipeline is running. This is prevented by code in `tensor_converter` and
`tensor_filter` that, upon reconfigure events, requre the new caps to be
'compatible' with the orignal caps on the sink pads. Requiring the caps
to have matching framerates seems unnecessary and too restrictive.

This patch removes that restriction from both `tensor_converter` and
`tensor_filter` elements.

A new unit test for this case has been added.

Signed-off-by: Torsten Schulz <torsten@eyepop.ai>
[Filter] Allow dynamic framerate changes
@taos-ci
Copy link
Collaborator

taos-ci commented May 11, 2023

📝 TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #4078. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/.

@taos-ci
Copy link
Collaborator

taos-ci commented May 11, 2023

:octocat: cibot: @eyepop-ai, tests/nnstreamer_filter_rate_change/unittest_filter_rate_change.cc does not include Doxygen tags such as @file @brief @author @bug. You must include the Doxygen tags in the source code. Please refer to a Doxygen manual at http://github.com/nnstreamer/TAOS-CI/blob/main/ci/doc/doxygen-documentation.md

@tschulz tschulz marked this pull request as ready for review May 11, 2023 20:48
@tschulz tschulz requested review from a team and myungjoo as code owners May 11, 2023 20:48
@tschulz tschulz requested a review from yeonykim2 as a code owner March 19, 2024 01:07
@tschulz tschulz changed the title [DRAFT] [Filter] Allow dynamic framerate changes @open sesame 05/27 00:15 [Filter] Allow dynamic framerate changes @open sesame 05/27 00:15 Mar 19, 2024
@myungjoo myungjoo changed the title [Filter] Allow dynamic framerate changes @open sesame 05/27 00:15 [Filter] Allow dynamic framerate changes Mar 28, 2024
@myungjoo
Copy link
Member

 7/18 unittest_filter_rate_change RUNNING       
>>> NNSTREAMER_CONVERTERS=/home/runner/work/nnstreamer/nnstreamer/build/ext/nnstreamer/tensor_converter NNSTREAMER_FILTERS=/home/runner/work/nnstreamer/nnstreamer/build/ext/nnstreamer/tensor_filter NNSTREAMER_SOURCE_ROOT_PATH=/home/runner/work/nnstreamer/nnstreamer NNSTREAMER_BUILD_ROOT_PATH=/home/runner/work/nnstreamer/nnstreamer/build GST_PLUGIN_PATH=/home/runner/work/nnstreamer/nnstreamer/build/gst:/home/runner/work/nnstreamer/nnstreamer/build/ext NNSTREAMER_DECODERS=/home/runner/work/nnstreamer/nnstreamer/build/ext/nnstreamer/tensor_decoder ASAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1 MALLOC_PERTURB_=155 UBSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 LD_LIBRARY_PATH=/home/runner/work/nnstreamer/nnstreamer/build/tests:/home/runner/work/nnstreamer/nnstreamer/build/gst/nnstreamer NNSTREAMER_CONF=/home/runner/work/nnstreamer/nnstreamer/build/nnstreamer-test.ini NNSTREAMER_TRAINERS=/home/runner/work/nnstreamer/nnstreamer/build/ext/nnstreamer/tensor_trainer /home/runner/work/nnstreamer/nnstreamer/build/tests/unittest_filter_rate_change

 7/18 unittest_filter_rate_change FAIL             0.31s   killed by signal 6 SIGABRT
――――――――――――――――――――――――――――――――――――― ✀  ―――――――――――――――――――――――――――――――――――――
stdout:
[==========] Running 3 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 3 tests from NNSFilterRateChangeTest
[ RUN      ] NNSFilterRateChangeTest.passthrough
[       OK ] NNSFilterRateChangeTest.passthrough (105 ms)
[ RUN      ] NNSFilterRateChangeTest.static_throttle
[       OK ] NNSFilterRateChangeTest.static_throttle (102 ms)
[ RUN      ] NNSFilterRateChangeTest.throttling_dynamic_change_dec
Bail out! ERROR:../gst/nnstreamer/tensor_filter/tensor_filter.c:1121:gst_tensor_filter_configure_tensor: assertion failed: (gst_tensors_config_is_equal (&priv->in_config, &in_config))
stderr:
**
ERROR:../gst/nnstreamer/tensor_filter/tensor_filter.c:1121:gst_tensor_filter_configure_tensor: assertion failed: (gst_tensors_config_is_equal (&priv->in_config, &in_config))
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

@@ -3,19 +3,20 @@ Section: libs
Priority: optional
Maintainer: MyungJoo Ham <myungjoo.ham@samsung.com>
Build-Depends: gcc-9 | gcc-8 | gcc-7 | gcc-6 | gcc-5 (>=5.4),
<<<<<<< HEAD
ninja-build, meson (>=0.50), debhelper (>=9),
=======
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not merged properly.

@@ -1,14 +1,82 @@
nnstreamer (2.4.1.0) unstable bionic focal jammy sid; urgency=medium
nnstreamer (2.3.0.0+ep1) UNRELEASED; urgency=medium
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not merged properly. Overwritten by older version descriptions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove this empty file.


third_party/package_tflite/
third_party/build_tflite/
third_party/embedded_root/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you really need these?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't remove these files. We have flatbuf/protobuf users.

@@ -681,6 +681,7 @@ gst_join_request_new_pad (GstElement * element, GstPadTemplate * templ,
return sinkpad;
}


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Meaningless change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants