Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Long camera update #810

Closed
Levi-Armstrong opened this issue Jan 22, 2023 · 17 comments
Closed

Long camera update #810

Levi-Armstrong opened this issue Jan 22, 2023 · 17 comments
Labels
bug Something isn't working
Projects
Core development

Comments

@Levi-Armstrong
Copy link
Contributor

Environment

  • OS Version: Ubuntu 20.04
  • Source or binary build?
    • binary, rendering6

image

Description

  • I have noticed that after adding a geometry that I am seeing long camera updates around 30 seconds and not sure how to resolve the issue. After the geometry is added I am able to change joint state and the visualization updated without any issues but as soon as I use the mouse to try and zoom or manipulate the camera it freezes for around 30 seconds. After the long update I have no issues manipulating the camera until new objects are added to the scene again. I attached a debugger and bellow is the back trace where it is at during this long pause. I have also tried turning of shadows and turning of anti-aliasing but it has no impact on the long camera update. I am not sure where to look next or what could be the issue. Also I load the same environment in RVIZ and do not have any issues.

I have tested this with both Ogre and Ogre2 rendering and they both have the same long camera update..

image
@Levi-Armstrong Levi-Armstrong added the bug Something isn't working label Jan 22, 2023
@osrf-triage osrf-triage added this to Inbox in Core development Jan 22, 2023
@Levi-Armstrong
Copy link
Contributor Author

I have found that if this object is not in the scene it responds as I would expect. Any thoughts on what might be causing the issues? Here is a link to the dae files.

image

@iche033
Copy link
Contributor

iche033 commented Jan 23, 2023

I assume you mean the GUI camera right and not a camera sensor? I don't see anything weird with these 2 dae meshes. I just did a quick test and created a model consisting of 2 visuals, one for each dae mesh (base_link.dae and link_1.dae) but I can't seem to reproduce the long camera update problem. I did the test in ign-rendering6 with both ogre and ogre2 render engines

@Levi-Armstrong
Copy link
Contributor Author

I will try just this link and see if I see an issue.

@Levi-Armstrong
Copy link
Contributor Author

Levi-Armstrong commented Jan 24, 2023 •
edited

It looks like if you only load the model independently they have less of an affect. Also the more models in the environment in addition to these two models increase the pause time.

  • Compliance device: 3 seconds
    image

  • Spindle: 6 seconds
    image

  • Compliance device + Spindle: 24 seconds

@iche033 Here is the spindle model which combined with the other model is causing the 24 second pause. It almost looks like it is loading the model again because it is around the same amount of time. Also these models are not water tight if that may have an affect.

Original Model (Compliance device)

Camera Update: 0.257443ms
Camera Update: 0.250413ms
Camera Update: 3405.7ms // Load model
Camera Update: 88.5545ms
Camera Update: 83.4895ms
Camera Update: 83.9671ms
Camera Update: 83.6664ms
Camera Update: 83.6583ms
Camera Update: 83.8001ms
Camera Update: 84.3852ms
Camera Update: 85.1221ms
Camera Update: 83.1592ms
Camera Update: 84.3725ms
Camera Update: 84.697ms
Camera Update: 84.8974ms
Camera Update: 84.6631ms
Camera Update: 84.5477ms
Camera Update: 3550.15ms // Click and drag
Camera Update: 95.3022ms
Camera Update: 91.2949ms

Just Spindle

Camera Update: 0.237799ms
Camera Update: 0.239147ms
Camera Update: 0.2653ms
Camera Update: 0.27567ms
Camera Update: 6331.76ms // Load Model
Camera Update: 130.058ms
Camera Update: 125.33ms
Camera Update: 126.79ms
Camera Update: 126.314ms
Camera Update: 125.869ms
Camera Update: 126.432ms
Camera Update: 127.742ms
Camera Update: 128.475ms
Camera Update: 127.231ms
Camera Update: 125.232ms
Camera Update: 6561.65ms // Click and drag
Camera Update: 140.579ms
Camera Update: 135.95ms

Original Model + Spindle

Camera Update: 0.275657ms
Camera Update: 0.24785ms
Camera Update: 22861.1ms // Load model
Camera Update: 226.377ms
Camera Update: 218.293ms
Camera Update: 217.807ms
Camera Update: 217.28ms
Camera Update: 217.498ms
Camera Update: 217.151ms
Camera Update: 215.762ms
Camera Update: 216.684ms
Camera Update: 218.78ms
Camera Update: 219.693ms
Camera Update: 218.943ms
Camera Update: 217.011ms
Camera Update: 216.935ms
Camera Update: 218.397ms
Camera Update: 216.872ms
Camera Update: 216.411ms
Camera Update: 23417.1ms // Click and drag
Camera Update: 246.824ms
Camera Update: 237.128ms

@Levi-Armstrong
Copy link
Contributor Author

Loading this environment below only takes 11ms and click drag takes 4ms.

image

Camera Update: 0.264061ms
Camera Update: 0.312692ms
Camera Update: 11.3222ms // Load model
Camera Update: 1.83305ms
Camera Update: 1.43624ms
Camera Update: 3.26565ms
Camera Update: 3.44781ms
Camera Update: 3.26614ms
Camera Update: 3.48336ms
Camera Update: 3.41699ms
Camera Update: 3.5079ms
Camera Update: 3.38546ms
Camera Update: 3.45223ms
Camera Update: 3.23643ms
Camera Update: 4.00466ms // Click and drag
Camera Update: 2.97609ms
Camera Update: 3.42399ms
Camera Update: 3.08402ms
Camera Update: 3.42905ms
Camera Update: 3.25529ms
Camera Update: 3.16946ms

@iche033
Copy link
Contributor

iche033 commented Jan 24, 2023

hmm I created a minimal example to test this but still haven't been able to reproduce the long pause and camera updates.

Can you give the following test world a try and see if you get the same issue?

ign gazebo -v 4 test_dae.sdf
test_dae.sdf 
<?xml version="1.0" ?>

<sdf version="1.6">
  <world name="test_dae">

    <gui fullscreen="0">

      <!-- 3D scene -->
      <plugin filename="MinimalScene" name="3D View">
        <ignition-gui>
          <title>3D View</title>
          <property type="bool" key="showTitleBar">false</property>
          <property type="string" key="state">docked</property>
        </ignition-gui>

        <engine>ogre2</engine>
        <scene>scene</scene>
        <ambient_light>0.4 0.4 0.4</ambient_light>
        <background_color>0.8 0.8 0.8</background_color>
        <camera_pose>-6 0 6 0 0.5 0</camera_pose>
        <camera_clip>
          <near>0.25</near>
          <far>10000</far>
        </camera_clip>
      </plugin>

      <!-- Plugins that add functionality to the scene -->
      <plugin filename="EntityContextMenuPlugin" name="Entity context menu">
        <ignition-gui>
          <property key="state" type="string">floating</property>
          <property key="width" type="double">5</property>
          <property key="height" type="double">5</property>
          <property key="showTitleBar" type="bool">false</property>
        </ignition-gui>
      </plugin>
      <plugin filename="GzSceneManager" name="Scene Manager">
        <ignition-gui>
          <property key="resizable" type="bool">false</property>
          <property key="width" type="double">5</property>
          <property key="height" type="double">5</property>
          <property key="state" type="string">floating</property>
          <property key="showTitleBar" type="bool">false</property>
        </ignition-gui>
      </plugin>
      <plugin filename="InteractiveViewControl" name="Interactive view control">
        <ignition-gui>
          <property key="resizable" type="bool">false</property>
          <property key="width" type="double">5</property>
          <property key="height" type="double">5</property>
          <property key="state" type="string">floating</property>
          <property key="showTitleBar" type="bool">false</property>
        </ignition-gui>
      </plugin>
      <plugin filename="CameraTracking" name="Camera Tracking">
        <ignition-gui>
          <property key="resizable" type="bool">false</property>
          <property key="width" type="double">5</property>
          <property key="height" type="double">5</property>
          <property key="state" type="string">floating</property>
          <property key="showTitleBar" type="bool">false</property>
        </ignition-gui>
      </plugin>
      <plugin filename="MarkerManager" name="Marker manager">
        <ignition-gui>
          <property key="resizable" type="bool">false</property>
          <property key="width" type="double">5</property>
          <property key="height" type="double">5</property>
          <property key="state" type="string">floating</property>
          <property key="showTitleBar" type="bool">false</property>
        </ignition-gui>
      </plugin>
      <plugin filename="SelectEntities" name="Select Entities">
        <ignition-gui>
          <anchors target="Select entities">
            <line own="right" target="right"/>
            <line own="top" target="top"/>
          </anchors>
          <property key="resizable" type="bool">false</property>
          <property key="width" type="double">5</property>
          <property key="height" type="double">5</property>
          <property key="state" type="string">floating</property>
          <property key="showTitleBar" type="bool">false</property>
        </ignition-gui>
      </plugin>
      <plugin filename="VisualizationCapabilities" name="Visualization Capabilities">
        <ignition-gui>
          <property key="resizable" type="bool">false</property>
          <property key="width" type="double">5</property>
          <property key="height" type="double">5</property>
          <property key="state" type="string">floating</property>
          <property key="showTitleBar" type="bool">false</property>
        </ignition-gui>
      </plugin>

      <plugin filename="Spawn" name="Spawn Entities">
        <ignition-gui>
          <anchors target="Select entities">
            <line own="right" target="right"/>
            <line own="top" target="top"/>
          </anchors>
          <property key="resizable" type="bool">false</property>
          <property key="width" type="double">5</property>
          <property key="height" type="double">5</property>
          <property key="state" type="string">floating</property>
          <property key="showTitleBar" type="bool">false</property>
        </ignition-gui>
      </plugin>

      <!-- World control -->
      <plugin filename="WorldControl" name="World control">
        <ignition-gui>
          <title>World control</title>
          <property type="bool" key="showTitleBar">false</property>
          <property type="bool" key="resizable">false</property>
          <property type="double" key="height">72</property>
          <property type="double" key="width">121</property>
          <property type="double" key="z">1</property>

          <property type="string" key="state">floating</property>
          <anchors target="3D View">
            <line own="left" target="left"/>
            <line own="bottom" target="bottom"/>
          </anchors>
        </ignition-gui>

        <play_pause>true</play_pause>
        <step>true</step>
        <start_paused>true</start_paused>
        <use_event>true</use_event>

      </plugin>

      <!-- World statistics -->
      <plugin filename="WorldStats" name="World stats">
        <ignition-gui>
          <title>World stats</title>
          <property type="bool" key="showTitleBar">false</property>
          <property type="bool" key="resizable">false</property>
          <property type="double" key="height">110</property>
          <property type="double" key="width">290</property>
          <property type="double" key="z">1</property>

          <property type="string" key="state">floating</property>
          <anchors target="3D View">
            <line own="right" target="right"/>
            <line own="bottom" target="bottom"/>
          </anchors>
        </ignition-gui>

        <sim_time>true</sim_time>
        <real_time>true</real_time>
        <real_time_factor>true</real_time_factor>
        <iterations>true</iterations>
      </plugin>

      <!-- Insert simple shapes -->
      <plugin filename="Shapes" name="Shapes">
        <ignition-gui>
          <property key="resizable" type="bool">false</property>
          <property key="x" type="double">0</property>
          <property key="y" type="double">0</property>
          <property key="width" type="double">250</property>
          <property key="height" type="double">50</property>
          <property key="state" type="string">floating</property>
          <property key="showTitleBar" type="bool">false</property>
          <property key="cardBackground" type="string">#666666</property>
        </ignition-gui>
      </plugin>

      <!-- Insert lights -->
      <plugin filename="Lights" name="Lights">
        <ignition-gui>
          <property key="resizable" type="bool">false</property>
          <property key="x" type="double">250</property>
          <property key="y" type="double">0</property>
          <property key="width" type="double">150</property>
          <property key="height" type="double">50</property>
          <property key="state" type="string">floating</property>
          <property key="showTitleBar" type="bool">false</property>
          <property key="cardBackground" type="string">#666666</property>
        </ignition-gui>
      </plugin>

      <!-- Translate / rotate -->
      <plugin filename="TransformControl" name="Transform control">
        <ignition-gui>
          <property key="resizable" type="bool">false</property>
          <property key="x" type="double">0</property>
          <property key="y" type="double">50</property>
          <property key="width" type="double">250</property>
          <property key="height" type="double">50</property>
          <property key="state" type="string">floating</property>
          <property key="showTitleBar" type="bool">false</property>
          <property key="cardBackground" type="string">#777777</property>
        </ignition-gui>

        <!-- disable legacy features used to connect this plugin to GzScene3D -->
        <legacy>false</legacy>
      </plugin>

      <!-- Screenshot -->
      <plugin filename="Screenshot" name="Screenshot">
        <ignition-gui>
          <property key="resizable" type="bool">false</property>
          <property key="x" type="double">250</property>
          <property key="y" type="double">50</property>
          <property key="width" type="double">50</property>
          <property key="height" type="double">50</property>
          <property key="state" type="string">floating</property>
          <property key="showTitleBar" type="bool">false</property>
          <property key="cardBackground" type="string">#777777</property>
        </ignition-gui>
      </plugin>

      <!-- Video recorder -->
      <plugin filename="VideoRecorder" name="VideoRecorder">
        <ignition-gui>
          <property key="resizable" type="bool">false</property>
          <property key="x" type="double">300</property>
          <property key="y" type="double">50</property>
          <property key="width" type="double">50</property>
          <property key="height" type="double">50</property>
          <property key="state" type="string">floating</property>
          <property key="showTitleBar" type="bool">false</property>
          <property key="cardBackground" type="string">#777777</property>
        </ignition-gui>

        <record_video>
          <use_sim_time>true</use_sim_time>
          <lockstep>true</lockstep>
          <bitrate>4000000</bitrate>
        </record_video>

        <!-- disable legacy features used to connect this plugin to GzScene3D -->
        <legacy>false</legacy>
      </plugin>

      <!-- Inspector -->
      <plugin filename="ComponentInspector" name="Component inspector">
        <ignition-gui>
          <property type="string" key="state">docked_collapsed</property>
        </ignition-gui>
      </plugin>

      <!-- Entity tree -->
      <plugin filename="EntityTree" name="Entity tree">
        <ignition-gui>
          <property type="string" key="state">docked_collapsed</property>
        </ignition-gui>
      </plugin>

      <!-- View angle -->
      <plugin filename="ViewAngle" name="View angle">
        <ignition-gui>
          <property type="string" key="state">docked_collapsed</property>
        </ignition-gui>

        <!-- disable legacy features used to connect this plugin to GzScene3D -->
        <legacy>false</legacy>
      </plugin>

    </gui>


    <scene>
      <ambient>1.0 1.0 1.0</ambient>
      <background>0.8 0.8 0.8</background>
    </scene>

    <light type="directional" name="sun">
      <cast_shadows>true</cast_shadows>
      <pose>0 0 10 0 0 0</pose>
      <diffuse>0.8 0.8 0.8 1</diffuse>
      <specular>0.2 0.2 0.2 1</specular>
      <attenuation>
        <range>1000</range>
        <constant>0.9</constant>
        <linear>0.01</linear>
        <quadratic>0.001</quadratic>
      </attenuation>
      <direction>-0.5 0.1 -0.9</direction>
    </light>

    <model name="ground_plane">
      <static>true</static>
      <link name="link">
        <collision name="collision">
          <geometry>
            <plane>
              <normal>0 0 1</normal>
              <size>3000 3000</size>
            </plane>
          </geometry>
        </collision>
        <visual name="visual">
          <geometry>
            <plane>
              <normal>0 0 1</normal>
              <size>3000 3000</size>
            </plane>
          </geometry>
          <material>
            <ambient>0.8 0.8 0.8 1</ambient>
            <diffuse>0.8 0.8 0.8 1</diffuse>
            <specular>0.8 0.8 0.8 1</specular>
          </material>
        </visual>
      </link>
    </model>

    <include>
      <static>true</static>
      <name>test_Dae</name>
      <pose>0 0 0 0 0 0</pose>
      <uri>https://fuel.gazebosim.org/1.0/iche033/models/test_dae</uri>
    </include>

  </world>
</sdf>

Launching the world will download this test_dae model which consists of the 3 daes that I got from your repo (compliance device and spindle; spindle's base_link.dae is renamed to spindle_base_link). Maybe I'm doing something different.

@Levi-Armstrong
Copy link
Contributor Author

I ran the command but it prints out the help info. Do I need to install something for this command to work?

The 'ign' command provides a command line interface to the ignition tools.

  ign <command> [options]

List of available commands:

  help:          Print this help text.
  gui:           Launch graphical interfaces.
  log:           Record or playback topics.
  topic:         Print information about topics.
  service:       Print information about services.
  sdf:           Utilities for SDF files.
  msg:           Print information about messages.
  fuel:          Manage simulation resources.
  plugin:        Print information about plugins.

Options:

  --force-version <VERSION>  Use a specific library version.
  --versions                 Show the available versions.
  --commands                 Show the available commands.
Use 'ign help <command>' to print help for a command.

@iche033
Copy link
Contributor

iche033 commented Jan 24, 2023

oh I just assumed you're running Fortress and have gazebo already installed. You'll need libignition-gazebo6-dev.
So you are using gz-rendering in your own custom application?

@Levi-Armstrong
Copy link
Contributor Author

Levi-Armstrong commented Jan 24, 2023 •
edited

I will give it a try. It looks like these models had a lot of objects. One had ~80 and the other had ~150. I went through and cleaned up each of the models and made them water tight where each one has around 8 objects in the dae. After cleaning things up everything loads fast and the update rate is around 10ms and the load takes around 85ms.

@Levi-Armstrong
Copy link
Contributor Author

After install I get the following error. I did a quick google search but nothing worked. I mostly use ign/gz libraries directly in my own application so not familiar with a lot of the tools.

[Wrn] [ign.cc:99] Fuel world download failed because Fetch failed. Other errors
Unable to find or download file

@Levi-Armstrong
Copy link
Contributor Author

So I tried downloading the model locally and navigating to the folder and running the following command, but the GUI pops up and then promptly closes.

~/Documents/test_dae$ ign gazebo -v 4 model.sdf 
[Msg] Ignition Gazebo GUI    v6.12.0
[Dbg] [ign.cc:161] Subscribing to [/gazebo/starting_world].
[Dbg] [ign.cc:163] Waiting for a world to be set from the GUI...
[Dbg] [Gui.cc:253] Waiting for subscribers to [/gazebo/starting_world]...
[Msg] Received world [model.sdf] from the GUI.
[Dbg] [ign.cc:167] Unsubscribing from [/gazebo/starting_world].
[Msg] Ignition Gazebo Server v6.12.0
[Msg] Loading SDF world file[/home/larmstrong/Documents/test_dae/model.sdf].
[Msg] Serving world names on [/gazebo/worlds]
[Msg] Resource path add service on [/gazebo/resource_paths/add].
[Msg] Resource path get service on [/gazebo/resource_paths/get].
[Msg] Resource path resolve service on [/gazebo/resource_paths/resolve].
[Msg] Resource paths published on [/gazebo/resource_paths].
[Msg] Server control service on [/server_control].
[Dbg] [ign.cc:410] Shutting down ign-gazebo-server
[Dbg] [SignalHandler.cc:141] Received signal[2].
[Dbg] [Application.cc:92] Initializing application.
[GUI] [Dbg] [Application.cc:555] Create main window
[GUI] [Dbg] [PathManager.cc:66] Requesting resource paths through [/gazebo/resource_paths/get]
[GUI] [Wrn] [Application.cc:797] [QT] file::/Gazebo/GazeboDrawer.qml:147:3: QML Dialog: Binding loop detected for property "implicitHeight"
[GUI] [Wrn] [Application.cc:797] [QT] file::/Gazebo/GazeboDrawer.qml:147:3: QML Dialog: Binding loop detected for property "implicitHeight"
[GUI] [Dbg] [Application.cc:140] Terminating application.

@Levi-Armstrong
Copy link
Contributor Author

Okay, after creating the file below and using this file to load everything worked. Using this I did not see a pause.

In my case I am not using gazebo or qml, but instead I have ported the qml render widget to Qt Widget here specifically the QOpenGLWidget. It currently is not running the update in another thread like how the qml component is doing so the update call is happening in the main gui thread which may be causing the issue. If you have time to take a look a look at the render widget let me know if you see any issues. My hope is once I have worked out the issues is create an example to add to gui repository.

<?xml version="1.0" ?>
<sdf version="1.6">
  <world name="test_world">
    <include>
      <static>true</static>
      <name>test</name>
      <pose>0 0 0 0 0 0</pose>
      <uri>https://fuel.gazebosim.org/1.0/iche033/models/test_dae</uri>
    </include>
  </world>
</sdf>

@Levi-Armstrong Levi-Armstrong mentioned this issue Jan 25, 2023
Closed
@iche033
Copy link
Contributor

iche033 commented Jan 25, 2023

I see. Having a separate rendering thread from gui is a good idea. Another thing to try is throttle camera updates to something like 60Hz.

Another minor optimization is that the screenToScene function (specifically the ClosestPoint function) can be expensive. I think we are unnecessary calling this function too often in gz-gui but we can't remove it as some gui plugins depend on it. If you don't need to broadcast hover events, you can try avoiding that call.

@Levi-Armstrong
Copy link
Contributor Author

I have made the update rate configurable with the default set to 60Hz. I am going to create a minimal example only using the API and no GUI to see if the update take a long time.

@azeey azeey moved this from Inbox to In progress in Core development Feb 6, 2023
@azeey
Copy link
Contributor

azeey commented May 31, 2024

From reading through the thread, this doesn't seem to be an issue with gz-rendering. I'll go ahead and close the issue, but feel free to reopen.

@azeey azeey closed this as not planned Won't fix, can't repro, duplicate, stale May 31, 2024
@Levi-Armstrong
Copy link
Contributor Author

I think there is still an issue with gz-rendering because the same mesh was not an issue when using rviz, but not sure what the difference is.

@azeey
Copy link
Contributor

azeey commented Jun 3, 2024

I was under the impression that it is working fine in Gazebo (ign gazebo or gz sim), but not in your application that uses QtWidget. Is it possible to reproduce the issue with just gz-rendering (without Qt) or in Gazebo? Otherwise, I'm not sure how we debug the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
Core development
Archived in project
Core development
In progress
Milestone
No milestone
Development

No branches or pull requests
3 participants
@azeey @iche033 @Levi-Armstrong