rough draft of scene qa script #1971
Draft
+1,485
−0
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
jturner65
force-pushed
the
qa-scenes
branch
3 times, most recently
from
a0a3104
to
2d9499b
Compare
jturner65
force-pushed
the
qa-scenes
branch
4 times, most recently
from
457d833
to
d1b3789
Compare
jturner65
force-pushed
the
qa-scenes
branch
3 times, most recently
from
91e4d2a
to
f7be4e2
Compare
jturner65
force-pushed
the
qa-scenes
branch
2 times, most recently
from
0ece367
to
c915125
Compare
…ead of specifying on the command line when running the qa_scene.py script. Also added colored print statements to make the output easier to read. Also added a qa_scene_utils.py file to separate the important processing code from helper functions
…Application so that we can test how fast the rendering is
…abstracted some big chunks of scene/stage processing code into functions, and created a new qa scene json config file
…ate unique filenames for output files
…ofile_scene() from inside the iteratively_test_all_scenes() function. Added ability to specify initial transform of agent. Started writing code to generate a default scene if none exists in the dataset config file
…u specify in the config file. You can press 'c' to watch the collision test occur, or if you don't run the app with the viewer on, it will run all the tests specified in the config file you want to test. Refactoring some of the functions in qa_scenes.py, and abstracting some code snippets into separate methods bc some of the functions were getting long and difficult to read. Also added code to time how long the collision test takes at each position
…er times to the comprehensive json
…over a given number of seconds. It saves the results in the comprehensive json file
…object sleep test
…aries for each scene
…till work when some of the tests aren't run, e.g. if we are running the render test, but not the collision nor sleep tests
…Now runs on both stages and scenes
…nd path before it
…some config files and updated the old ones
…better crash tracking
…ig options for reduced metric output. Minor cleanup.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Introduces a tool for automated SceneDataset performance metric analysis.
NOTE : Requires colorama library
To use the tool:
data/directory.tools/qa_scenes/configs/overriding default values fromtools/qa_scenes/qa_scene_settings.pypython tools/qa_scenes/qa_scenes.py --config_file_path path/to/my.qa_scene_config.jsonResults will print to terminal and images/json/csv will output to
tools/qa_scenes/qa_scenes_output/Features:
tools/qa_scenes/qa_scene_settings.pyHow Has This Been Tested
Locally
Types of changes
Checklist