Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Hold for 4.2] add pseudo oblique function to CMD 206 CAM_TRIGG_DIST #9103

Draft
wants to merge 83 commits into
base: master
Choose a base branch
from

Conversation

igorsgcampos
Copy link

Problem
A solution commonly used to increase the coverage and efficiency in mapping surveys is to have multiple cameras at oblique angles to capture a wider areas per flight line, this also has potential benefits of capturing more data from facades and tall objects. However, adding cameras increase cost, weight and complexity of the equipment, on the other hand, mapping cameras are generally capable of shooting multiple frames per second and are usually triggered about once every two seconds or less frequently in mapping missions. We could better use the hardware capabilities if we moved the camera mount between shots while decreasing the shooting distance to keep the overlap settings.

Solution
I decided to implement a solution that uses the already existing MAV_CMD_DO_SET_CAM_TRIGG_DIST, plus some unused parameters, 4~6, to issue MAV_CMD_DO_MOUNT_CONTROL between camera shots to move the camera from left to right in stages, right after it passes half the shooting distance, allowing time for mount actuation and camera exposure. To keep the frontlap the solution expects the GCS to adjust the trigger distance accordingly. The parameters usage are as follows:

4: number of oblique positions (this will generally be 2 or 3, but could be more, depending on desired frontlap, flying speed and hardware capabilities (how fast are the camera and mount))
5: the maximum angle to roll the camera left and right from the center, the solution calculates the interval angle between positions from this information and the previous parameter
6: the pitch angle for mounts that support pitching

Implementation
I proposed the change to mavlink standard here: mavlink/mavlink#1508
I implemented the proposed solution here: PX4/PX4-Autopilot#15882

Alternative
Other way would be to add a new command, but since it has many aspects that are essentially the same as the CAM_TRIGG_DIST, I thought it would be better to add a few parameters to the existing command, as it would also allow it to work under Mavlink 1 with the limited number of 255 commands.

Test data / coverage
I tested it with the gazebo_typhoon_h480 model and checked the outcome from the video stream in QGC.

Additional context
Later I plan to write a paper comparing the gains in area coverage and any accuracy change from using this solution with relation to the traditional survey mode. I am feeling confident about a positive outcome.

@DonLakeFlyer
Copy link
Contributor

Later I plan to write a paper comparing the gains in area coverage and any accuracy change from using this solution with relation to the traditional survey mode. I am feeling confident about a positive outcome.

To me that needs to be the first step to prove this will actually work. With specific details on how things like front and sidelap need to be calculated based on settings.

@Antiheavy FYI

@Antiheavy
Copy link

@Antiheavy FYI

seems ok. or at least, this shouldn't negatively impact the way we use the camera trigger commands.

@igorsgcampos
Copy link
Author

build will break until mavlink submodule is updated with the changes from mavlink/mavlink#1508

@DonLakeFlyer DonLakeFlyer changed the title add pseudo oblique function to CMD 206 CAM_TRIGG_DIST [Hold for 4.2] add pseudo oblique function to CMD 206 CAM_TRIGG_DIST Nov 20, 2020
@DonLakeFlyer DonLakeFlyer added this to the Release V4.2 milestone Nov 20, 2020
@igorsgcampos igorsgcampos marked this pull request as ready for review December 3, 2020 01:32
@DonLakeFlyer
Copy link
Contributor

I don't know if you've rebased to master recently. But of not my recent changes to support terrain adjustment in turnaround segments has likely caused conflicts with this code. Mainly because the building of mission items had to be reworked. It should be fairly straightforward to understand how the new code works. There wasn't any other way to fix this problem without the change. Sorry about that.

@igorsgcampos
Copy link
Author

igorsgcampos commented Dec 19, 2020

I don't know if you've rebased to master recently. But of not my recent changes to support terrain adjustment in turnaround segments has likely caused conflicts with this code. Mainly because the building of mission items had to be reworked. It should be fairly straightforward to understand how the new code works. There wasn't any other way to fix this problem without the change. Sorry about that.

No problem Don, already rebased/fixed my commits. Thanks for the heads-up!

I also appended to the surveys DO_MOUNT_CONTROL commands to point the gimbal down and forward before and after the survey, as this had to be manually done. If the user forgot about this, it would end up with a survey facing forward.

@igorsgcampos
Copy link
Author

@DonLakeFlyer one more question, when I build this with the target to Android apk, the file ends up being ~42MB, whereas the latest daily is ~53MB. Plus, my build does not get the solo video link, despite the fact that it is already rebased onto master, whereas the latest daily does get it successfully.

Having the camera feed would help seeing the camera behavior, but I can either have the feed or the oblique survey command. Could you point me to instructions on how to get this properly working on my android build?

@DonLakeFlyer
Copy link
Contributor

Sounds like you may not have android gstreamer installer: https://github.com/mavlink/qgroundcontrol/tree/master/src/VideoReceiver

@igorsgcampos
Copy link
Author

igorsgcampos commented Dec 19, 2020

I followed the instrucitons, but I must have some conflicting software (maybe ROS, it likes to give headaches):

The following packages have unmet dependencies:
gstreamer1.0-python3-dbg-plugin-loader : Conflicts: gstreamer1.0-python3-plugin-loader but 1.14.5-0ubuntu1~18.04.1 is to be installed
gstreamer1.0-python3-plugin-loader : Conflicts: gstreamer1.0-python3-dbg-plugin-loader but 1.14.5-0ubuntu1~18.04.1 is to be installed
E: Unable to correct problems, you have held broken packages.

@DonLakeFlyer
Copy link
Contributor

Qmake output should tell you whether it is building with or without gstreamer support.

@DonLakeFlyer
Copy link
Contributor

Qmake output should tell you whether it is building with or without gstreamer support. Any linux complaints should be irrelevant I believe since the package is for android not linux.

@igorsgcampos
Copy link
Author

Actually, I noticed it was a conflict between 2 packages on the list and kept the dbg one, it installed successfully, but the apk build still has no video, is there any config to tell the compiler to build with video?

@igorsgcampos
Copy link
Author

nevermind, I noticed additional instructions for android at the bottom, I will follow those instead of the linux ones.

@igorsgcampos
Copy link
Author

Cleaning and rebuilding seemed to solve it! Thanks for the directions!

@igorsgcampos
Copy link
Author

Damn, I'm having a hard time fixing these tests.

@DonLakeFlyer
Copy link
Contributor

FYI: I haven't look at what is failing. But there is currently some sort of race condition in master VehicleLinkManagerTest which can cause it to fail intermittently. Haven't figured it out yet.

@igorsgcampos
Copy link
Author

It is likely due to this:

I also appended to the surveys DO_MOUNT_CONTROL commands to point the gimbal down and forward before and after the survey, as this had to be manually done. If the user forgot about this, it would end up with a survey facing forward.

The mission is having an unexpected number of waypoints or having the wrong waypoint types.

@DonLakeFlyer
Copy link
Contributor

If you add a survey to a Plan the gimbal will already be pointed down since the Mission Start will be modified to do this when the survey gets added. Why isn't that good enough?

@igorsgcampos
Copy link
Author

igorsgcampos commented Dec 23, 2020

If you add a survey to a Plan the gimbal will already be pointed down since the Mission Start will be modified to do this when the survey gets added. Why isn't that good enough?

It wasn't behaving like this when I tested a couple months ago with PX4 in SITL and more recently when I flight tested on Solo. It kept the camera pointing at 0,0,0 orientation, which faces forward normally. The one at the end is needed to ensure that the camera is at 0 degrees roll at the end with this new command. I noticed the need for it when I landed the Solo and the gimbal touched the ground because it was at 20 degrees roll left.

patrickelectric and others added 28 commits October 21, 2022 10:54
Update jurplel/install-qt-action to latest version

Signed-off-by: Patrick José Pereira <patrickelectric@gmail.com>
Signed-off-by: Patrick José Pereira <patrickelectric@gmail.com>
Signed-off-by: Patrick José Pereira <patrickelectric@gmail.com>
"QNmeaPositionInfoSource supports reporting the accuracy of the
horizontal and vertical position. To enable position accuracy reporting
an estimate of the User Equivalent Range Error associated with the NMEA
source must be set with setUserEquivalentRangeError."

Source: https://doc.qt.io/qt-6/qnmeapositioninfosource.html#details

Commit 2a9ecab introduced new behaviour
that broke use of external GPS. This change fixes the problem reported
in mavlink#10271.
People wants to usue fullscreen without some text on the mididle of the screen,
10 seconds is enough to get the message.

Signed-off-by: Patrick José Pereira <patrickelectric@gmail.com>
Signed-off-by: Patrick José Pereira <patrickelectric@gmail.com>
* custom-example: Update from GIT_VERSION to new APP_VERSION_STR

Signed-off-by: Patrick José Pereira <patrickelectric@gmail.com>

* cmake: Fix usage of new variable APP_VERSION_STR over GIT_VERSION

Signed-off-by: Patrick José Pereira <patrickelectric@gmail.com>
Co-authored-by: Luiz Kressin <luiz.kressin@gmail.com>
Github workflow image ubuntu-20.04 version 20230313.1 have added Android
SDK Platform android-33-ext5. Something, possible the gradle version
bundled with qt does not parse this correctly, and sets
androidCompileSdkVersion in android-build/gradle.properties to "ext5"
instead of expected "33". This commit is a quick fix to the problem, so
the build will revert to android-33 which was used before 20230109.1.

This fix was previously added for android-33-ext4 in image 20230109.1 in
commit 6f636e7.
This fix a warning message in clang 11.
Moves from C struct definition to C++ standard style.

Signed-off-by: Patrick José Pereira <patrickelectric@gmail.com>
Add Kakute H7 v2 & Kakute H7 Mini to QGC FirmwareUpgradeController.cc
Ability to digest climb speed and descent speed with mavlink command MAV_CMD_DO_CHANGE_SPEED
add firmware upgrade support for the peach k1/r1 boards
…k#10509)

* PlanManager: drop spurious items for unexpected mission types
* PlanManager: drive-by bug fix
Releasing a hat button did not lead to _executeButtonAction being called with BUTTON_UP. This adds the hat buttons to lastBbuttonValues, such that the button-up transitions for hat buttons are recorded.
Signed-off-by: Patrick José Pereira <patrickelectric@gmail.com>
@julianoes
Copy link
Contributor

Looks like the feature went into MAVLink and PX4 but not here yet. @igorsgcampos any chance you want to resurrect the PR?

@HTRamsey HTRamsey marked this pull request as draft April 12, 2024 12:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet