Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FFmpeg: Update FFmpeg.AutoGen to 7.0.0 #6256

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

FFmpeg: Update FFmpeg.AutoGen to 7.0.0 #6256

wants to merge 4 commits into from

Conversation

FreezyLemon
Copy link
Contributor

@FreezyLemon FreezyLemon commented Apr 19, 2024 •
edited

Closes #5051. There is still a VP9 bug (discussed in that issue), but there hasn't been any movement upstream so I assume this is not going to be fixed. I have added a rudimentary (untested) workaround for that issue in this PR.

Prerequisite:

FFmpeg.AutoGen had a large API change somewhere around 5.0 or 5.1.

The gist is that ffmpeg.GetOrLoadLibrary was removed in favour of the IFunctionResolver API. The default resolvers are not usable in o!f1, so I added a simple FFmpegFunctionResolver class.

Note:

Feel free to start testing / giving feedback, but this still needs a bit of work, and a lot of testing.

Testing:

  • macOS (x64)
  • macOS (arm64)
  • macOS (Rosetta)
  • Windows (x64)
  • Linux (x64)
  • iOS (arm64)
  • iOS simulator (x86_64)

Footnotes

  1. The built-in LibraryDependenciesMap is wrong for the custom libraries used in o!f. It also requires fiddling with the RootPath property and calls system APis (dlopen etc.) directly. Using NativeLibrary instead should be more ergonomic. ↩

@FreezyLemon FreezyLemon mentioned this pull request Apr 19, 2024
Open
@smoogipoo
Copy link
Contributor

👍 macOS (x64):

master:
[runtime] 2024-04-22 08:39:04 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_VIDEOTOOLBOX for codec h264
[runtime] 2024-04-22 08:39:05 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_VIDEOTOOLBOX for codec hevc

pr:
[runtime] 2024-04-22 08:40:03 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_VIDEOTOOLBOX for codec h264
[runtime] 2024-04-22 08:40:04 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_VIDEOTOOLBOX for codec hevc

👍 macOS (Rosetta):

master:
[runtime] 2024-04-22 08:30:59 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_VIDEOTOOLBOX for codec h264
[runtime] 2024-04-22 08:31:00 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_VIDEOTOOLBOX for codec hevc

pr:
[runtime] 2024-04-22 08:33:51 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_VIDEOTOOLBOX for codec h264
[runtime] 2024-04-22 08:33:53 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_VIDEOTOOLBOX for codec hevc
[runtime] 2024-04-22 08:33:55 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_VIDEOTOOLBOX for codec vp9

👍 macOS (arm64):

master:
[runtime] 2024-04-22 08:36:22 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_VIDEOTOOLBOX for codec h264
[runtime] 2024-04-22 08:36:23 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_VIDEOTOOLBOX for codec hevc

pr:
[runtime] 2024-04-22 08:35:05 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_VIDEOTOOLBOX for codec h264
[runtime] 2024-04-22 08:35:06 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_VIDEOTOOLBOX for codec hevc
[runtime] 2024-04-22 08:35:09 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_VIDEOTOOLBOX for codec vp9

Looks about as expected. Looks like OSArchitecture is working under Rosetta, though no one should be running o!f under Rosetta.

@smoogipoo
Copy link
Contributor

👍 Windows (x64):

master:
[runtime] 2024-04-22 08:43:38 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_D3D11VA for codec h264
[runtime] 2024-04-22 08:43:39 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_D3D11VA for codec hevc
[runtime] 2024-04-22 08:43:41 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_D3D11VA for codec vp9

pr:
[runtime] 2024-04-22 08:46:24 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_D3D11VA for codec h264
[runtime] 2024-04-22 08:46:26 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_D3D11VA for codec hevc
[runtime] 2024-04-22 08:46:28 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_D3D11VA for codec vp9

@smoogipoo
Copy link
Contributor

smoogipoo commented Apr 22, 2024 •
edited

No hardware decoders load for me on Linux (either master or PR) but that's likely a configuration issue my side (or ffmpeg compile things™)

@smoogipoo
Copy link
Contributor

Should FFmpegFunctionResolver support bringing a different set of libraries?

Let's not.

Is the OSArchitecture check correct?

Going by my above testing it seems to be fine as is. Haven't tested VM but that's a pretty esoteric use case that we'll cross if it ever comes up. None of our testing considers Rosetta right now, so I'd say it's also irrelevant (even though I tested it).

Maybe move setting the FunctionResolver into a static constructor for FFmpegFuncs.

It may be safer, but I also don't expect users to be using FFmpeg directly themselves.

@smoogipoo
Copy link
Contributor

Is this ready to be merged? You mention it needs a bit of work, but it seems to be working alright.

@FreezyLemon
Copy link
Contributor Author

No hardware decoders load for me on Linux (either master or PR) but that's likely a configuration issue my side (or ffmpeg compile things™)

This is expected (see #6025).

Is this ready to be merged? You mention it needs a bit of work, but it seems to be working alright.

That's good, but I have a feeling this will break video playback on mobile platforms (those still bundle old, pre-5.0 FFmpeg libraries which might cause binary incompatibilities).

Otherwise, I feel pretty good about the state of the PR. I'll probably do that one thing with the FFmpegFuncs static constructor and then take a look at Android and iOS

@FreezyLemon
Copy link
Contributor Author

This should be fine now. I thought the mobile platforms might need some changes on the .NET side, but it looks like they'll work just fine as long as the native libraries follow the same conventions.

@FreezyLemon FreezyLemon marked this pull request as ready for review April 23, 2024 15:57
@frenzibyte
Copy link
Member

👍 iOS (arm64):

master:
2024-05-15 02:28:28.733 osu.Framework.Tests[1894:494188] [runtime] 2024-05-14 23:28:28 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_VIDEOTOOLBOX for codec h264
2024-05-15 02:28:28.734 osu.Framework.Tests[1894:494188] [runtime] 2024-05-14 23:28:28 [verbose]: Successfully initialized decoder: h264

pr:
2024-05-15 02:24:22.766 osu.Framework.Tests[1883:490066] [runtime] 2024-05-14 23:24:22 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_VIDEOTOOLBOX for codec h264
2024-05-15 02:24:22.766 osu.Framework.Tests[1883:490066] [runtime] 2024-05-14 23:24:22 [verbose]: Successfully initialized decoder: h264

👍 iOS simulator (x86_64):

master:
2024-05-15 02:31:08.698145+0300 osu.Framework.Tests[60243:6304306] [runtime] 2024-05-14 23:31:08 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_VIDEOTOOLBOX for codec h264
2024-05-15 02:31:08.701010+0300 osu.Framework.Tests[60243:6304306] [runtime] 2024-05-14 23:31:08 [verbose]: Successfully initialized decoder: h264
[h264 @ 0x7fa502d35400] Failed setup for format videotoolbox_vld: hwaccel initialisation returned error.

pr:
2024-05-15 02:34:44.993640+0300 osu.Framework.Tests[64562:6328122] [runtime] 2024-05-14 23:34:44 [verbose]: Successfully opened hardware video decoder context AV_HWDEVICE_TYPE_VIDEOTOOLBOX for codec h264
2024-05-15 02:34:44.994314+0300 osu.Framework.Tests[64562:6328122] [runtime] 2024-05-14 23:34:44 [verbose]: Successfully initialized decoder: h264
[h264 @ 0x7fd4da7199c0] Failed setup for format videotoolbox_vld: hwaccel initialisation returned error.

(hardware acceleration never "works" on iOS simulator so ignoring)

@FreezyLemon
Copy link
Contributor Author

FreezyLemon commented May 15, 2024 •
edited

I just noticed that you removed the old iOS libraries from this branch, @frenzibyte. I would've preferred doing that in the auto-generated PR that will be created after #6255 is merged and new binaries are built (example of this PR: #6227). Android has old libraries too that should be removed, and I would like to keep it separated.

Do you mind if we remove that change from this PR and include it in the binaries update?

@frenzibyte
Copy link
Member

Sure, I pushed it here because I found there's no reason not to, as this PR is already meant to be merged after the FFmpeg libraries update.

@FreezyLemon
Copy link
Contributor Author

Fair enough. It's just my preference to keep it separated, so that merging only one PR would not result in having double or no library files. Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@frenzibyte frenzibyte frenzibyte requested changes

Requested changes must be addressed to merge this pull request.

Assignees
No one assigned
Labels
area:libraries area:video size/L
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Upgrade FFmpeg libraries to 5.0
3 participants
@FreezyLemon @smoogipoo @frenzibyte