Latest FFMPEG version is incompatible

[rbwabd]

I was installing FFME for first time today and encountered an issue with linking to the correct version of ffmpeg. Just wanted to share my solution here.

I was trying to make the sample application work and no matter what I did it wouldn't recognize the path where ffmpeg was installed. After digging into the FFME code it appears that the version of the ffmpeg library files is hard coded in FFME (FFmpeg\FFLibrary.cs, line 68 public static FFLibrary LibAVCodec { get; } = new FFLibrary(Names.AVCodec, 58, 1); etc. For example here the 58 is the version of the library)

So with current version of FFME what you need to install is version 4.4 of ffmpeg. There is a more recent version of ffmpeg that has been released that will not work because all the library files have different version numbers.

Specifically, you want to install the full GPL (LGPL won't do because 2 dlls would be missing) shared version of ffmpeg 4.4, e.g. ffmpeg-n4.4-latest-win64-gpl-shared-4.4.zip, which you can find at https://github.com/BtbN/FFmpeg-Builds/releases

Note that I also tried the more expedient way of changing the library numbers directly in FFlibrary.cs. Unfortunately it then raises another error where it says it can't load the dll file. Am pretty sure the version number is hard coded a second time somewhere else in the code and one would need to go change that as well. At that point I couldn't bother and gave up trying to update FFME (also thinking about the possibility there is other code in FFME that may be incompatible with new ffmpeg version) and just went and found the correct ffmpeg version to use with this.

assume the FFME devs will update the code here at some point, but in meantime hope this helps someone.

[mprevot]

You need to update also FFmpeg.AutoGen to the latest version which supports ffmpeg 6 (to date). But then it introduces few breaking changes on FFME. I'm trying to make this work and maybe try a pull request.

[zgabi]

You need to update also FFmpeg.AutoGen to the latest version which supports ffmpeg 6 (to date). But then it introduces few breaking changes on FFME. I'm trying to make this work and maybe try a pull request.

I think you should fork this project and create a new nuget package since it seems that this package is not maintained anymore.

[mprevot]

I managed to make it work with net7 and ffmpeg6 (and using an updated autoffmpeg package). I could indeed make a nuget package, but I'm not sure I'll have time to maintain, support and expand what I did.

[MrBean2016]

I have updated to support for ffmpeg 5.1.2, .Net 6 and some of the deadlock-fixes in a fork.
No Nuget-packet, just clone the repo, build and then reference the ffme.win.dll in your project (or the whole repo)

https://github.com/MrBean2016/ffmediaelement
https://github.com/GyanD/codexffmpeg/releases/download/5.1.2/ffmpeg-5.1.2-full_build-shared.zip

[zgabi]

I've updated it to 6.0.0.0
package: https://www.nuget.org/packages/zgabi.FFME.Windows
source: https://github.com/zgabi/ffmediaelement
compatible ffmpeg for example: https://www.gyan.dev/ffmpeg/builds/ffmpeg-release-full-shared.7z

[MrBean2016]

Great work zgabi! I saw that you included the dead lock fixes as well. I test it right away

[jpwkeeper]

I can't get custom media sources to work in this version. When you open, you get one single read, then an Access Violation in avformat_open_input.

[zgabi]

Could you please createa a small project to repoduce the issue?

[jpwkeeper]

I'm reproducing it using the sample app with no modifications. If I play a file just using the path, it works. If I use the customfile:// prefix, it reads one buffer from my custom source, then crashes.

[zgabi]

Was it working in version 4.x in this way? It looks stange... Personally I don't thinkt that it should do anything when you put for example the customfile://C:\1\1.mkv string to the open box...

I also have a custom source, and it works with the following class:

internal unsafe class Stream1 : IMediaInputStream
{
    private Stream _stream;

    public Stream1(Stream stream)
    {
        _stream = stream;
    }
    public void Dispose()
    {
    }

    public int Read(void* opaque, byte* targetBuffer, int targetBufferLength)
    {
        return _stream.Read(new Span<byte>(targetBuffer, targetBufferLength));
    }

    public long Seek(void* opaque, long offset, int whence)
    {
        if (whence == FFmpeg.AutoGen.ffmpeg.AVSEEK_SIZE)
        {
            return _stream.Length;
        }

        return _stream.Seek(offset, (SeekOrigin)whence);
    }

    public Uri StreamUri => new Uri("customfile://temp.mp4");

    public bool CanSeek => true;

    public int ReadBufferLength => 4096;

    public InputStreamInitializing OnInitializing { get; }

    public InputStreamInitialized OnInitialized { get; }
}

[jpwkeeper]

I don't know if it worked in 4.4; I'm new to this project. I came here to try to find a way to stream video from a custom source, which is oddly hard to find.
customfile:// causes it to use FileInputStream, which is provided with the sample.

[zgabi]

Ahh, so this is a hack in the sample app... OK, I found it:

ffmediaelement/Unosquare.FFME.Windows.Sample/AppCommands.cs

Lines 55 to 58 in c7fe5ab

	if (target.ToString().StartsWith(FileInputStream.Scheme, StringComparison.OrdinalIgnoreCase))
	await m.Open(new FileInputStream(target.LocalPath));
	else
	await m.Open(target);

I don't know why is it not working. For me it works with the class I mentioned above.

[jpwkeeper]

That class is not working for me either. Perhaps something wrong in the sample with how it gets initialized with a custom source?

[zgabi]

It seems that it works when the stream is not seeakable (CanSeek = false)
Maybe depends on the video stream... if it wants to seek or not.

[jpwkeeper]

That did it. I'm OK with that; my custom stream won't be able to seek. Thanks for the help.

[jpwkeeper]

Curiously, it's still seeking, a lot, even though CanSeek is set to false.

[zgabi]

Yes.... Now I remember, probably this is an FFmpeg.AutoGen issue:
Ruslan-B/FFmpeg.AutoGen#255

So just remove the following 3 lines (2 + 1 empty) and it will work:

                        // Set the seekable flag based on the custom input stream implementation
                        CustomInputStreamContext->seekable = CustomInputStream.CanSeek ? 1 : 0;

Since it is unnecessary to set the seekable manually because the avio_alloc_context() sets it when the function is not null. dubiousconst282 wrote this in the linked issue, I havem't tried it, but it seems to be true, since as you also mentioned, the Seek function is still called.

If you need it I can create a new zgabi.FFME.Windows nuget package with this fix.

[jpwkeeper]

That depends. What are the ramifications of it thinking I can seek but I really can't?

[zgabi]

You can seek, since the seekable is set to 1 internally when the seek function is not null... the "mapping" is wrong in FFmpeg.Autogen.

[jpwkeeper]

But I need to set Seek to null, and there doesn't appear to be a convenient way to do that. My source won't be able to seek.

Having CanSeek null the function pointer would fix the issue in the avio_alloc_context call.

[zgabi]

I added this change and it seems to wokr:

[zgabi]

@MrBean2016 I saw that you reverted the allignment bindings... I created my from from your fork, so I also have them. should I also remove it? I plan to create a new package with the seek fix.

Edit: I found your comment at PR #632 , so I also removed it.

[MrBean2016]

@MrBean2016 I saw that you reverted the allignment bindings... I created my from from your fork, so I also have them. should I also remove it? I plan to create a new package with the seek fix.

Edit: I found your comment at PR #632 , so I also removed it.

Great work!

[SundayGoGo]

@zgabi hi zgabi ,I used the above configuration to reduce the playback delay, but after playing for a while, the delay will become higher and higher. Is it a configuration parameter issue? How to solve this problem?

[seiim-master]

@zgabi
Hello, I am using your library very well Thank you.
But when I see the video, I use a slider, but why do I see symptoms of lag on the screen? It does not move as the slider moves. It lag a little less when you move slowly, and it lag video when you scroll fast

[zgabi]

@SundayGoGo @seiim-master I think those problems are not related to the ffmpeg upgrade.... were they working with version 4.x? I only updtated this library to ffmpeg 6.0, so I don't know too much about the internals. Of course if you find a solution, I'll accept a PR and integrate to my nuget package version.

[seiim-master]

@zgabi
Thank you so much for your answer.
I don't stand out when scrollingI found the library It was a library called flyleaf, but it's a winform hosting method, so I can't use it easily because of a zorder problem.
I don't know much about the video player, so I don't know where to refer to even after looking at the code.
If possible, could you please refer to the flyleaf library and improve the ffme library to make the navigation smoother?

https://github.com/SuRGeoNix/Flyleaf

[zgabi]

I think it would not be easy. I would rather not add an external reference. The problem shoud be fixed in this library, but I don't know how. Otherwiser if I refer the Flyleaf, it will use Winforms as you mentioned. I'd like to avoid it.

[SundayGoGo]

@zgabi Thank you for your reply, it really has nothing to do with the ffmpeg version.
Let me research whether it has something to do with ffmpeg parameter configuration

[MrBean2016]

@zgabi Hello, I am using your library very well Thank you. But when I see the video, I use a slider, but why do I see symptoms of lag on the screen? It does not move as the slider moves. It lag a little less when you move slowly, and it lag video when you scroll fast

It probably depends on several things. Are you trying to achieve smooth scrubbing?
You can try to set VideoBlockCache to a high number (the file will be cached in memory but the cost is longer loading time) and also UseParallelDecoding and UseParallelRendering. Different formats has different pro/cons if smooth scrubbing and also the hardware specs.

[MrBean2016]

@zgabi Thank you so much for your answer. I don't stand out when scrollingI found the library It was a library called flyleaf, but it's a winform hosting method, so I can't use it easily because of a zorder problem. I don't know much about the video player, so I don't know where to refer to even after looking at the code. If possible, could you please refer to the flyleaf library and improve the ffme library to make the navigation smoother?

https://github.com/SuRGeoNix/Flyleaf

I don´t know of your z-order problem and maybe it is a bit OT but place the winformhost in a grid and specify the z-order. I suppose you know that the elements are rendered in the order they are declared if not the z-order is specified.

<Grid  Panel.ZIndex="1000">
      <WindowsFormsHost>
                <:PictureBox >
...
                </PictureBox>
            </WindowsFormsHost>
</Grid>

And If you think another library solves what you need, use it that instead! But investigate this first, only forward scrubbing is supported for now
SuRGeoNix/Flyleaf#270 (comment)

[mprevot]

@zgabi Hello, I am using your library very well Thank you. But when I see the video, I use a slider, but why do I see symptoms of lag on the screen? It does not move as the slider moves. It lag a little less when you move slowly, and it lag video when you scroll fast

It probably depends on several things. Are you trying to achieve smooth scrubbing? You can try to set VideoBlockCache to a high number (the file will be cached in memory but the cost is longer loading time) and also UseParallelDecoding and UseParallelRendering. Different formats has different pro/cons if smooth scrubbing and also the hardware specs.

All this conversation is off topic and should be moved to another issue.

[seiim-master]

@zgabi Thank you so much for your answer. I don't stand out when scrollingI found the library It was a library called flyleaf, but it's a winform hosting method, so I can't use it easily because of a zorder problem. I don't know much about the video player, so I don't know where to refer to even after looking at the code. If possible, could you please refer to the flyleaf library and improve the ffme library to make the navigation smoother?
https://github.com/SuRGeoNix/Flyleaf

I don´t know of your z-order problem and maybe it is a bit OT but place the winformhost in a grid and specify the z-order. I suppose you know that the elements are rendered in the order they are declared if not the z-order is specified.

<Grid  Panel.ZIndex="1000">
      <WindowsFormsHost>
                <:PictureBox >
...
                </PictureBox>
            </WindowsFormsHost>
</Grid>

And If you think another library solves what you need, use it that instead! But investigate this first, only forward scrubbing is supported for now SuRGeoNix/Flyleaf#270 (comment)

Thank you for your advice. The zorder problem has been a method of using the existing FFME library and putting usercontrol on it, but when using flyleaf, the basic control cannot be put on the playback screen, and the zorder problem can be solved only by using the window.
This is known as a unique constraint of wpf's winformhost.
Please explain again where you can find the VideoBlockCache option or UseParallelDecoding option that you mentioned.

[Mitra-M]

version 6.0 hangs randomly. (Heap memory corruption)

[MrBean2016]

@zgabi Thank you so much for your answer. I don't stand out when scrollingI found the library It was a library called flyleaf, but it's a winform hosting method, so I can't use it easily because of a zorder problem. I don't know much about the video player, so I don't know where to refer to even after looking at the code. If possible, could you please refer to the flyleaf library and improve the ffme library to make the navigation smoother?
https://github.com/SuRGeoNix/Flyleaf

I don´t know of your z-order problem and maybe it is a bit OT but place the winformhost in a grid and specify the z-order. I suppose you know that the elements are rendered in the order they are declared if not the z-order is specified.

<Grid  Panel.ZIndex="1000">
      <WindowsFormsHost>
                <:PictureBox >
...
                </PictureBox>
            </WindowsFormsHost>
</Grid>

And If you think another library solves what you need, use it that instead! But investigate this first, only forward scrubbing is supported for now SuRGeoNix/Flyleaf#270 (comment)

Thank you for your advice. The zorder problem has been a method of using the existing FFME library and putting usercontrol on it, but when using flyleaf, the basic control cannot be put on the playback screen, and the zorder problem can be solved only by using the window. This is known as a unique constraint of wpf's winformhost. Please explain again where you can find the VideoBlockCache option or UseParallelDecoding option that you mentioned.

They are found in /Common/MediaOptions.cs
Implement something like the code in MediaOpening (sample app MainWindow.MediaEvents.cs)

              e.Options.VideoBlockCache = 200;
              e.Options.UseParallelDecoding = true;
              e.Options.UseParallelRendering = false;

You will also see how hardware acceleration is configured.

[MrBean2016]

version 6.0 hangs randomly. (Heap memory corruption)

You need to give more information, how to reproduce, sample code that generates the error. Post this as a new issue.

[insel-maz]

zgabi.FFME.Windows@6.0.2 crashes after some Open() as does the original version FFME.Windows@4.4.350.
I am using .NET 8 and ffmpeg-n6.1-latest-win64-gpl-shared-6.1.zip.

I am already trying to approach the matter very carefully.

                    if (mediaElement.IsPlaying)
                        await mediaElement.Stop();
                    await Task.Delay(2000);
                    if (mediaElement.IsOpen)
                        await mediaElement.Close();
                    await Task.Delay(2000);

                    Debug.WriteLine(string.Format("Opening file \"{0}\"...", file));
                    await mediaElement.Open(new Uri(file));

[zgabi]

Could you please create a sample console application which is repdoducing the issue. Inclusing the media file (or the url if it is available online)
Btw: I think you should try ffmpeg 6.0, not 6.1, since my package is using that version. Maybe I'll update it to 7.0 if it seems stable :)

[insel-maz]

If I add a call to the garbage collector, I can reproduce the crash. The first song plays without any problems, the second song crashes.

        private async void PlayFiles()
        {
            var files = Directory.GetFiles(@"music\Cool Instrumentals", "*.mp3");
            foreach (var file in files)
            {
                var fullFile = Path.GetFullPath(file);
                var uri = new Uri(fullFile);

                Debug.WriteLine(string.Format("Playing {0}...", fullFile));
                await MediaElement.Open(uri);

                await Task.Delay(200);
                await MediaElement.Play();

                await Task.Delay(10_000);

                await MediaElement.Stop();
                await Task.Delay(200);
                await MediaElement.Close();
                await Task.Delay(200);

                GC.Collect();
            }
        }

[zgabi]

I cannot reproduce the crash:

It worked until I closed the window.
So probably it is a problm with your native ffmpeg dlls.. please use version 6.0.

I'm using this dlls: https://goliat.eik.bme.hu/~zgabi/ffmpeg.x64.7z
It does not contain any virus, but you should download it from some official site to be sure :) Then compare the files :)

[insel-maz]

Hey zgabi, I have forked the FFmpeg-Builds to build the version 4.4 and the versions 6.0 and 6.1.
If you would like, I would be happy if you could try the dlls from https://github.com/PinJuke/FFmpeg-Builds/releases (built by Github Actions). The link you provided results in a time out.

Are you on .NET 8.0?

[zgabi]

Ok, I try them.

My link... hmm.. I don't know what happened with that server (my old university is the provider)... now it works only with http for some reason,... http://goliat.eik.bme.hu/~zgabi/ffmpeg.x64.7z

Yes, I'm on .net 8.0

I tried them:
The 6.x versions are crashing
The version 4.4 has different dll versions (filenames, e.g avcodec-58.dll vs. avcodec-60.dll) so it does not start playing.

[insel-maz]

Hey zgabi, I have continued to test on it.
The binaries in your mentioned archive are identical to https://github.com/GyanD/codexffmpeg/releases/download/6.0/ffmpeg-6.0-full_build-shared.zip. (I have Get-FileHashed them.) It seems to be more stable. Thanks so far.

[vertigra]

Ok, I try them.

My link... hmm.. I don't know what happened with that server (my old university is the provider)... now it works only with http for some reason,... http://goliat.eik.bme.hu/~zgabi/ffmpeg.x64.7z

Yes, I'm on .net 8.0

I tried them: The 6.x versions are crashing The version 4.4 has different dll versions (filenames, e.g avcodec-58.dll vs. avcodec-60.dll) so it does not start playing.

Hi!
This should simplify the installation https://www.nuget.org/packages/FFmpeg.zgabi.FFME.Windows.6.0.2/1.0.13#readme-body-tab