The codecs and media frameworks support meta-tracker

[HonkingGoose]

To all who read this issue report and thread:

I've migrated the tables to my repository at https://github.com/HonkingGoose/proton_codecs_media_support_tracker
If you want to update a table entry, you can open a pull request there. 😉
This way I don't need to keep track of changes all by myself.

---

Proton 5.13-1 has improvements for some games. Some quartz games may start working now.

Quote from @aeikum original comment here

There won't be any improvements for Media Foundation-based games in 5.13-1.

We put in a bunch of work improving quartz video between 5.0 and 5.13, so some games may start working (I don't have specific titles, sorry). If you see a color bars pattern appearing now, that's an indicator of our work on certain codecs. You will hopefully see the real video instead of those color bars very soon.

---

Feature Request

I confirm:

• that I haven't found another request for this feature. Technically there are multiple requests, but they are scattered between multiple issue reports.
• that I have checked whether there are updates for my system available that
  contain this feature already.

Description

There are multiple games which use .wmv (Windows Media Video) files, for their cinematics/cutscenes. These files will not be played back in-game on a standard installation of the game using Proton.

Because I've seen multiple bug-reports with regards to .wmv file playback, I wanted to make a generic request, so it doesn't get lost in between all the specific game issues.

Justification

There are multiple games which have .wmv file playback issues, these games use .wmv for cutscenes, and so are a integral part of the story and experience of the games which use it.

For #137 the fix for .wmv files would probably make the game ready for official whitelist process.
There might be other games which would qualify for whitelisting too, after this feature has been implemented.

Integration into Proton would fix .wmv file playback for the games referenced below.

Risks

• The fix for .wmv file playback might break something else in Proton.
• There might be some other unknown risks associated with including the .wmv file playback, like for example patent/IP issues regarding including .wmv codecs/playback.

References

The old tables can be seen from the edit history. I recommend you use the newer tables in my repository though.

[ghost]

The problem is that there are games like Obduction that need Windows Media Foundation/Windows Media Feature support as they use a dll not installed otherwise by a wmv player like WMP. #327

So its not just being able to play a video, they literally need this dll (and whatever else is tied to it) to playback the video. UE4 games all have the option of needing this dll if it is used in a Windows release for video playback. Possibly Unity games as well.

Although, as long as some version of it can be found by the game (and used), it could even just be a mock one that uses a different means.

While, some games seem to be able to get by with WMP installed as far as I can tell from various posts on the internet, it would be better to go the whole mile and support more than just simple wmv format. WMV is a part of a whole pie.

[HonkingGoose]

@byte1024 Oh, I didn't know that more might be needed beside just fixing "simple" .wmv file playback.

I understand your point correctly:

1. Some games also require Windows Media Foundation and/or Windows Media Feature support.
2. Those games use a dll (and all that is tied to this dll) which is not installed by a wmv player like WMP.
3. Therefore just fixing .wmv playback will not solve the whole problem games have with playback.
4. So it's better to implement support for the whole "Windows Media Foundation and/or Windows Media Feature" framework.

[ghost]

@byte1024 Oh, I didn't know that more might be needed beside just fixing "simple" .wmv file playback.

I understand your point correctly:

1. Some games also require Windows Media Foundation and/or Windows Media Feature support.

2. Those games use a dll (and all that is tied to this dll) which is not installed by a wmv player like WMP.

3. Therefore just fixing .wmv playback will not solve the whole problem games have with playback.

4. So it's better to implement support for the whole "Windows Media Foundation and/or Windows Media Feature" framework.

It looks like there might even be more. The only log I found in all the links you posted (here: #137) needs quartz.dll to find system codecs and uses fmod for in-game sound.

The log for that one shows some stuff used in quartz.dll can't be found:

41489.934:0008:0032:err:quartz:GetClassMediaFile Media class not found
41489.935:0008:0032:err:quartz:GetClassMediaFile Media class not found
41489.936:0008:0032:err:quartz:GetClassMediaFile Media class not found

That's all I noticed in that log, but its easier to find this stuff when you know the filename of the video being played since a search goes right to it if its found in the log.

And an explanation that Wine's quartz.dll is fake and missing stuff.
https://www.winehq.org/pipermail/wine-users/2002-November/009217.html

And the corresponding log entry shows this is still true:
41489.932:0008:0032:trace:module:load_dll L"C:\\windows\\system32\\quartz.dll" is a fake Wine dll

Some seem to say quartz is a part of the WMP install, but quartz is also DirectShow (DirectX media) which was replaced with WMF that I mentioned. https://en.wikipedia.org/wiki/DirectShow

and Microsoft also mentions WMF in the what's new section of DirectShow: https://docs.microsoft.com/en-us/windows/desktop/directshow/whats-new-in-directshow

Its a media pie, all these pieces are a part of it, but Wine has been failing on them and now Proton is too.

[aeikum]

Wine has a directshow implementation, which depends on gstreamer. Since we can't depend on the system having gstreamer installed, we will need to build and ship it with Proton. Worse, we also would need to ship the codecs, which may have patent/licensing issues. It hasn't been looked into yet.

Wine does not have a MF implementation yet. There's some work being done on it, but it's not nearly ready. If anyone is looking to contribute to Wine, that may be a decent place to get started.

[niikoo]

+1

I have tried a lot of different configurations.
Haven't had any success with codec packs / wmp.
VLC in Proton plays those videos though, so it should be possible.

Tried Proton 3.16-4 beta and 3.7-8
Graphics: AMD RX580

[HonkingGoose]

@niikoo Thanks for trying things out. 😄

I must confess, my understanding of this issue is very limited. @aeikum knows way more about this than me... So @aeikum, could you perhaps give your thoughts on this?

[basxto]

VLC has it’s own codecs, afaik.

[PythonicChemist]

+1

it affects all Blazblue games. I tried manually installing quartz into their prefixes which doesn't work with a proton prefix but with a normal wine one.

[f0rmatme]

this also effects the new RE2

[kisak-valve]

Hello @james-munson, the libraries you've shared are not free and it's problematic to redistribute them.

[ipr]

#issuecomment-1046181344

I think it's a matter of just different build with different configuration. When I see something with different sizes the questions that pop-up:

1. are they both 32- or 64-bit?
2. is debug-information stripped? (means: reduced or removed)
3. what options were used to build them?

Third point is important: there are tons of different configuration options of what is included and what is not, features that are supported and those that are not. Unlike closed-source programs they can be tailored simply by different build options to a large degree. They aren't monolithic pieces at all these days. Maybe they are just stub-libraries with the bare minimum included, I haven't looked.

Anyway, there's Gstreamer stable 1.20 out and Proton uses 1.18 currently. Also ffmpeg has 5.0 out now. Hopefully Proton is updated soon.

[ClearlyClaire]

I've noticed since a few weeks that I've started seeing videos instead of the placeholder… except these videos have wrong colors and are heavily distorted (seems like the pixel format and line order are wrong?); I'm not sure this is the best place to report it, but I haven't found a better place nor any existing report of that issue, although I doubt I'm the only one experiencing it.

Proton Experimental and Proton 7.0-1 (earlier versions do not seem to load the videos at all and get stuck instead)

[aeikum]

@ClearlyClaire In general, that should be reported on the issue for each game, but it would be convenient if you listed the affected titles here, too.

[ClearlyClaire]

I haven't kept track of everything, but I recently tried at least Cook-out (in the animated sections of the tutorial) and Haven (the introduction movie) that had this issue. I expected this to be a generic issue with Proton/Steam Play but it indeed seems to work correctly for other games (Satisfactory) that previously displayed the test card pattern for embedded videos.

EDIT: Turns out it's actually reported already for Haven: #4433 (comment)

[aeikum]

@ClearlyClaire Thank you. We'll take a look at Cook-out and Haven and try to get those working.

[ryester19]

Should we ask for a Github label to be created that will help point out the games that are still having video playback issues?

[aeikum]

IMO it's not needed. We have no shortage of stuff to work on here, and I think the labels would fall out of date quickly with how much work we are putting into this area.

[Bitwolfies]

Is there anyway on our end to check if games have serverside transcoded videos? With proton 7.0 MMZX collection now actually displays the test pattern, but no videos will actually play. Reinstalled the game a few times but nothing, no idea if it's just not transcoded or there's another issue.

[ghost]

@Bitwolfies

Reinstalled the game a few times but nothing, no idea if it's just not transcoded or there's another issue.

Reinstalling the game will do nothing but waste bandwidth.

The thing that gets rid of the test pattern and makes the videos start working is when Steam says there's an update for the game and it's actually a new transcoded_video.foz file.

When will that happen? No one knows, possibly a few days, at some point after a player interacts with a cutscene and sees the test pattern, Valve's server will eventually get around to it.

[ryester19]

Using the bleeding edge branch of Proton Experimental, I now see the test pattern during the opening of Atelier Rorona DX 🎉These movies are WMV files, btw

On a different note: Does Valve scan game depots to look for video files that need transcoding?

[CSahajdacny]

Using the bleeding edge branch of Proton Experimental, I now see the test pattern during the opening of Atelier Rorona DX tadaThese movies are WMV files, btw

On a different note: Does Valve scan game depots to look for video files that need transcoding?

That will be the right method, but sadly no...

[ryester19]

Using the bleeding edge branch of Proton Experimental, I now see the test pattern during the opening of Atelier Rorona DX tadaThese movies are WMV files, btw
On a different note: Does Valve scan game depots to look for video files that need transcoding?

That will be the right method, but sadly no...

Does Proton need extra data from "discovering" these videos in-game, before it can be transcoded and used successfully? If not, then there's going to be occasional problems with day 1 players running into unconverted cutscenes that will probably hamper their experience

[Bitwolfies]

Anyway for us to generate these offline? The code is in the repo but I don't really see a way to run it, any way we can do it?

[basxto]

For games that allow to view all discovered cut scenes, you can at least load a 100% savegame and go through all cut scenes before you start to play.

[CSahajdacny]

Tiny intro videos of Chronos: Before The Ashes still don't works...
They have audio, but no video; just the TV test pattern.
steam-967390.log

Something is still wrong with Valve's version of GStramer.

[basxto]

They have audio, but no video; just the TV test pattern.

The pattern is to be expected if it’s not converted yet. Sounds like video and sound were separate from each other. And the queue is slow, I added (or tried to) some videos three weeks ago and they are still not converted yet.

[Elmapul]

Guilty Gear X2 #Reload (314030)

the opening video was fixed, it can be watched as soon as you open the game, or in the gallery screen, but there is an video on gallery called Extra, that show the colors bars:

https://raw.githubusercontent.com/Elmapul/proton-good-metric/main/Captura%20de%20tela%20de%202022-05-20%2003-52-14.png

as you can see the color bars dont fill in the whole screen, i suppose they should?

waiting a few seconds on the title screen do work (activate some presentation between 2 characters fighting.)

the game is slow, im pretty sure that my computer can run it faster than that (even on protons, an old version was able to start the fights but not to play the videos) i will try to close a few windows to see if i can get an better performance or maybe its a bug on the new version of proton.

btw: waiting on the buged video (colors bars) may crash the game.

i forgot to mention, before you open the game, you have to chose if you want to open the game or configure the settings, the configure settings windows isnt completely rendered, and you cant resize, you cant read what is writen on the right side of this window.
https://raw.githubusercontent.com/Elmapul/proton-good-metric/main/guilty.png

[alkaris2]

This seems to be a problem for Spyro Reignited Trilogy. MP4 video formats don't play back, it just skips over them. Something related to mfplat.dll

#3035

[steventylerseanherr]

Ditto on Spyro having problems
Out of the box reignited wouldn't play cutscenes at all
I had to reenable shader precaching and download like 2gb of shaders
After that cutscenes would play
Or so I thought
All the cutscenes through Spyro 1 worked till the very last cutscenes
Where it played that color test for a bit then skipped the cutscenes

This game really should not be deck verified if it doesn't work out of the box yet

[Elmapul]

Guilty Gear X2 #Reload (314030)
had another regression.
the cutscenes arent playing anymore.
honestly i dont remember when they were playing (what version of proton i was using, i will have to guess based on proton relase dates+the time i tested) but now they arent playing.
tested on experimental and proton 4.11-13, the other versions i think it wont even open (7.0-4 take forever to load , i dont know if it will ever open or if something crashed)

[kisak-valve]

Hello @FanderWasTaken, your comment was removed because your workaround references a legally problematic workaround.

[fedor4ever]

Do you have demo which play sounded video at start? In various formats: wmv+wma pro, wmw+wma etc. It was very useful at automated testing.