"Load State Automatically" work less well than manual load state

[schmurtzm]

Hi, I know that states are not perfect in mame2003-plus as it is indicated as "game-dependent" in libretro documentation. But it seems not normal to have a very big difference of behavior between an automatic load state and a manual load state in Retroarch.

Problem description :
"Load State Automatically" work less well than manual load state

How to reproduce :
The 194x games are really a good test panel 😄 :

• When using "Load State Automatically" in Retroarch, the save state is restored directly when the rom is loaded :

  • 1941 resume without sound (just an horrible tone)
  • 1942 resume correctly
  • 1943 resume without image : Only some entries like scores but no sprites or background (but sound OK).

• If I disable "Load State Automatically" in retroarch, load my game and then do a manual resume, all this 3 games resume correctly.

Remark : when the state is loaded automatically and failed, you can select "restart" in retroarch and then manually load the state to make it work.

I have tested with a fresh retroarch compiled and the very last version of mame2003-plus too. I reproduce this behaviour on the Miyoo Mini (little Linux handheld console) and on PC / Windows.

The issue is probably old because I think that it is the same behavior described in this issue for mame2003 : libretro/mame2003-libretro#72

Logs
Here some verbose debug logs from auto-resume state with the 3 games: RA debug logs - auto resume state - 194x .zip

@markwkidd Seems to think that it could come from Retroarch and not from the core. Let met know if you need that I create this issue on Retroarch repo.

Thank you !

[MistyDreams]

this is a feature I dont use, if you can give me the steps to reproduce. I can try see whats going on seems if manual works auto should surely.

[schmurtzm]

Yes it should have the same behavior but it's not the case.

Reproduce is quite simple :

1 - run retroarch and go to settings -> saving and activate auto save state and load state automatically
2 - load the rom 1943 (generally I just drag and drop the rom in RA windows or use command line)
3 - quit retroarch (it will automatically generate a state called 1943.state.auto )
4 - launch retroarch and load 1943 again (do not use history menu otherwise it doesn't restore the state)
Now you see that it resumes without image : only some entries like scores but no sprites or background (but sound OK).
5 - choose the option Restart in retroarch menu and then Load state -> the previous state is now working when you do it manually.

Thank you for your support :)

[schmurtzm]

PS: the -e arg allows to load a specific state. This command line triggers the same problem as load state automatically.

To use it, rename your save state like this : 1943.state1.entry and call it like this :
retroarch.exe -L "cores\mame2003_plus_libretro.dll" -e 1 "arcade_test\1943.zip"

[schmurtzm]

To illustrate :
a failed automatic load state :

a manual load state (obtained with the same save state file) :

[MistyDreams]

Might take a minute to look at this will need to check manual vs auto and if they both SERIALIZATION or the core functions. This will require looking at the RA code.

[schmurtzm]

You make me dream because I know that a lot of persons are annoyed by this problem since a long time 😄

[schmurtzm]

in this area?
https://github.com/libretro/RetroArch/blob/60030e373e955e9a4cc86ad0186e35a14477a6f8/command.c#L1257

[MistyDreams]

not really seems to get the info from the core.info file rather than the core. The code needs traced to see what its doing to see why one is failing and the other isint. Its quite bizarre this is set from a external file and not the core itself.

[MistyDreams]

@mahoneyt944 @arcadez2003 do you guys know how ra supports savestate is it serialization or core based for both auto and manual?

[mahoneyt944]

Admittedly save states is something I've ignored for awhile since it's functionally is spotty in 2003+. But I believe the issue is because the machine needs to load before the state is applied or you'll get this kind of behavior. Maybe we could flag when the machine is "ready" and then allow the autostate to be applied

[schmurtzm]

Admittedly save states is something I've ignored for awhile since it's functionally is spotty in 2003+. But I believe the issue is because the machine needs to load before the state is applied or you'll get this kind of behavior. Maybe we could flag when the machine is "ready" and then allow the autostate to be applied

Thanks for your investigations 😀 !
You are very probably right because I've just made a new test : load the 1943 rom and press F4 (frantically) to load manually the last state quicker as possible .... and the issue comes back again !
So yes it is definitively related to the fact that the core is loading state before having terminate to load the rom.

[barbudreadmon]

But I believe the issue is because the machine needs to load before the state is applied or you'll get this kind of behavior. Maybe we could flag when the machine is "ready" and then allow the autostate to be applied

I believe automatic load is calling retro_unserialize before any call to retro_run has been made, you'd need to either leave retro_load after the machine is fully ready to accept savestates, or to improve savestate support to the point they will work even if the machine has yet to run a single cycle. FWIW, FBNeo is doing the later, and this is also needed for libretro features like runahead or netplay.

[MistyDreams]

Ill need to look the libretro.h I dont think we have any control over that initialisation order of the order of retro_load/unload. The front end allows cores to do very little flexibility between cores and itself you would require communication to do that. @mahoneyt944 how would you flag auto_load per say for example? Some games do work with rewind funnily enough very odd will need to look through the libretro docs to see how the condition if serialisation is ready or not is sent to the front end.

For this testing im going to have to redo the mame078 fixes to compile in msys2 need to know if load states are working. There was a memory addressing in the 68k as well when I done it last time but we need a debugger in some cases to see what going on with the core like with the system16 issues for the sound not working. I do think its interrupt related though need to look back into it as the 68k was having issues think its due to 64/32 bit issues.

[MistyDreams]

https://www.libretro.com/index.php/develop/ the step by step last link is missing from the docs suppose ill need to look at the ra source to figure this out either. I cant see the implementation details of serialisation documents anywhere or just cant find them.

[markwkidd]

MistyDreams there is a more current version of that doc at this location: https://docs.libretro.com/development/frontends/

…

On Fri, Sep 2, 2022 at 8:48 AM MistyDreams ***@***.***> wrote: https://www.libretro.com/index.php/develop/ the step by step last link is missing from the docs suppose ill need to look at the ra source to figure this out either. I cant see the implementation details of serialisation documents anywhere or just cant find them. — Reply to this email directly, view it on GitHub <#1433 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEVGC5QW5HAMSFPCYJT44KLV4HZQ3ANCNFSM6AAAAAAQCR2JZA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Mark W. Kidd (he/him/his) http://facebook.com/markwkidd (606)536-0115

[MistyDreams]

@markwkidd I have searched this and in older versions of RA the calls are made however something has changed on the condition of these fallbacks.

this is the only relevant info I can see.

https://docs.libretro.com/development/cores/developing-cores/#retro_serialize_size-retro_serialize-and-retro_unserialize

retro_serialize_size(), retro_serialize(), and retro_unserialize() do not log any output on log_cb it is possible this is too early for the logging system to be implemented though so i also tried printf to no avail to these functions being called.

all i get is
[ERROR] Rewind unavailable because this core lacks serialized save state support. Ill need to trace this to find the condition as it documented as retro_serialize_size() should be called.

I also added
savestate = "true"
savestate_features = "serialized"

to the .info file as per
https://github.com/libretro/RetroArch/blob/6226d0442f324d1c6adc9e2e87684d45a5edc900/core_info.c#L1846-L1872

[MistyDreams]

Ok seem to be making progress after looking at the code its a core info cache issue. I just turned it off at Settings > Core > Cache Core Info Files

savestate = "true"
savestate_features = "serialized"

will need to be added for the .info file for plus. start 1943 rewind seems to work for this game. The try the auto loads again.

some work needs done on this neogeo(if used too soon) and flying shark have issues

[schmurtzm]

Hi again, thanks again for your investigation :)

I have added the 2 config parameters in mame2003_plus_libretro.info and disabled "Cache Core Info Files" but I still have the same texture problem when I autoload 1943.

Here my RA config file + info core file + core override config :
RA.zip

[MistyDreams]

test rewind an see if that works for you also make sure any previous saves where deleted.

if reind does work

change
memory_descriptors = "false"
libretro_saves = "false"

as true.

if rewind is working i dont see why auto load isint but the could be initialization issues. At least we are at a point we can start debugging it

[MistyDreams]

pacman seems to have fewer issues still has some though still have to check if auto load uses serialization though

[schmurtzm]

change
memory_descriptors = "false"
libretro_saves = "false"

Where ? In retroarch.cfg ?

I tried, rewind works but I still have the same result 😞

[MistyDreams]

its ok I tested myself get the same result for 1943. The file was the .info file again, Right now i need to get the original mame 078 to compile and fix the issues i had last time. This will let me know if our driver save states are good so i can debug RA end of things.

[schmurtzm]

I tested myself get the same result for 1943.

Same result as me ? You mean that you have the black screen too ?
This 078 debugging is a little esoteric for me but do not hesitate if I can help for something 😉

[MistyDreams]

yea black screenwith scores and energy bar. The rest will take a while but it could be worth looking into

[schmurtzm]

Great that you reproduce.
Yes, I can't imagine the complexity of states loading in RA with Mame2003+ as core 😅 I'll be patient 😉

[MistyDreams]

I did test netplay for giggles and ironically it does work you just need to add the analog descriptors. You would need to sacrifice the flexible input system for if conditions to set maps per game like fbneo(libretro port only) does.

Only problem is these remaps don't translate well to anything that isint a gampad layout. Again its a libretro port so they can do as they please but I wouldn't be interested in pursuing it. I have megadrive 6 button pads and arcade sticks so that not something I personally would want for the sake of netplay. Ill let you know if I make any progress on this issue, don't expect it too soon though depends on work and free time.

[schmurtzm]

Making netplay on this core is a real feat 😅

Ill let you know if I make any progress on this issue, don't expect it too soon though depends on work and free time.

That's OK, I'm already happy that you take a look on this issue. For now we have released our new version OS for Miyoo Mini with auto save state disabled for arcade. If one day you have a great news it will be a good surprise for the numerous arcade gamers 😉 Thank you !

[MistyDreams]

You should consider fbalpha if fbneo is too slow for older hardware(struggles on quiet a bit unless you have a pi 4). I believe netplay and autosave works for that core. If the hardware if good enough use fbneo instead of fba.

ps that project looks very interesting!

[MistyDreams]

Little update rewind is possible with an init fix and driver updates if required millage will vary.

Autoload/Save will need more fixing again possible general but a load of work would be required.

Later mames certainly have better savestate support and current mame can 100% do rewind for games with savestate support added properly.

[schmurtzm]

I don't know if your last commits concern this problem. I'm going to build it to make some tests 😃
Thanks for your investigations and feedback !

[MistyDreams]

wont help the problems more work needs done newer mames arent an issue though if implemented properly. if hooked up properly soundcore, cpu cores drivers, video ect on this core it could be fixed too much work to go through every gave that needs fixing for me. rewind support is less intensive that autoload for support but still lots of work.

[mahoneyt944]

@MistyDreams hey I asked Jd about this, and he said this:

It sounds like mame2003 doesn't have complete save state support....
When auto load state is enabled, RA loads the state directly after `retro_load_game()` - it seems that some mame drivers need one or more `retro_run()` calls before the state can actually be loaded

What if we run a frame before leaving retro_load_game()?

[barbudreadmon]

What if run a frame before leaving retro_load_game()?

@mahoneyt944 That was one of my previous suggestions, but you might need a database to know exactly how many frames a given game needs before MAME2003+ is ready to accept its savestates. It's also more of a workaround than a fix as it's not really required (FBNeo doesn't do this).

[MistyDreams]

What if run a frame before leaving retro_load_game()?

The simple answer is you need to reset the cpus(init) before entering the run loop(that sets the save states for the cpu). This is originally in the mame run loop which was re factored. It was changed to start init after the retro run call.

Its nothing to do with with how many frames that have run. If a games like neogeo and rtype have save state support it will work. I have no idea where barbudreadmon is getting his information from clearly this core is not fbneo. Save sate support can be added where it is needed as far as rewind is concerned. There is no need to run extra frames.

[MistyDreams]

https://github.com/mamedev/historic-mame/blob/54a1619c9596ddbdc58fa345726b493f209c4af8/src/cpuexec.c#L377-L410

as you can see the original mame this is the main loop. Only difference is we need to prepare everything before retro_run is called because save states require a size in ra before retro_run this will apply to all mames

[mahoneyt944]

I was just guessing that if we were at a point where a frame ran successfully, the machine would also be ready to auto load a savestate. But this is likely not the best way to resolve this as you pointed out.

[MistyDreams]

run maglord autoload will work as neogeo have pretty good save state support. We done some updates might need looked at will possibly be fine for all games.

rtypeleo points out some issues that will need dealt with though autoload will implode on this one as every variable needed has to be set. Im pretty sure is a stray pointer on that one.

RA is also complaining about misalignment on save or load cant remember which. If someone had a request per game it could probably be fixed. 1943 for example probably just needs the video variables accounted for as the cpus are done already.

[barbudreadmon]

I have no idea where barbudreadmon is getting his information from clearly this core is not fbneo

Just so that we are clear, i know absolutely nothing about MAME2003+ libretro implementation, i was just assuming things were done properly, meaning the machine init was done before the first call to retro_run, as it should. That should indeed be your main concern here, and most likely the main culprit behind this.

I doubt that's the only problem though, since the OP apparently can reproduce the problem just by spamming manual load at boot, and i don't think manual load is even available before retro_run has been called at least once.

[MistyDreams]

yes like i said save state support would need added to that game. You have little to no knowledge of whats going on in this core. Try reading what i posted you might see thats been said already.

[barbudreadmon]

I was talking to @mahoneyt944 because what he said was pretty much what i previously said.
As a rule of thumb, i ignore your posts whenever possible, so sorry that i didn't saw the one pinpointing this. You are right, retro_load_game MUST indeed finish initing the machine, reset included.

[MistyDreams]

lol