Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] manifests for some Steam apps are now dicts, not plain IDs, causing CDNClient.get_manifests to crash #436

Open
smcv opened this issue May 5, 2023 · 9 comments · May be fixed by #437
Open

[BUG] manifests for some Steam apps are now dicts, not plain IDs, causing CDNClient.get_manifests to crash #436

smcv opened this issue May 5, 2023 · 9 comments · May be fixed by #437
Labels
bug

Comments

@smcv
Copy link
Contributor

smcv commented May 5, 2023

Description
Some Steam apps (the one I'm aware of is 1628350 "Steam Linux Runtime - Sniper") have changed their ['apps'][id]['manifests'][branch] from a plain manifest ID (ASCII decimal string) to a key/value dictionary with members gid, size and download, causing CDNClient.get_manifests to crash.

Steps to Reproduce the behavior
Run https://gist.github.com/smcv/8430646f3cf5e502711ddf3ea620eaea with --interactive. You might need a Steam account that has added 1628350 "Steam Linux Runtime - Sniper" to the Steam library - it's free, and is used by Proton 8.0.

Expected behavior
The script can download and display VERSION.txt from each branch of each of the Steam Linux Runtime depots.

Screenshots
For 1391110 "Steam Linux Runtime - Soldier" my script is still working:

App ID: 1391110 (Steam Linux Runtime - Soldier)
	Branch: public
		Build ID: 10940185
		Updated: 2023-04-26T15:48:12+00:00
		Depot: 1391111
			Manifest: 5271457197581595980
				#Name	Version		Runtime	Runtime_Version	Comment
				depot	0.20230405.47174			# Overall version number
				pressure-vessel	0.20230405.0	scout		# pressure-vessel-bin.tar.gz
				scripts	0.20230405.0			# from steam-runtime-tools
				soldier	0.20230405.47174	soldier	0.20230405.47174	# soldier_platform_0.20230405.47174/

but for 1628350 "Steam Linux Runtime - Sniper" it now crashes:

App ID: 1628350 (Steam Linux Runtime - Sniper)
	Branch: public
		Build ID: 10940192
		Updated: 2023-04-26T15:50:07+00:00
		Depot: 1628351
			Manifest: {'gid': '820291660013951810', 'size': '703682881', 'download': '188124800'}
Traceback (most recent call last):
  File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run
  File "REDACTED/python-steam/steam/client/cdn.py", line 785, in async_fetch_manifest
    app_id, depot_id, int(manifest_gid), branch_name, branch_pass
                      ^^^^^^^^^^^^^^^^^
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'dict'
2023-05-05T16:09:39Z <Greenlet at 0x7f0665bee200: async_fetch_manifest(1628350, 1628351, {'gid': '820291660013951810', 'size': '703682881',, True, 1628351, branch_name='public', branch_pass=None)> failed with TypeError

Traceback (most recent call last):
  File "REDACTED/python-steam/./describe-depots.py", line 204, in <module>
    main()
  File "REDACTED/python-steam/./describe-depots.py", line 183, in main
    Main(**vars(args)).run()
  File "REDACTED/python-steam/./describe-depots.py", line 158, in run
    for manifest in cdn.get_manifests(
  File "REDACTED/python-steam/steam/client/cdn.py", line 857, in get_manifests
    result = task.get()
  File "src/gevent/greenlet.py", line 831, in gevent._gevent_cgreenlet.Greenlet.get
  File "src/gevent/greenlet.py", line 373, in gevent._gevent_cgreenlet.Greenlet._raise_exception
  File "/usr/lib/python3/dist-packages/gevent/_compat.py", line 66, in reraise
    raise value.with_traceback(tb)
  File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run
  File "REDACTED/python-steam/steam/client/cdn.py", line 785, in async_fetch_manifest
    app_id, depot_id, int(manifest_gid), branch_name, branch_pass
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'dict'

Versions Report

python -m steam.versions_report (Run python -m steam.versions_report and paste the output below) 
steam: 1.4.4

Dependencies:
                 vdf: 3.4
            protobuf: 4.21.12
            requests: 2.28.1
          cachetools: 5.2.0
              gevent: 22.10.2
 gevent-eventemitter: Not Installed
       pycryptodomex: 3.11.0
              enum34: Not Installed
       win-inet-pton: Not Installed

Python runtime:
          executable: /usr/bin/python3
             version: 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0]
            platform: linux

System info:
              system: Linux
             machine: x86_64
             release: 6.1.0-7-amd64
             version: #1 SMP PREEMPT_DYNAMIC Debian 6.1.20-1 (2023-03-19)
@smcv smcv changed the title [BUG] [BUG] manifests for some Steam apps are now dicts, not plain IDs, causing CDNClient.get_manifests to crash May 5, 2023
@smcv
Copy link
Contributor Author

smcv commented May 5, 2023

This appears to be caused by the VDF returned for EMsg.ClientPICSProductInfoRequest having changed. For soldier I get

	"depots"
	{
		"1391111"
		{
			"config"
			{
				"oslist"		"linux"
				"osarch"		"64"
				"language"		""
			}
			"manifests"
			{
				"public"		"5271457197581595980"
				"client_beta"		"3684425101439344717"
				"previous_release"		"1306141847442826428"
			}
...

but for sniper I now get

	"depots"
	{
		"1628351"
		{
			"config"
			{
				"oslist"		"linux"
				"osarch"		"64"
				"language"		""
			}
			"manifests"
			{
				"public"
				{
					"gid"		"820291660013951810"
					"size"		"703682881"
					"download"		"188124800"
				}
				"client_beta"
				{
...

smcv added a commit to smcv/python-steam that referenced this issue May 5, 2023
@smcv
CDNClient: Cope with longer manifest descriptions for some depots
783f023 
Some depots now have a dict like {"public": {"gid": GID, ...}, ...}
where we would traditionally have seen a flat dict {"public": GID, ...}.
Not all depots have this: at the time of writing,
1628350 "Steam Linux Runtime - Sniper" does, but
1391110 "Steam Linux Runtime - Soldier" does not. Presumably this is a
change that is being rolled out gradually to let the Steam client give
better estimates of the size of a depot.

Cope with either format, by retrieving the gid field if the nested dict
is present.

Resolves: ValvePython#436
Signed-off-by: Simon McVittie <smcv@collabora.com>
@smcv smcv linked a pull request May 5, 2023 that will close this issue
Open
@smcv
Copy link
Contributor Author

smcv commented May 5, 2023

Fix proposed in #437.

Gobot1234 added a commit to Gobot1234/steam.py that referenced this issue May 5, 2023
@Gobot1234
Fix for ValvePython/steam#436
2292ca9
@rossengeorgiev
Copy link
Contributor

Can confirm that there a changes to product info data. I'll need to review all parts that are affected in the code base.

@DevilXD
Copy link

DevilXD commented Jun 11, 2023

This issue is breaking my use-case. Is there any ETA on when will this be merged?

App ID I'm using: 596350.

@smcv
Copy link
Contributor Author

smcv commented Jun 13, 2023

This issue is breaking my use-case

Does #437 solve this for the subset of the API you use? If yes, perhaps the maintainer could consider merging that as a stopgap solution, even if there are other locations needing a similar change?

@Totengeist
Copy link

Does #437 solve this for the subset of the API you use?

It works fine for my very simplistic use-case.

@DevilXD
Copy link

DevilXD commented Jun 13, 2023

@smcv Yes, I'm actually already using that PR instead of the official package while I'm waiting, and it does work. Haven't encountered any issues. My use-case involves downloading a single file from via the CDN, from a particular app ID and depot, latest manifest.

@mommalongnips
Copy link

My use case is also suffering from this issue. @smcv your fix resolves it for me. Can we merge in this change? Please!

@MichaelHinrichs
Copy link

Using file_list = mycdn.iter_files(appID) works with @smcv's solution, but now i can't use next(file_list), print(file_list), print(my_file) one at a time, or list(file_list)[:len(file_list)]. I just get the same error from the beginning of this issue.

@fredi-68 fredi-68 mentioned this issue Jan 22, 2024
Open
@StalkR StalkR mentioned this issue Apr 25, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

CDNClient: Cope with longer manifest descriptions for some depots smcv/python-steam
6 participants
@smcv @rossengeorgiev @DevilXD @mommalongnips @MichaelHinrichs @Totengeist