WIP: Infrastructure: sceNetInet and sceNetResolver support. #18578

JustAWhiteBloodCell · 2023-12-18T22:29:49Z

Description

Support infrastructure mode in some(?) games by implementing a portion of the sceNetInet and sceNetResolver modules. Haven't tested for longer than 5 mins in game but I don't think there are memory issues etc, it is a fairly straightforward shim.

Summary of changes made

sceNet: Move sceNetInet and sceNetResolver into dedicated files for maintenance.
sceNetInet: Support most inet functions, excluding those which are not executed for the SOCOMs. Will look into more games.
sceNetResolver: Took anr2me's existing PR and incorporated it into the larger refactor of sceNet. There is also some work to add DNS aliasing a la CNAME in order to support revived infrastructure servers. Hopefully these revived services will decide to open source their code to make sure people in the future can set up their own when they inevitably shut down.

Changes yet to make

Move away from SceNet prefixing for SceSocket and similar. Thinking just InetSocket and similar.
Lots of TODOs in the code still.
Core/Net/* should be moved to Common/Net.
Use spaces instead of tabs. (my IDE defaulted to tabs I guess? it was a new installation)
Ensure braces are unified in all new code with preferred style.

Caveats, assumptions, callouts

Socket file descriptors in game are not the same as the native file descriptors. I'm somewhat concerned about using the same descriptors as the host. I would like to switch to using the native UID and actually at least emulate those but there's no guarantee that's possible or practical.
Primarily developed (and consequently tested) on Linux platform.
FreeBSD, OpenBSD and macOS should fall under the same path as Linux since they treat non-blocking sockets the same (as far as I can tell). Will test on at least FreeBSD or macOS before finally submitting.
During WIP: Windows does not work well, the behavior seems off and it seems to be making way too many calls and behaving overall strangely. I think there is a timeout that needs to be adjusted somewhere or something, will do before marking this as not-a-draft.

JustAWhiteBloodCell · 2023-12-18T23:12:32Z

Core/HLE/sceNetInet.cpp

+	if (error != ERROR_WHEN_NONBLOCKING_CALL_OCCURS && error != 0) {
+		INFO_LOG(SCENET, "Requested sceNetInetGetErrno %i=%s", error, strerror(error));
+	}
+	switch (error) {


TODO: Create a map for O(1) lookups.

JustAWhiteBloodCell · 2023-12-18T23:13:10Z

Core/HLE/sceNetInet.cpp

+
+int sceNetInetTerm() {
+    ERROR_LOG(SCENET, "UNTESTED sceNetInetTerm()");
+	SceNetInet::Shutdown();


TODO: Figure out why CLion felt like using tabs instead of spaces ......

Was just gonna comment on that :D

hrydgard · 2023-12-18T23:13:35Z

This is, of course, extremely cool! Nice job!

JustAWhiteBloodCell · 2023-12-18T23:20:50Z

Core/HLE/sceNetInet.cpp

+	static constexpr int gFdsBitsCount = 8 * static_cast<int>(sizeof(fdMask));
+
+	struct FdSet {
+		fdMask mFdsBits[256 / gFdsBitsCount];


TODO: document 256 being max FD_SET size for PSP

JustAWhiteBloodCell · 2023-12-18T23:23:18Z

Core/HLE/sceNetInet.cpp

+#pragma comment(lib, "ws2_32.lib")
+#define close closesocket
+#define ERROR_WHEN_NONBLOCKING_CALL_OCCURS WSAEWOULDBLOCK
+using netBufferType = char;


TODO: document why this is necessary, C++ reinterpret_cast dealing with differing C headers between winsock and sane POSIX sockets implementations

LunaMoo · 2023-12-19T02:05:46Z

Use spaces instead of tabs. (my IDE defaulted to tabs I guess? it was a new installation)

Don't you mean the other way around? It uses spaces now, while the rest of the project and probably most of us in general uses tabs.

JustAWhiteBloodCell · 2023-12-19T03:23:53Z

I actually did mean what I said, but I realized after I posted that tabs was in fact used for this project. I am just too used to spaces instead of tabs 😅. I'll make sure that that is fixed.

anr2me · 2023-12-19T09:16:55Z

Socket file descriptors in game are not the same as the native file descriptors. I'm somewhat concerned about using the same descriptors as the host. I would like to switch to using the native UID and actually at least emulate those but there's no guarantee that's possible or practical.

PSP only supports socket id/file descriptors of 0 to 255 (8 bits), some games even use the lowest 8-bit value only, if you use Windows's socket fd directly without mapping it to [0..255] many games will be facing crashes (due to out of bound access during FD_SET/etc. by the game) and inaccuracy during select (since fd number higher than 255 can't be mapped into the 256-bits bitmap), as PSP use a bitmap of socket's fds (similar to Linux/BSD) instead of array of fds like what Windows's use. Linux should also face the same issue as it supports more than 255 (1023 or more i think) fd number.
And as i remembered these 0..255 range ids also shared with Adhoc's sockets (ie. creating PTP/PDP sockets will increase the id number too)

You should create a socket objects and map the id there, you will also need to store non-blocking information and implement a simulation of blocking behavior instead of using blocking mode directly (which will blocks the emulation).
Simply replacing blocking socket with non-blocking one without handling/converting the error codes properly may not always works on some games (similar issue happened before blocking simulation being implemented on Adhoc sockets in the past)

JustAWhiteBloodCell · 2023-12-19T14:59:01Z

Thanks for that, I think I saw similar comments by you (and others) elsewhere and so that's what I've done.

socket ids are base-1, controlled by mCurrentSocketId in SceNetInet class which is to say that when inet is re-initiailized the number will reset and there is a mapping stored to provide quick translation.
I have created SceSocket which contains non-blocking information and the native socket id. I will most likely rename this class to InetSocket.
Non-blocking functionality is implemented as you describe.
I've also translated the PSP 256-socket-limited fd_set into the platform-specific fd_set specifically for select and I've observed it to work. I also have a check primarily meant for Windows to ensure that the max translated fd_set is not too large. I find it very unlikely that a PSP game would have more than maybe a dozen sockets open at once though.
Currently working on the error code translation. I did at least translate the get/setsockopt optnames so far.

If you have any examples of such games please let me know because I'd like to validate its robustness as much as possible.

anr2me · 2023-12-19T19:04:44Z

I'll upload my stashed code later after i fixes the conflicts with the latest master, it probably not the latest one i had, since i couldn't found the latest one where i already fix the conflicts and separating the NetResolver out. (Edit: Looks like NetResolver has been separated out from this stashed code, but still have the sceHttp included)
I remembered putting my whole Projects folder on a single RAR file before i lost all my partitions due to bad sectors, but couldn't found that RAR file on my external drive :( may be i forgot to copy that file to external drive.

Anyway, you can use my stashed code as reference if you want, as i already implement nearly all the Inet syscalls (i think i only missing the SceNetInetSendmsg, but you can use my SceNetInetRecvmsg implementation as reference, both of them have a complex structure containing flags that need to be converted, but only 2 games so far known to use such syscalls)

Edit: ugh i'm facing an error i don't know how to resolve after i'm done fixing the conflicts.

Severity	Code	Description	Project	File	Line	Suppression State	Details
Error	RC2104	undefined keyword or key name: PPSSPP_WIN_VERSION_STRING	PPSSPPWindows	D:\Data\Projects\ppsspp\Windows\version.rc	28

PS: i'm using a fresh installation of Win11Pro 23H2 + Visual Studio Community 2022 (Desktop C++ Dev & UWP components) + Python 3.12.1, did i missed anything else?

Edit2: Oops i forgot that i use the embedded version of git in Sourcetree, thus not in the PATH env var (didn't know that the build process will calls git command)

JustAWhiteBloodCell · 2023-12-20T00:45:33Z

Thanks for that! You don't need to resolve conflicts etc I can handle cherry-picking it in even if its just a patch file and include you as a co-author, if that works for you? I plan to resume working on this weekend which is when my vacation starts.

anr2me · 2023-12-20T02:34:37Z

I've uploaded it as PR at my fork anr2me#21

The reason i need to fix the conflicts (and also merge back the NetResolver library) was because i need to build an up to date test builds for testers at #14256

I also need to create new yml file to generate IPA for iOS later (taking it out from #15450)

JustAWhiteBloodCell · 2023-12-20T15:06:31Z

Thank you! I should be able to get this PR updated against it this weekend and then move onto fixing all the builds..

Co-authored-by: AdamN <anr2000@gmail.com>

…mes impl and begin converting ERROR_LOGS into hleLogError + ensuring last error is set.

JustAWhiteBloodCell commented Dec 18, 2023

View reviewed changes

hrydgard added the Networking / adhoc label Dec 18, 2023

JustAWhiteBloodCell commented Dec 18, 2023

View reviewed changes

JustAWhiteBloodCell force-pushed the infra-master branch from 62ef20b to fe8d0c3 Compare December 25, 2023 17:45

sceNetInet and sceNetResolver support.

4ddf3ea

Co-authored-by: AdamN <anr2000@gmail.com>

JustAWhiteBloodCell force-pushed the infra-master branch from fe8d0c3 to 4ddf3ea Compare December 25, 2023 17:48

JustAWhiteBloodCell added 2 commits December 26, 2023 11:53

Implement sceNetInetGetpeername and sceNetInetGetsockname as per anr2…

a096367

…mes impl and begin converting ERROR_LOGS into hleLogError + ensuring last error is set.

Add header to sceNetInet and audit hleLogError calls for SetLastError.

8cd4a65

JustAWhiteBloodCell force-pushed the infra-master branch from e885c54 to 8cd4a65 Compare December 27, 2023 18:15

Fix bad formatting in CMakeLists.

d49c3ec

JustAWhiteBloodCell force-pushed the infra-master branch from c1dad25 to d49c3ec Compare December 27, 2023 18:20

JustAWhiteBloodCell added 2 commits December 27, 2023 11:46

Translate all input parameters to socket().

516877c

Move flags and other constants from InetSocket onto SceNetInet.

03ac8fc

anr2me mentioned this pull request Jan 1, 2024

Infrastructure/online support #14256

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: Infrastructure: sceNetInet and sceNetResolver support. #18578

WIP: Infrastructure: sceNetInet and sceNetResolver support. #18578

JustAWhiteBloodCell commented Dec 18, 2023 •

edited

JustAWhiteBloodCell Dec 18, 2023

JustAWhiteBloodCell Dec 18, 2023

hrydgard Dec 18, 2023

hrydgard commented Dec 18, 2023

JustAWhiteBloodCell Dec 18, 2023

JustAWhiteBloodCell Dec 18, 2023

LunaMoo commented Dec 19, 2023

JustAWhiteBloodCell commented Dec 19, 2023

anr2me commented Dec 19, 2023 •

edited

JustAWhiteBloodCell commented Dec 19, 2023 •

edited

anr2me commented Dec 19, 2023 •

edited

JustAWhiteBloodCell commented Dec 20, 2023 •

edited

anr2me commented Dec 20, 2023

JustAWhiteBloodCell commented Dec 20, 2023

WIP: Infrastructure: sceNetInet and sceNetResolver support. #18578

Are you sure you want to change the base?

WIP: Infrastructure: sceNetInet and sceNetResolver support. #18578

Conversation

JustAWhiteBloodCell commented Dec 18, 2023 • edited

Description

Summary of changes made

Changes yet to make

Caveats, assumptions, callouts

JustAWhiteBloodCell Dec 18, 2023

Choose a reason for hiding this comment

JustAWhiteBloodCell Dec 18, 2023

Choose a reason for hiding this comment

hrydgard Dec 18, 2023

Choose a reason for hiding this comment

hrydgard commented Dec 18, 2023

JustAWhiteBloodCell Dec 18, 2023

Choose a reason for hiding this comment

JustAWhiteBloodCell Dec 18, 2023

Choose a reason for hiding this comment

LunaMoo commented Dec 19, 2023

JustAWhiteBloodCell commented Dec 19, 2023

anr2me commented Dec 19, 2023 • edited

JustAWhiteBloodCell commented Dec 19, 2023 • edited

anr2me commented Dec 19, 2023 • edited

JustAWhiteBloodCell commented Dec 20, 2023 • edited

anr2me commented Dec 20, 2023

JustAWhiteBloodCell commented Dec 20, 2023

JustAWhiteBloodCell commented Dec 18, 2023 •

edited

anr2me commented Dec 19, 2023 •

edited

JustAWhiteBloodCell commented Dec 19, 2023 •

edited

anr2me commented Dec 19, 2023 •

edited

JustAWhiteBloodCell commented Dec 20, 2023 •

edited