Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Asynchronous Inbox + asynchronous house saving #4561

Open
wants to merge 29 commits into
base: master
Choose a base branch
from

Conversation

gunzino
Copy link
Contributor

@gunzino gunzino commented Nov 8, 2023

  • I have followed [proper The Forgotten Server code styling][code].
  • I have read and understood the [contribution guidelines][cont] before making this PR.
  • I am aware that this PR may be closed if the above-mentioned criteria are not fulfilled.

Changes Proposed

I'm happy to share a contribution to the TFS which consists of async, zero-blocking Inbox implementation to prevent abusing the current sync system with large Inboxes and causing lags due to database IO like mentioned here #1431. The similar implementation is rotating between server owners but does not include complete zero-blocking IO for both saving and loading and parcel/item delivery to Inbox when player is offline. The implementation can be further used for more components like depots, store inbox etc. but those are currently either limited or controlled by server. The basic concept was created by kondra (otclient@otclient.ovh) few years ago but the code is already widespread across server owners which some of them resell for real money.

The implementation highlights:

  • The database IO and some of the serialization is done in separate thread (secondary Dispatcher g_dispatcherInbox) with dedicated database connection
  • Data structures protected by mutex
  • Async zero-blocking delivery of parcels and items from market/houses

Items delivery to inbox currently done for:

  • Houses
  • Parcels
  • Market

EDIT (21th November): Added asynchronous houses saving

I would be really happy for your hints/recommendations regarding so this contribution can be pushed to the master branch.

src/ioinbox.cpp Fixed Show fixed Hide fixed
src/container.cpp Outdated Show resolved Hide resolved
Copy link
Contributor

@ramon-bernardo ramon-bernardo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I liked the PR, it looks promising.

I made some improvements, removed duplicate code, see
https://github.com/ramon-bernardo/forgottenserver/pull/2

I'm still reviewing IOInbox!

src/player.cpp Outdated
{
inbox = _inbox;
if (depotLocker) {
depotLocker->internalAddThing(2, inbox);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the index here could be a const?

src/player.cpp Outdated Show resolved Hide resolved
src/game.cpp Outdated Show resolved Hide resolved
src/mailbox.cpp Outdated Show resolved Hide resolved
@EPuncker EPuncker added the enhancement Increase or improvement in quality, value, or extent label Nov 9, 2023
…ng inbox, the delivery should now be guaranteed and IOInbox::flush is now only necessary when g_dispatcherInbox does not accept new tasks, needs further research if IOInbox::flush() can be removed
@gunzino gunzino changed the title [WIP] Asynchronous Inbox implementation Asynchronous Inbox implementation Nov 10, 2023
@gunzino gunzino marked this pull request as ready for review November 11, 2023 15:50
@gunzino gunzino changed the title Asynchronous Inbox implementation Asynchronous Inbox + Asynchronous house saving Nov 21, 2023
@gunzino
Copy link
Contributor Author

gunzino commented Nov 21, 2023

@ramon-bernardo
I've added async house saving today as well which uses similar approach. Can you review?

@gunzino gunzino changed the title Asynchronous Inbox + Asynchronous house saving Asynchronous Inbox + asynchronous house saving Nov 21, 2023
{
static Database asyncInstance;
return asyncInstance;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC getInstance already returns an "async" instance, used for other purposes but should be suitable. Why having another instance?

On that note, singleton sucks, but a "doubleton" is even worse 😆 let's just make it not a global and pass it around

Copy link
Contributor Author

@gunzino gunzino Dec 2, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean byt getInstance returns "async" instance? Is Database class thread-safe? Can database class execute multiple parallel queries from several threads? The asyncInstance is there only to be used by g_asyncTasks thread for async database IO- in this PR it is now only House and Inbox.

std::map<uint32_t, PlayerDBEntryPtr> inboxCache;
std::map<uint32_t, std::list<ItemBlockList>> pendingItemsToSave;
std::set<uint32_t> pendingPlayerSet;
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make it a namespace and move everything private to the .cpp file only as this is not used anywhere else

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there similar approach somewhere else in the code? What about recursive mutex?

@gunzino gunzino requested a review from ranisalt December 6, 2023 07:51
@Codinablack
Copy link
Contributor

This looks like a very promising PR, what is the status of this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Increase or improvement in quality, value, or extent
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants