You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It seems like 95% of our deployment problems to ITRB are the "databases", i.e. massive files that need to be copied to our servers every time we deploy something and the wind is blowing from the west.
This does create a single point of failure for all our instances. But we effectively already have a single point of failure that is the arax-responses MySQL server, and it has never failed. While all these individual database files are independent points of failure and are in a broken state somewhere about 5% of the time.
These databases cause about 20% of deployments to take a hugely long time while databases are downloading of which 50% of those deployments fail because at least one database failed to download completely.
There is the added complexity of managing a RDBMS and loading all our artifacts into it. Perhaps a slight speed penalty, too.
What do you think?
The text was updated successfully, but these errors were encountered:
Based on today's conversation-- there is interest in an AWS solution using an S3 snapshot of a volume containing an ext4 filesystem containing all the databases, and then (at build time) just provision an EBS volume based on that snapshot, and attach the volume into the instance and mount the filesystem as /translatordb or whatever.
edeutsch
changed the title
Have we considered moving our databases to MySQL?
Have we considered moving our databases to at EBS volume?
Nov 17, 2023
Today/tonight after merging some provisioning PR (https://github.com/RTXteam/RTX/pull/2206/files), Pouyan tried to get KG2 up and running again. It resulted in two separate downtimes of 2h45m while it was "downloading the databases":
It seems like 95% of our deployment problems to ITRB are the "databases", i.e. massive files that need to be copied to our servers every time we deploy something and the wind is blowing from the west.
Might we solve most of our unpleasant problems by just migrating all our databases to a central MySQL server? these:
https://github.com/RTXteam/RTX/blob/master/code/config_dbs.json
This does create a single point of failure for all our instances. But we effectively already have a single point of failure that is the arax-responses MySQL server, and it has never failed. While all these individual database files are independent points of failure and are in a broken state somewhere about 5% of the time.
These databases cause about 20% of deployments to take a hugely long time while databases are downloading of which 50% of those deployments fail because at least one database failed to download completely.
There is the added complexity of managing a RDBMS and loading all our artifacts into it. Perhaps a slight speed penalty, too.
What do you think?
The text was updated successfully, but these errors were encountered: