Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

backup database #4359

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

backup database #4359

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
2d6748a
Add migrate script
broquemonsieur Dec 18, 2023
49f5154
Merge branch 'iv-org:master' into invidious_migrations
broquemonsieur Dec 21, 2023
acf0c19
Add backup creation
broquemonsieur Dec 21, 2023
559d3e3
Remove auto migrate line
broquemonsieur Dec 21, 2023
4eee5cf
Run crystal format
broquemonsieur Dec 22, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions docker/init-invidious-db.sh
View file
Open in desktop
@@ -1,6 +1,8 @@
#!/bin/bash
set -eou pipefail

psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" -c "CREATE SCHEMA IF NOT EXISTS backup"

psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < config/sql/channels.sql
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < config/sql/videos.sql
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < config/sql/channel_videos.sql
Expand Down
35 changes: 35 additions & 0 deletions src/invidious/database/migrator.cr
View file
Open in desktop
Expand Up @@ -10,10 +10,17 @@ class Invidious::Database::Migrator
versions = load_versions

ran_migration = false
backed_up = false
load_migrations.sort_by(&.version)
.each do |migration|
next if versions.includes?(migration.version)

if !backed_up
puts "New migration(s) found: creating database backup"
back_up_database
backed_up = true
end

puts "Running migration: #{migration.class.name}"
migration.migrate
ran_migration = true
Expand Down Expand Up @@ -46,4 +53,32 @@ class Invidious::Database::Migrator
)
SQL
end

private def back_up_database
table_names_request = <<-SQL
SELECT tablename FROM pg_catalog.pg_tables
WHERE schemaname = 'public'
SQL

table_names = @db.query_all(table_names_request, as: String)

table_names.try &.each do |name|
copy_table(name)
end
end

private def copy_table(table_name : String)
@db.exec <<-SQL
CREATE TABLE IF NOT EXISTS backup.#{table_name} (
id bigserial PRIMARY KEY
)
SQL

@db.exec("DROP TABLE backup.#{table_name}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem I see with that is that it might nuke a perfectly valid backup if invidious is interrupted mid-migration and the user tries to run it again immediately after.


@db.exec <<-SQL
SELECT * INTO backup.#{table_name}
FROM public.#{table_name}
SQL
end
end