Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

404 Not Found when restoring from backup #673

Open
PacketFiend opened this issue Apr 19, 2018 · 2 comments
Open

404 Not Found when restoring from backup #673

PacketFiend opened this issue Apr 19, 2018 · 2 comments

Comments

@PacketFiend
Copy link

PacketFiend commented Apr 19, 2018

I'm trying to restore a production Cassandra database to a test cluster I have running. The test cluster is the same size as the production cluster, and I have SimpleDB set up in my test cluster to point to the production cluster's backups for restores.

So far, I have been able to make a snapshot backup, and restore the snapshot to the same cluster it was backed up from, but I cannot bootstrap a new cluster, and restore to that. I get the following:

root@cassandra-1:/opt/jetty/apps/priam/logs# curl -i http://127.0.0.1:8080/Priam/REST/v1/backup/restore
HTTP/1.1 404 Not Found
Content-Length: 0
Server: Jetty(9.3.8.v20160314)

Other API calls work fine - I can create a snapshot, list backups, etc... I just can't seem to restore from a pre-existing snapshot. The snapshot is there.

I've tried setting the date range specifically in the API call, as well as the region, and a bunch of other possibilities, but keep getting blocked by this. I'm stumped here.

Any ideas as to how I should proceed from here?

@PacketFiend
Copy link
Author

PacketFiend commented Apr 19, 2018

Well, apparently the documentation may be incorrect - if I restore with curl -i http://127.0.0.1:8080/Priam/REST/v1/restore, it appears to attempt a restore, but now returns the following:

<pre>    Server Error</pre></p><h3>Caused by:</h3><pre>java.lang.RuntimeException: Error transforming file /cassandra/data/meta.json to JSON format.  Msg:/cassandra/data/meta.json (No such file or directory)

I looks to me that the download step just prior to this conversion is failing silently, but I'm not sure about that - I can't find anything relevant in the logs.

I should add - the restore is being performed in an account separate from the one the DB was backed up from. I can confirm that the development account being used to restore the backup has access to the S3 bucket and can download objects from it, but this may still be relevant somehow.

@PacketFiend
Copy link
Author

It's not failing silently - I missed this in the error logs before:

2018-04-24 21:41:13.0761 INFO qtp997608398-13 com.netflix.priam.restore.AbstractRestore Snapshot Meta file for restore content_backups/us-east-1/content/56713727820156410577229101240436610842/201804242042/META/meta.json
2018-04-24 21:41:13.0761 INFO qtp997608398-13 com.netflix.priam.restore.AbstractRestore Bypassing restoring file "/cassandra/data/meta.json" as it is system keyspace
2018-04-24 21:41:13.0766 ERROR qtp997608398-13 com.netflix.priam.restore.AbstractRestore Error while trying to restore: Error transforming file /cassandra/data/meta.json to JSON format.  Msg:/cassandra/data/meta.json (No such file or directory)

So it's deliberately not downloading the meta.json file, and then throwing an exception when it can't find it locally. But why?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant