Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot resume upload after server unexpected shutdown #34

Open
antoniolucasnobar opened this issue Sep 3, 2020 · 0 comments
Open

Cannot resume upload after server unexpected shutdown #34

antoniolucasnobar opened this issue Sep 3, 2020 · 0 comments

Comments

@antoniolucasnobar
Copy link

antoniolucasnobar commented Sep 3, 2020

Expected Behaviour

I was trying the https://github.com/tomdesair/tus-java-server-spring-demo with a simple Java Client (based on official tus-java-client)
If I kill the client and resume the upload everything is just fine.
I would expect the same when I kill the server with pending uploads.

Actual Behaviour

After a unexpected shutdown (CTRL+C or power loss) of the tus-java-server, I could not resume the upload that was in progress.
I shutdown the server during a big (1gb) upload.
Then I restarted the server and tried to run the client again. And the server can not handle this upload anymore. (I put the stacktrace and java client at the end)

If I try this same scenario with tusd (official server), everything works as expected (I can resume the upload)

I have the following stacktrace

java.io.EOFException: null
	at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2681) ~[na:1.8.0_221]
	at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3156) ~[na:1.8.0_221]
	at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:862) ~[na:1.8.0_221]
	at java.io.ObjectInputStream.<init>(ObjectInputStream.java:358) ~[na:1.8.0_221]
	at me.desair.tus.server.util.Utils.readSerializable(Utils.java:86) ~[tus-java-server-1.0.0-2.0.jar!/:na]
	at me.desair.tus.server.upload.disk.DiskStorageService.getUploadInfo(DiskStorageService.java:95) ~[tus-java-server-1.0.0-2.0.jar!/:na]
	at me.desair.tus.server.upload.cache.ThreadLocalCachedStorageAndLockingService.getUploadInfo(ThreadLocalCachedStorageAndLockingService.java:53) ~[tus-java-server-1.0.0-2.0.jar!/:na]
	at me.desair.tus.server.upload.cache.ThreadLocalCachedStorageAndLockingService.getUploadInfo(ThreadLocalCachedStorageAndLockingService.java:61) ~[tus-java-server-1.0.0-2.0.jar!/:na]
	at me.desair.tus.server.core.validation.IdExistsValidator.validate(IdExistsValidator.java:24) ~[tus-java-server-1.0.0-2.0.jar!/:na]
	at me.desair.tus.server.util.AbstractTusExtension.validate(AbstractTusExtension.java:37) ~[tus-java-server-1.0.0-2.0.jar!/:na]
	at me.desair.tus.server.TusFileUploadService.validateRequest(TusFileUploadService.java:431) ~[tus-java-server-1.0.0-2.0.jar!/:na]
	at me.desair.tus.server.TusFileUploadService.processLockedRequest(TusFileUploadService.java:406) ~[tus-java-server-1.0.0-2.0.jar!/:na]
	at me.desair.tus.server.TusFileUploadService.process(TusFileUploadService.java:301) ~[tus-java-server-1.0.0-2.0.jar!/:na]
	at me.desair.tus.server.TusFileUploadService.process(TusFileUploadService.java:274) ~[tus-java-server-1.0.0-2.0.jar!/:na]
	at me.desair.spring.tus.FileUploadController.processUpload(FileUploadController.java:24) ~[classes!/:0.0.1-SNAPSHOT]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_221]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_221]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_221]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_221]
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[spring-web-4.3.21.RELEASE.jar!/:4.3.21.RELEASE]
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) ~[spring-web-4.3.21.RELEASE.jar!/:4.3.21.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) ~[spring-webmvc-4.3.21.RELEASE.jar!/:4.3.21.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:849) ~[spring-webmvc-4.3.21.RELEASE.jar!/:4.3.21.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:760) ~[spring-webmvc-4.3.21.RELEASE.jar!/:4.3.21.RELEASE]
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.3.21.RELEASE.jar!/:4.3.21.RELEASE]
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967) ~[spring-webmvc-4.3.21.RELEASE.jar!/:4.3.21.RELEASE]
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) ~[spring-webmvc-4.3.21.RELEASE.jar!/:4.3.21.RELEASE]
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) ~[spring-webmvc-4.3.21.RELEASE.jar!/:4.3.21.RELEASE]
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861) ~[spring-webmvc-4.3.21.RELEASE.jar!/:4.3.21.RELEASE]
	at javax.servlet.http.HttpServlet.doHead(HttpServlet.java:245) ~[tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:658) ~[tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) ~[spring-webmvc-4.3.21.RELEASE.jar!/:4.3.21.RELEASE]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) ~[tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) ~[tomcat-embed-websocket-8.5.35.jar!/:8.5.35]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) ~[spring-web-4.3.21.RELEASE.jar!/:4.3.21.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.21.RELEASE.jar!/:4.3.21.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109) ~[spring-web-4.3.21.RELEASE.jar!/:4.3.21.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.21.RELEASE.jar!/:4.3.21.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93) ~[spring-web-4.3.21.RELEASE.jar!/:4.3.21.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.21.RELEASE.jar!/:4.3.21.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197) ~[spring-web-4.3.21.RELEASE.jar!/:4.3.21.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.3.21.RELEASE.jar!/:4.3.21.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) ~[tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493) [tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) [tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) [tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) [tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:800) [tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806) [tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498) [tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_221]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_221]
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.35.jar!/:8.5.35]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_221]

Steps to reproduce (you have to put the path of a big file)

I start the server demo
I run this java client
I kill server
I kill client
I start the server demo
I run this java client
(I got error above)

public class Client {

  public static void main(String args[]) throws IOException, ProtocolException {
    Path testFile = Paths.get("/path/to/big/file");
    // Upload file to server
    var client = new TusClient();
    client.setUploadCreationURL(URI.create("http://localhost:8080/test/api/upload").toURL());
    client.enableResuming(new MyTusURLPropertiesStore("./fingerprints.properties"));
    TusUpload upload = new TusUpload(testFile.toFile());
    var executor = new TusExecutor() {
      @Override
      protected void makeAttempt() throws ProtocolException, IOException {
        TusUploader uploader = client.resumeOrCreateUpload(upload);
        uploader.setChunkSize(1024);
        do {
          long totalBytes = upload.getSize();
          long bytesUploaded = uploader.getOffset();
          double progress = (double) bytesUploaded / totalBytes * 100;
          System.out.printf("Upload at %6.2f %%.\n", progress);
        } while (uploader.uploadChunk() > -1);
        uploader.finish();
      }
    };
    System.out.println(executor.makeAttempts() ? "Upload successful" :  "Upload interrupted");
  }
}

class MyTusURLPropertiesStore implements TusURLStore {

  private final Path arquivoConfig;
  String propertiesFile;

  Properties properties;

  public MyTusURLPropertiesStore(String propertiesFile) throws IOException {
    this.propertiesFile = propertiesFile;
    arquivoConfig = Paths.get(propertiesFile);
    if (!Files.exists(arquivoConfig)) {
      Files.createDirectories(arquivoConfig.getParent());
      Files.createFile(arquivoConfig);
    }
    try (InputStream inputStream = Files.newInputStream(arquivoConfig)) {
      this.properties = new Properties();
      this.properties.load(inputStream);
    }
  }

  private void gravar() {
    try (OutputStream outputStream = Files.newOutputStream(arquivoConfig)) {
      this.properties.store(outputStream, null);
    } catch (IOException e) {
      e.printStackTrace();
    }
  }

  @Override
  public void set(String fingerprint, URL url) {
    try {
      this.properties.put(fingerprint, url.toURI().toASCIIString());
    } catch (URISyntaxException e) {
      e.printStackTrace();
    }
    gravar();
  }

  @Override
  public URL get(String fingerprint) {
    Object o = this.properties.get(fingerprint);
    if (o != null) {
      try {
        return URI.create((String) o).toURL();
      } catch (MalformedURLException e) {
        e.printStackTrace();
      }
    }
    return null;
  }

  @Override
  public void remove(String fingerprint) {
    this.properties.remove(fingerprint);
    gravar();
  }
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant