Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

post multipart entity error #161

Open
fuhongwei041 opened this issue Mar 4, 2019 · 10 comments
Open

post multipart entity error #161

fuhongwei041 opened this issue Mar 4, 2019 · 10 comments
Labels

Comments

@fuhongwei041
Copy link

i found an error, missing multipart processing logic. like this:
if (ServletFileUpload.isMultipartContent(httpServletRequest)) {
this.handleMultipart(postMethodProxyRequest, httpServletRequest);
} else {
this.handleStandard(postMethodProxyRequest, httpServletRequest);
}

@dsmiley
Copy link
Collaborator

dsmiley commented Mar 7, 2019

Huh?

@maxid
Copy link

maxid commented Sep 25, 2019

fixed method newProxyRequestWithEntity

private HttpRequest newProxyRequestWithEntity(
            String method,
            String proxyRequestUri,
            HttpServletRequest servletRequest,
            boolean isMultipart
    ) throws IOException, ServletException {
        HttpEntityEnclosingRequest request = new BasicHttpEntityEnclosingRequest(method, proxyRequestUri);
        HttpEntity entity;
        if (isMultipart) {
            MultipartEntityBuilder builder = MultipartEntityBuilder.create();
            builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
            StandardMultipartHttpServletRequest multipart = (StandardMultipartHttpServletRequest) servletRequest;
            Collection<Part> parts = multipart.getParts();
            for (Part part : parts) {
                String name = part.getName(), fileName = part.getSubmittedFileName();
                if (fileName == null) {
                    builder.addPart(name, new PartBody(part.getInputStream(), ContentType.MULTIPART_FORM_DATA, part.getSize()));
                } else {
                    builder.addPart(name, new PartBody(part.getInputStream(), fileName, part.getSize()));
                }
            }
            entity = builder.build();
        } else {
            entity = new InputStreamEntity(servletRequest.getInputStream(), getContentLength(servletRequest));
        }
        request.setEntity(entity);
        return request;
    }

    public class PartBody extends InputStreamBody {
        private long size;

        public PartBody(InputStream in, ContentType contentType, long size) {
            super(in, contentType);
            this.size = size;
        }

        public PartBody(InputStream in, String filename, long size) {
            super(in, filename);
            this.size = size;
        }

        @Override
        public long getContentLength() {
            return size;
        }
    }

StandardMultipartHttpServletRequest Refer to the checkMultipart method of DispatcherServlet

@dsmiley
Copy link
Collaborator

dsmiley commented Sep 30, 2019

I don't know where the code you refer to is. Perhaps submit a PR and failing test that this will fix, if you would like it accepted.

@roncking
Copy link

roncking commented Oct 1, 2019

Hi Maxid, looks like your solution requires Spring? That's an extra dependency, right? Won't work for me, I'm not using Spring! I think StandardMultipartHttpServletRequest is from Spring?

@xiaolizhanshi
Copy link

Hi dsmiley:
when upload file with content-type of mult-type in spirng boot, the server will happens this error:

[2020-05-25 16:34:52] [ERROR] [http-nio-8081-exec-1] [/] [c.i.p.s.c.e.BaseExceptionHandler:baseError:20] BaseException报错: {}
org.springframework.web.multipart.MultipartException: Failed to parse multipart servlet request; nested exception is java.io.IOException: org.apache.tomcat.util.http.fileupload.FileUploadException: java.net.SocketTimeoutException
at org.springframework.web.multipart.support.StandardMultipartHttpServletRequest.handleParseFailure(StandardMultipartHttpServletRequest.java:124)
at org.springframework.web.multipart.support.StandardMultipartHttpServletRequest.parseRequest(StandardMultipartHttpServletRequest.java:115)
at org.springframework.web.multipart.support.StandardMultipartHttpServletRequest.(StandardMultipartHttpServletRequest.java:88)
at org.springframework.web.multipart.support.StandardServletMultipartResolver.resolveMultipart(StandardServletMultipartResolver.java:87)
at org.springframework.web.servlet.DispatcherServlet.checkMultipart(DispatcherServlet.java:1178)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1012)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: org.apache.tomcat.util.http.fileupload.FileUploadException: java.net.SocketTimeoutException
at org.apache.catalina.connector.Request.parseParts(Request.java:2917)
at org.apache.catalina.connector.Request.getParts(Request.java:2772)
at org.apache.catalina.connector.RequestFacade.getParts(RequestFacade.java:1098)
at org.springframework.web.multipart.support.StandardMultipartHttpServletRequest.parseRequest(StandardMultipartHttpServletRequest.java:95)
... 35 common frames omitted
Caused by: org.apache.tomcat.util.http.fileupload.FileUploadException: java.net.SocketTimeoutException
at org.apache.tomcat.util.http.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:308)
at org.apache.catalina.connector.Request.parseParts(Request.java:2870)
... 38 common frames omitted
Caused by: org.apache.catalina.connector.ClientAbortException: java.net.SocketTimeoutException
at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:340)
at org.apache.catalina.connector.InputBuffer.checkByteBufferEof(InputBuffer.java:632)
at org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:362)
at org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:132)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at org.apache.tomcat.util.http.fileupload.util.LimitedInputStream.read(LimitedInputStream.java:132)
at org.apache.tomcat.util.http.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:977)
at org.apache.tomcat.util.http.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:881)
at java.io.InputStream.read(InputStream.java:101)
at org.apache.tomcat.util.http.fileupload.util.Streams.copy(Streams.java:98)
at org.apache.tomcat.util.http.fileupload.util.Streams.copy(Streams.java:68)
at org.apache.tomcat.util.http.fileupload.MultipartStream.readBodyData(MultipartStream.java:572)
at org.apache.tomcat.util.http.fileupload.MultipartStream.discardBodyData(MultipartStream.java:596)
at org.apache.tomcat.util.http.fileupload.MultipartStream.skipPreamble(MultipartStream.java:614)
at org.apache.tomcat.util.http.fileupload.impl.FileItemIteratorImpl.findNextItem(FileItemIteratorImpl.java:217)
at org.apache.tomcat.util.http.fileupload.impl.FileItemIteratorImpl.(FileItemIteratorImpl.java:131)
at org.apache.tomcat.util.http.fileupload.FileUploadBase.getItemIterator(FileUploadBase.java:255)
at org.apache.tomcat.util.http.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:279)
... 39 common frames omitted
Caused by: java.net.SocketTimeoutException: null
at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.fillReadBuffer(NioEndpoint.java:1252)
at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.read(NioEndpoint.java:1158)
at org.apache.coyote.http11.Http11InputBuffer.fill(Http11InputBuffer.java:736)
at org.apache.coyote.http11.Http11InputBuffer.access$300(Http11InputBuffer.java:42)
at org.apache.coyote.http11.Http11InputBuffer$SocketInputBuffer.doRead(Http11InputBuffer.java:1099)
at org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:102)
at org.apache.coyote.http11.Http11InputBuffer.doRead(Http11InputBuffer.java:247)
at org.apache.coyote.Request.doRead(Request.java:551)
at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:336)
... 56 common frames omitted

@xiaolizhanshi
Copy link

maxid:
Do not copy header of content-type with multypart can resolve this problem

@harry12800
Copy link

Hi dsmiley: when upload file with content-type of mult-type in spirng boot, the server will happens this error:

[2020-05-25 16:34:52] [ERROR] [http-nio-8081-exec-1] [/] [c.i.p.s.c.e.BaseExceptionHandler:baseError:20] BaseException报错: {} org.springframework.web.multipart.MultipartException: Failed to parse multipart servlet request; nested exception is java.io.IOException: org.apache.tomcat.util.http.fileupload.FileUploadException: java.net.SocketTimeoutException at org.springframework.web.multipart.support.StandardMultipartHttpServletRequest.handleParseFailure(StandardMultipartHttpServletRequest.java:124) at org.springframework.web.multipart.support.StandardMultipartHttpServletRequest.parseRequest(StandardMultipartHttpServletRequest.java:115) at org.springframework.web.multipart.support.StandardMultipartHttpServletRequest.(StandardMultipartHttpServletRequest.java:88) at org.springframework.web.multipart.support.StandardServletMultipartResolver.resolveMultipart(StandardServletMultipartResolver.java:87) at org.springframework.web.servlet.DispatcherServlet.checkMultipart(DispatcherServlet.java:1178) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1012) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909) at javax.servlet.http.HttpServlet.service(HttpServlet.java:660) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: org.apache.tomcat.util.http.fileupload.FileUploadException: java.net.SocketTimeoutException at org.apache.catalina.connector.Request.parseParts(Request.java:2917) at org.apache.catalina.connector.Request.getParts(Request.java:2772) at org.apache.catalina.connector.RequestFacade.getParts(RequestFacade.java:1098) at org.springframework.web.multipart.support.StandardMultipartHttpServletRequest.parseRequest(StandardMultipartHttpServletRequest.java:95) ... 35 common frames omitted Caused by: org.apache.tomcat.util.http.fileupload.FileUploadException: java.net.SocketTimeoutException at org.apache.tomcat.util.http.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:308) at org.apache.catalina.connector.Request.parseParts(Request.java:2870) ... 38 common frames omitted Caused by: org.apache.catalina.connector.ClientAbortException: java.net.SocketTimeoutException at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:340) at org.apache.catalina.connector.InputBuffer.checkByteBufferEof(InputBuffer.java:632) at org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:362) at org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:132) at java.io.FilterInputStream.read(FilterInputStream.java:133) at org.apache.tomcat.util.http.fileupload.util.LimitedInputStream.read(LimitedInputStream.java:132) at org.apache.tomcat.util.http.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:977) at org.apache.tomcat.util.http.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:881) at java.io.InputStream.read(InputStream.java:101) at org.apache.tomcat.util.http.fileupload.util.Streams.copy(Streams.java:98) at org.apache.tomcat.util.http.fileupload.util.Streams.copy(Streams.java:68) at org.apache.tomcat.util.http.fileupload.MultipartStream.readBodyData(MultipartStream.java:572) at org.apache.tomcat.util.http.fileupload.MultipartStream.discardBodyData(MultipartStream.java:596) at org.apache.tomcat.util.http.fileupload.MultipartStream.skipPreamble(MultipartStream.java:614) at org.apache.tomcat.util.http.fileupload.impl.FileItemIteratorImpl.findNextItem(FileItemIteratorImpl.java:217) at org.apache.tomcat.util.http.fileupload.impl.FileItemIteratorImpl.(FileItemIteratorImpl.java:131) at org.apache.tomcat.util.http.fileupload.FileUploadBase.getItemIterator(FileUploadBase.java:255) at org.apache.tomcat.util.http.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:279) ... 39 common frames omitted Caused by: java.net.SocketTimeoutException: null at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.fillReadBuffer(NioEndpoint.java:1252) at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.read(NioEndpoint.java:1158) at org.apache.coyote.http11.Http11InputBuffer.fill(Http11InputBuffer.java:736) at org.apache.coyote.http11.Http11InputBuffer.access$300(Http11InputBuffer.java:42) at org.apache.coyote.http11.Http11InputBuffer$SocketInputBuffer.doRead(Http11InputBuffer.java:1099) at org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:102) at org.apache.coyote.http11.Http11InputBuffer.doRead(Http11InputBuffer.java:247) at org.apache.coyote.Request.doRead(Request.java:551) at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:336) ... 56 common frames omitted

这个问题解决了吗?
The Problem ,How to resolve?

@dsmiley
Copy link
Collaborator

dsmiley commented Sep 12, 2022

PR welcome, especially one that includes a test that fails without the fix.

@dsmiley dsmiley added the bug label Sep 12, 2022
@bechhansen
Copy link

fixed method newProxyRequestWithEntity

private HttpRequest newProxyRequestWithEntity(
            String method,
            String proxyRequestUri,
            HttpServletRequest servletRequest,
            boolean isMultipart
    ) throws IOException, ServletException {
        HttpEntityEnclosingRequest request = new BasicHttpEntityEnclosingRequest(method, proxyRequestUri);
        HttpEntity entity;
        if (isMultipart) {
            MultipartEntityBuilder builder = MultipartEntityBuilder.create();
            builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
            StandardMultipartHttpServletRequest multipart = (StandardMultipartHttpServletRequest) servletRequest;
            Collection<Part> parts = multipart.getParts();
            for (Part part : parts) {
                String name = part.getName(), fileName = part.getSubmittedFileName();
                if (fileName == null) {
                    builder.addPart(name, new PartBody(part.getInputStream(), ContentType.MULTIPART_FORM_DATA, part.getSize()));
                } else {
                    builder.addPart(name, new PartBody(part.getInputStream(), fileName, part.getSize()));
                }
            }
            entity = builder.build();
        } else {
            entity = new InputStreamEntity(servletRequest.getInputStream(), getContentLength(servletRequest));
        }
        request.setEntity(entity);
        return request;
    }

    public class PartBody extends InputStreamBody {
        private long size;

        public PartBody(InputStream in, ContentType contentType, long size) {
            super(in, contentType);
            this.size = size;
        }

        public PartBody(InputStream in, String filename, long size) {
            super(in, filename);
            this.size = size;
        }

        @Override
        public long getContentLength() {
            return size;
        }
    }

StandardMultipartHttpServletRequest Refer to the checkMultipart method of DispatcherServlet

I'm struggelig with the same issue. Looks like Multipart requests are not very well supported.

I have an issue where the server I'm calling is surpose to return an 403 error when about to upload a large file.
Apparently the ProxyServlet is sending the entire file and then return an I/O exception (java.net.SocketException) caught... error message. I expect the proxy only to send the first part and then receive the 403 error and not send the large file.

Are you actually able to get the parts when calling multipart.getParts()?
When I do that I just get an java.lang.IllegalStateException: UT010057: multipart config was not present on Servlet error.

@bechhansen
Copy link

fixed method newProxyRequestWithEntity

private HttpRequest newProxyRequestWithEntity(
            String method,
            String proxyRequestUri,
            HttpServletRequest servletRequest,
            boolean isMultipart
    ) throws IOException, ServletException {
        HttpEntityEnclosingRequest request = new BasicHttpEntityEnclosingRequest(method, proxyRequestUri);
        HttpEntity entity;
        if (isMultipart) {
            MultipartEntityBuilder builder = MultipartEntityBuilder.create();
            builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
            StandardMultipartHttpServletRequest multipart = (StandardMultipartHttpServletRequest) servletRequest;
            Collection<Part> parts = multipart.getParts();
            for (Part part : parts) {
                String name = part.getName(), fileName = part.getSubmittedFileName();
                if (fileName == null) {
                    builder.addPart(name, new PartBody(part.getInputStream(), ContentType.MULTIPART_FORM_DATA, part.getSize()));
                } else {
                    builder.addPart(name, new PartBody(part.getInputStream(), fileName, part.getSize()));
                }
            }
            entity = builder.build();
        } else {
            entity = new InputStreamEntity(servletRequest.getInputStream(), getContentLength(servletRequest));
        }
        request.setEntity(entity);
        return request;
    }

    public class PartBody extends InputStreamBody {
        private long size;

        public PartBody(InputStream in, ContentType contentType, long size) {
            super(in, contentType);
            this.size = size;
        }

        public PartBody(InputStream in, String filename, long size) {
            super(in, filename);
            this.size = size;
        }

        @Override
        public long getContentLength() {
            return size;
        }
    }

StandardMultipartHttpServletRequest Refer to the checkMultipart method of DispatcherServlet

I'm struggelig with the same issue. Looks like Multipart requests are not very well supported.

I have an issue where the server I'm calling is surpose to return an 403 error when about to upload a large file. Apparently the ProxyServlet is sending the entire file and then return an I/O exception (java.net.SocketException) caught... error message. I expect the proxy only to send the first part and then receive the 403 error and not send the large file.

Are you actually able to get the parts when calling multipart.getParts()? When I do that I just get an java.lang.IllegalStateException: UT010057: multipart config was not present on Servlet error.

Looks like the cause for my issue is that httpcomponents-core 4.4.x does not support out-of-order responses, where we look for error response before the entity entity is uploaded.

Apparently this feature has been implemented in httpcomponents-core 5.1
https://hc.apache.org/httpcomponents-core-5.1.x/current/httpcore5/apidocs/org/apache/hc/core5/http/impl/io/MonitoringResponseOutOfOrderStrategy.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

7 participants