TransferUtility.UploadAsync() throws error: "content would exceed Content-Length" when update to .NET8 #3201

julian-dimitroff · 2024-02-27T12:34:55Z

Describe the bug

I recently migrated our project to .NET8 and updated all the packages to latest version, but I keep getting a strange error when I try to upload a stream to Amazon S3 storage. I try to upload a relatively small file ~500 KB.

Expected Behavior

Should not throw exception in at System.IO.Stream.<CopyToAsync>

Current Behavior

It throws exception:

System.Net.Http.HttpRequestException: Unable to write content to request stream; content would exceed Content-Length.
   at System.IO.Stream.<CopyToAsync>g__Core|27_0(Stream source, Stream destination, Int32 bufferSize, CancellationToken cancellationToken)
   at System.Net.Http.HttpContent.<CopyToAsync>g__WaitAsync|56_0(ValueTask copyTask)
   at System.Net.Http.HttpConnection.SendRequestContentAsync(HttpRequestMessage request, HttpContentWriteStream stream, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnection.SendRequestContentWithExpect100ContinueAsync(HttpRequestMessage request, Task`1 allowExpect100ToContinueTask, HttpContentWriteStream stream, Timer expect100Timer, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken)
   at Amazon.Runtime.Internal.HttpHandler`1.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RedirectHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.S3.Internal.AmazonS3ResponseHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Signer.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.S3.Internal.AmazonS3ExceptionHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.MetricsHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.S3.Transfer.Internal.SimpleUploadCommand.ExecuteAsync(CancellationToken cancellationToken)
   at STP.BlobStorage.Aws.AwsS3Repo.UploadAsync(Stream sourceStream, String fullPath, String contentType, CancellationToken cancellationToken

Reproduction Steps

public async Task UploadAsync(Stream sourceStream, string fullPath, string contentType, CancellationToken cancellationToken = default(CancellationToken))
{
    Guard.ValidateFullPath(fullPath, "fullPath");
    try
    {
        TransferUtility transferUtility = new TransferUtility(_s3Client);
        await transferUtility.UploadAsync(sourceStream, _bucketName, fullPath, cancellationToken);
        _logger.LogDebug("Uploaded stream to {fullPath}", fullPath);
    }
    catch (AmazonServiceException ex3)
    {
        AmazonServiceException ex2 = ex3;
        _logger.LogError(ex2, "Got AmazonServiceException in UploadAsync while uploading object to '{fullPath}'.", fullPath);
        throw new StorageProviderException($"Got AmazonServiceException in {"UploadAsync"} while uploading object to '{fullPath}'.", ex2);
    }
    catch (Exception ex4)
    {
        Exception ex = ex4;
        _logger.LogError(ex, "Got exception in UploadAsync while uploading object to '{fullPath}'.", fullPath);
        throw new StorageException($"Got exception in {"UploadAsync"} while uploading object to '{fullPath}'.", ex);
    }
}

Possible Solution

No response

Additional Information/Context

When I revert my project to .NET6 the issue disappears. If needed I can provide additional information

AWS .NET SDK and/or Package version used

AWSSDK.S3, Version=3.3.0.0

Targeted .NET Platform

.NET8

Operating System and version

Windows 11, Debian 12

The text was updated successfully, but these errors were encountered:

bhoradc · 2024-02-27T23:44:41Z

Hi @julian-dimitroff,

Thank you for reporting the issue. Unfortunately, I am unable to reproduce the exception using your code sample.

Can you please elaborate on how the sourceStream is populated? Preferably, if you can enable AWS SDK verbose logging that might also be helpful here. You can turn on verbose logging using the code below to see detailed logs.

Amazon.AWSConfigs.LoggingConfig.LogResponses = Amazon.ResponseLoggingOption.Always;
Amazon.AWSConfigs.LoggingConfig.LogTo = Amazon.LoggingOptions.SystemDiagnostics;
Amazon.AWSConfigs.AddTraceListener("Amazon", new System.Diagnostics.ConsoleTraceListener());

Regards,
Chaitanya

julian-dimitroff · 2024-02-28T07:32:19Z

Hello,
The sourceStream is coming from a multipart form in an api request:

var boundary = Request.GetBoundary();
var reader = new MultipartReader(boundary, Request.Body, _bufferSize);
MultipartSection section;
while ((section = await reader.ReadNextSectionAsync(cancellationToken)) != null)
{
    var contentDisposition = section.GetContentDispositionHeader();
    if (contentDisposition.IsFileDisposition())
    {
        var fileSection = section.AsFileSection();

        _logger.LogInformation("Received file to upload: {0}", fileSection.FileName);

        Validate.FormFile(fileSection.FileStream, fileSection.FileName, fileSection.Section.ContentType);
        var stream = new TypedStream
        {
            ContentType = fileSection.Section.ContentType,
            ContentStream = fileSection.FileStream,
            Filename = fileSection.FileName
        };
        await _documentManager.UploadDocument(tenantId, new DocumentReference(documentId), name, stream);
    }
}

the _documentManager.UploadDocument calls:

public async Task UploadDocument(Guid tenantId, IDocumentReference document,
    string name, ITypedStream stream)
{
    //... other code logic
    await _storageManager.UploadContentStream(tenantId, document.DocumentId, name, stream);
}

the _storageManager.UploadContentStream calls:

public async Task UploadContentStream(Guid tenantId, Guid id, string name, ITypedStream typedStream, CancellationToken cancellationToken = default)
{
	
	var fullPath = $"{tenantId}/{id}/{name.ToLower()}";
	await _blobStorageRepo.UploadAsync(typedStream.ContentStream, fullPath, typedStream.ContentType, cancellationToken);
	_logger.LogInformation("Content stream has been uploaded");
}

the _blobStorageRepo.UploadAsync is from a company's private nuget package that I do not have access to the code, I only can decompile and see what is called there (this is why I have the code snippet from the original question), so there is no possibility to turn on the detailed logging. Is there a way I can turn on this in this situation?

julian-dimitroff · 2024-02-28T08:47:52Z

One more thing. I altered the code

var boundary = Request.GetBoundary();
var reader = new MultipartReader(boundary, Request.Body, _bufferSize);
MultipartSection section;
while ((section = await reader.ReadNextSectionAsync(cancellationToken)) != null)
{
    var contentDisposition = section.GetContentDispositionHeader();
    if (contentDisposition.IsFileDisposition())
    {
        var fileSection = section.AsFileSection();

        _logger.LogInformation("Received file to upload: {0}", fileSection.FileName);

        Validate.FormFile(fileSection.FileStream, fileSection.FileName, fileSection.Section.ContentType);
        var stream = new TypedStream
        {
            ContentType = fileSection.Section.ContentType,
            ContentStream = fileSection.FileStream,
            Filename = fileSection.FileName
        };
        await _documentManager.UploadDocument(tenantId, new DocumentReference(documentId), name, stream);
    }
}

to this:

var fStream = System.IO.File.OpenRead(@"C:\original0001.pdf");
var stream = new TypedStream
{
    ContentType = "application/pdf",
    ContentStream = fStream,
    Filename = "original0001.pdf"
};
await _documentManager.UploadDocument(tenantId, new DocumentReference(documentId), name, stream);

Basically I do not read the stream from the multipart form from the request but read a local file, and the Upload is successful. If this can help narrow the possible cause of the problem.

Regards,

acorbin-vdm · 2024-03-04T16:57:41Z

Hello,
I recently upgraded the nuget package to latest version and I have this same issue since.

I downgraded the package to 3.7.305.2 and that's fine.
I'm really interrested by this fix too.

The use case is same: upload from multipart form data without store anything into memory (case of big files like 10GB) and upload chunks to S3 directly without know the content length in advance.

All the best,
Adrien

laifharwood · 2024-03-05T20:35:23Z

I'm also seeing the same issue.

laifharwood · 2024-03-05T21:23:11Z

Looks like TransferUtility doesn't support streams where CanSeek is false. https://jasonterando.medium.com/net-core-tee-streaming-and-buffered-s3-uploads-4a063230d99f

bhoradc · 2024-03-05T23:10:16Z

Hi @julian-dimitroff,

Thanks for providing more details. Support for uploading unseekable streams in Transfer Utility was added in S3 (3.7.202.0). You may want to upgrade to this version to confirm that it's an unseekable stream issue.

However, S3 (3.7.305.8) may have introduced some conflicts with the above feature release. Will discuss this with the team and keep you posted.

Regards,
Chaitanya

JFSiller · 2024-03-08T04:30:49Z

@bhoradc

Hi!

I noticed that the latest version of the AWSSDK.S3 package where this - sort of - works is: 3.7.305.7 while also using .NET 8

From then onwards we get the aforementioned error.

It is also worth noting that the following conditions must be met for this to work (please verify before trying them):

Buffering needs to be disabled for the particular request.
Synchronous operations must be allowed (this seems to be a problem within the UploadAsync call with TransferUtility related to Synchronous operations are disallowed. Call ReadAsync or set AllowSynchronousIO to true instead. #1534)

Hope this helps in any sort of way.

ashishdhingra · 2024-03-12T21:48:07Z

@julian-dimitroff Good afternoon. Using the latest version of AWSSDK.S3 (version 3.7.305.31) in an ASP.NET core project targeting .NET 8.0, doesn't reproduce the issue using the code below, using 500KB file (kindly note that code is only for demonstration purposes):
Views > Home > Index.cshtml

@{
    ViewData["Title"] = "Home Page";
}

<div class="text-center">
    <form method="post" enctype="multipart/form-data" asp-controller="FileUpload" asp-action="Index">
        <div class="form-group">
            <div class="col-md-10">
                <p>Upload one or more files using this form:</p>
                <input type="file" name="files" multiple />
            </div>
        </div>
        <div class="form-group">
            <div class="col-md-10">
                <input type="submit" value="Upload" />
            </div>
        </div>
    </form>
</div>

HomeController.cs

using Microsoft.AspNetCore.Mvc;
using TestWebAppNetCore.Models;

namespace TestWebAppNetCore.Controllers
{
    public class HomeController : Controller
    {
        public IActionResult Index()
        {
            return View();
        }
    }
}

FileUploadController.cs

using Amazon.Runtime;
using Amazon.S3;
using Amazon.S3.Transfer;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.WebUtilities;
using System;
using System.ComponentModel.DataAnnotations;
using System.Threading;
using System.Xml.Linq;

namespace TestWebAppNetCore.Controllers
{
    public class FileUploadController : Controller
    {
        [HttpPost("FileUpload")]
        public async Task<IActionResult> Index(List<IFormFile> files)
        {
            long size = files.Sum(f => f.Length);

            foreach (var formFile in files)
            {
                if (formFile.Length > 0)
                {
                    AmazonS3Client amazonS3Client = new AmazonS3Client();
                    await UploadAsync(amazonS3Client, "testbucket-issue3201", formFile.OpenReadStream(), formFile.FileName, formFile.ContentType);
                }
            }

            return Ok(new { count = files.Count, size });
        }

        public async Task UploadAsync(AmazonS3Client amazonS3Client, string bucketName, Stream sourceStream, string fullPath, string contentType, CancellationToken cancellationToken = default(CancellationToken))
        {
            try
            {
                TransferUtility transferUtility = new TransferUtility(amazonS3Client);
                await transferUtility.UploadAsync(sourceStream, bucketName, fullPath, cancellationToken);
            }
            catch (AmazonServiceException ex3)
            {
                AmazonServiceException ex2 = ex3;
                throw new Exception($"Got AmazonServiceException in {"UploadAsync"} while uploading object to '{fullPath}'.", ex2);
            }
            catch (Exception ex4)
            {
                Exception ex = ex4;
                throw new Exception($"Got exception in {"UploadAsync"} while uploading object to '{fullPath}'.", ex);
            }
        }
    }
}

Kindly note that in above code I'm relying on ASP.NET Model binding feature to bind incoming files to list of IFormFile objects in file upload controller.

Could you please:

Check if the issue goes away with the latest version of SDK.
Share the base minimum code to reproduce the issue (removing any internally used packages or code base)
List of NuGet packages used in the project.

Thanks,
Ashish

julian-dimitroff · 2024-03-13T07:51:22Z

As I said in the previous posts: "the _blobStorageRepo.UploadAsync is from a company's private nuget package that I do not have access to the code, I only can decompile and see what is called there". The only thing that I can verify is the version on the AWSSDK.S3 version, which is:

I'll ask my management to contact whoever is responsible to that package, to update the version. Since I be able to test it, I'll write the result here.
Thanks for your quick response

JFSiller · 2024-03-15T23:11:35Z

@julian-dimitroff Good afternoon. Using the latest version of AWSSDK.S3 (version 3.7.305.31) in an ASP.NET core project targeting .NET 8.0, doesn't reproduce the issue using the code below, using 500KB file (kindly note that code is only for demonstration purposes): Views > Home > Index.cshtml

@{
    ViewData["Title"] = "Home Page";
}

<div class="text-center">
    <form method="post" enctype="multipart/form-data" asp-controller="FileUpload" asp-action="Index">
        <div class="form-group">
            <div class="col-md-10">
                <p>Upload one or more files using this form:</p>
                <input type="file" name="files" multiple />
            </div>
        </div>
        <div class="form-group">
            <div class="col-md-10">
                <input type="submit" value="Upload" />
            </div>
        </div>
    </form>
</div>

HomeController.cs

using Microsoft.AspNetCore.Mvc;
using TestWebAppNetCore.Models;

namespace TestWebAppNetCore.Controllers
{
    public class HomeController : Controller
    {
        public IActionResult Index()
        {
            return View();
        }
    }
}

FileUploadController.cs

using Amazon.Runtime;
using Amazon.S3;
using Amazon.S3.Transfer;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.WebUtilities;
using System;
using System.ComponentModel.DataAnnotations;
using System.Threading;
using System.Xml.Linq;

namespace TestWebAppNetCore.Controllers
{
    public class FileUploadController : Controller
    {
        [HttpPost("FileUpload")]
        public async Task<IActionResult> Index(List<IFormFile> files)
        {
            long size = files.Sum(f => f.Length);

            foreach (var formFile in files)
            {
                if (formFile.Length > 0)
                {
                    AmazonS3Client amazonS3Client = new AmazonS3Client();
                    await UploadAsync(amazonS3Client, "testbucket-issue3201", formFile.OpenReadStream(), formFile.FileName, formFile.ContentType);
                }
            }

            return Ok(new { count = files.Count, size });
        }

        public async Task UploadAsync(AmazonS3Client amazonS3Client, string bucketName, Stream sourceStream, string fullPath, string contentType, CancellationToken cancellationToken = default(CancellationToken))
        {
            try
            {
                TransferUtility transferUtility = new TransferUtility(amazonS3Client);
                await transferUtility.UploadAsync(sourceStream, bucketName, fullPath, cancellationToken);
            }
            catch (AmazonServiceException ex3)
            {
                AmazonServiceException ex2 = ex3;
                throw new Exception($"Got AmazonServiceException in {"UploadAsync"} while uploading object to '{fullPath}'.", ex2);
            }
            catch (Exception ex4)
            {
                Exception ex = ex4;
                throw new Exception($"Got exception in {"UploadAsync"} while uploading object to '{fullPath}'.", ex);
            }
        }
    }
}

Kindly note that in above code I'm relying on ASP.NET Model binding feature to bind incoming files to list of IFormFile objects in file upload controller.

Could you please:

Check if the issue goes away with the latest version of SDK.
Share the base minimum code to reproduce the issue (removing any internally used packages or code base)
List of NuGet packages used in the project.

Thanks, Ashish

Could you please retry but, this time, use a stream instead of a FormFile as a parameter. This issue happens mainly when working with the stream directly (such as in multipart/content) and then trying to upload a stream that has a Length of 0.

julian-dimitroff · 2024-03-17T15:24:22Z

Okay, I'll try to setup a sample project that is able to reproduce the problem without this internal nuget package.
I'll get back to you when this is done

Have a great day!

julian-dimitroff · 2024-03-21T07:19:11Z

Hello again @JFSiller @ashishdhingra I made a simple setup using .NET 8 and AWSSDK.S3 v.3.7.307:

But I still got the error with the content length.
I'll attach the simple project, hope you can run it!
The request to the endpoint is made with VS Code's extension "Thunder Client":

The file that I'm using to upload is in the .zip archive

Hope that helps reproduce the problem
AWS Tests.zip

Best regards

ashishdhingra · 2024-03-21T19:14:29Z

@julian-dimitroff Thanks for providing the reproduction code and steps. I was able to reproduce the issue where it gave error Unable to write content to request stream; content would exceed Content-Length. with the following stack trace:

   at System.IO.Stream.<<CopyToAsync>g__Core|27_0>d.MoveNext()
   at System.Net.Http.HttpContent.<<CopyToAsync>g__WaitAsync|56_0>d.MoveNext()
   at System.Net.Http.HttpConnection.<SendRequestContentAsync>d__61.MoveNext()
   at System.Net.Http.HttpConnection.<SendRequestContentWithExpect100ContinueAsync>d__62.MoveNext()
   at System.Net.Http.HttpConnection.<SendAsync>d__57.MoveNext()
   at System.Net.Http.HttpConnectionPool.<SendWithVersionDetectionAndRetryAsync>d__89.MoveNext()
   at System.Net.Http.DiagnosticsHandler.<SendAsyncCore>d__10.MoveNext()
   at System.Net.Http.HttpClient.<<SendAsync>g__Core|83_0>d.MoveNext()
   at Amazon.Runtime.HttpWebRequestMessage.<GetResponseAsync>d__20.MoveNext()
   at Amazon.Runtime.Internal.HttpHandler`1.<InvokeAsync>d__9`1.MoveNext()
   at Amazon.Runtime.Internal.RedirectHandler.<InvokeAsync>d__1`1.MoveNext()
   at Amazon.Runtime.Internal.Unmarshaller.<InvokeAsync>d__3`1.MoveNext()
   at Amazon.S3.Internal.AmazonS3ResponseHandler.<InvokeAsync>d__1`1.MoveNext()
   at Amazon.Runtime.Internal.ErrorHandler.<InvokeAsync>d__5`1.MoveNext()
   at Amazon.Runtime.Internal.ErrorHandler.<InvokeAsync>d__5`1.MoveNext()
   at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__9`1.MoveNext()
   at Amazon.Runtime.Internal.Signer.<InvokeAsync>d__1`1.MoveNext()
   at Amazon.S3.Internal.S3Express.S3ExpressPreSigner.<InvokeAsync>d__5`1.MoveNext()
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.<InvokeAsync>d__2`1.MoveNext()
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.<InvokeAsync>d__2`1.MoveNext()
   at Amazon.Runtime.Internal.CredentialsRetriever.<InvokeAsync>d__7`1.MoveNext()
   at Amazon.Runtime.Internal.RetryHandler.<InvokeAsync>d__10`1.MoveNext()
   at Amazon.Runtime.Internal.RetryHandler.<InvokeAsync>d__10`1.MoveNext()
   at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__9`1.MoveNext()
   at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__9`1.MoveNext()
   at Amazon.S3.Internal.AmazonS3ExceptionHandler.<InvokeAsync>d__1`1.MoveNext()
   at Amazon.Runtime.Internal.ErrorCallbackHandler.<InvokeAsync>d__5`1.MoveNext()
   at Amazon.Runtime.Internal.MetricsHandler.<InvokeAsync>d__1`1.MoveNext()
   at Amazon.S3.Transfer.Internal.SimpleUploadCommand.<ExecuteAsync>d__10.MoveNext()
   at AWS_Tests.Controllers.WeatherForecastController.<UploadDocumentWithTenantId>d__6.MoveNext() in D:\source\repros\AWS.Tests\AWS Tests\AWS.Tests\Controllers\WeatherForecastController.cs:line 89

Based on stack trace, TransferUtility correctly used simple upload based on the file size. We would investigate the issue at our end.

Thanks,
Ashish

ashishdhingra · 2024-03-21T21:00:32Z

@julian-dimitroff Thanks for providing the reproduction code and steps. I was able to reproduce the issue where it gave error Unable to write content to request stream; content would exceed Content-Length. with the following stack trace:

   at System.IO.Stream.<<CopyToAsync>g__Core|27_0>d.MoveNext()
   at System.Net.Http.HttpContent.<<CopyToAsync>g__WaitAsync|56_0>d.MoveNext()
   at System.Net.Http.HttpConnection.<SendRequestContentAsync>d__61.MoveNext()
   at System.Net.Http.HttpConnection.<SendRequestContentWithExpect100ContinueAsync>d__62.MoveNext()
   at System.Net.Http.HttpConnection.<SendAsync>d__57.MoveNext()
   at System.Net.Http.HttpConnectionPool.<SendWithVersionDetectionAndRetryAsync>d__89.MoveNext()
   at System.Net.Http.DiagnosticsHandler.<SendAsyncCore>d__10.MoveNext()
   at System.Net.Http.HttpClient.<<SendAsync>g__Core|83_0>d.MoveNext()
   at Amazon.Runtime.HttpWebRequestMessage.<GetResponseAsync>d__20.MoveNext()
   at Amazon.Runtime.Internal.HttpHandler`1.<InvokeAsync>d__9`1.MoveNext()
   at Amazon.Runtime.Internal.RedirectHandler.<InvokeAsync>d__1`1.MoveNext()
   at Amazon.Runtime.Internal.Unmarshaller.<InvokeAsync>d__3`1.MoveNext()
   at Amazon.S3.Internal.AmazonS3ResponseHandler.<InvokeAsync>d__1`1.MoveNext()
   at Amazon.Runtime.Internal.ErrorHandler.<InvokeAsync>d__5`1.MoveNext()
   at Amazon.Runtime.Internal.ErrorHandler.<InvokeAsync>d__5`1.MoveNext()
   at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__9`1.MoveNext()
   at Amazon.Runtime.Internal.Signer.<InvokeAsync>d__1`1.MoveNext()
   at Amazon.S3.Internal.S3Express.S3ExpressPreSigner.<InvokeAsync>d__5`1.MoveNext()
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.<InvokeAsync>d__2`1.MoveNext()
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.<InvokeAsync>d__2`1.MoveNext()
   at Amazon.Runtime.Internal.CredentialsRetriever.<InvokeAsync>d__7`1.MoveNext()
   at Amazon.Runtime.Internal.RetryHandler.<InvokeAsync>d__10`1.MoveNext()
   at Amazon.Runtime.Internal.RetryHandler.<InvokeAsync>d__10`1.MoveNext()
   at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__9`1.MoveNext()
   at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__9`1.MoveNext()
   at Amazon.S3.Internal.AmazonS3ExceptionHandler.<InvokeAsync>d__1`1.MoveNext()
   at Amazon.Runtime.Internal.ErrorCallbackHandler.<InvokeAsync>d__5`1.MoveNext()
   at Amazon.Runtime.Internal.MetricsHandler.<InvokeAsync>d__1`1.MoveNext()
   at Amazon.S3.Transfer.Internal.SimpleUploadCommand.<ExecuteAsync>d__10.MoveNext()
   at AWS_Tests.Controllers.WeatherForecastController.<UploadDocumentWithTenantId>d__6.MoveNext() in D:\source\repros\AWS.Tests\AWS Tests\AWS.Tests\Controllers\WeatherForecastController.cs:line 89

Based on stack trace, TransferUtility correctly used simple upload based on the file size. We would investigate the issue at our end.

Thanks, Ashish

@julian-dimitroff While debugging, I noticed that fileSection.FileStream.Length has value 0. The issue persists in .NET 7.0. However, the code works fine if using .NET 6.0, where fileSection.FileStream.Length has value 17939.

One weird behavior noticed is that if I temporarily copy the contents of fileSection.FileStream to a MemoryStream, it causes fileSection.FileStream.Length property to be populated.

In other words, copy this code:

var memoryStream = new MemoryStream();
fileSection.FileStream.CopyTo(memoryStream);

after

ValidateFormFile(fileSection.FileStream, fileSection.FileName, fileSection.Section.ContentType);
fileSection.FileStream.Position = 0;
...

So there appears to be issue with Microsoft.AspNetCore.WebUtilities package (targeting .NET 7.0 and .NET 8.0) where the FileMultipartSection.FileStream.Length is missing value initially. Most likely this is an issue which should be reported to Microsoft.

Thanks,
Ashish

julian-dimitroff · 2024-03-22T07:00:06Z

@ashishdhingra Thank you again for your effort.
Please advice should I report the problem, or you can do it? My concern is if i report it I will not be able to explain in detail what is going on.

Best regards,
Yulian

ashishdhingra · 2024-03-22T17:40:04Z

@ashishdhingra Thank you again for your effort. Please advice should I report the problem, or you can do it? My concern is if i report it I will not be able to explain in detail what is going on.

Best regards, Yulian

@julian-dimitroff We have reviewed this created backlog item for this to be groomed for taking further course of action.

julian-dimitroff · 2024-03-25T06:52:27Z

@ashishdhingra Again thank you for your time and effort!

Regards,
Julian

julian-dimitroff added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Feb 27, 2024

bhoradc self-assigned this Feb 27, 2024

bhoradc added s3 and removed needs-triage This issue or PR still needs to be triaged. labels Feb 27, 2024

bhoradc added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. p3 This is a minor priority issue labels Feb 27, 2024

github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Feb 29, 2024

bhoradc added needs-review p1 This is a high priority issue and removed p3 This is a minor priority issue labels Mar 5, 2024

ashishdhingra added the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Mar 12, 2024

ashishdhingra added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. and removed response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. labels Mar 13, 2024

github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Mar 16, 2024

ashishdhingra unassigned bhoradc Mar 21, 2024

ashishdhingra added p2 This is a standard priority issue queued and removed p1 This is a high priority issue needs-review labels Mar 22, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TransferUtility.UploadAsync() throws error: "content would exceed Content-Length" when update to .NET8 #3201

TransferUtility.UploadAsync() throws error: "content would exceed Content-Length" when update to .NET8 #3201

julian-dimitroff commented Feb 27, 2024

bhoradc commented Feb 27, 2024

julian-dimitroff commented Feb 28, 2024

julian-dimitroff commented Feb 28, 2024

acorbin-vdm commented Mar 4, 2024

laifharwood commented Mar 5, 2024

laifharwood commented Mar 5, 2024

bhoradc commented Mar 5, 2024

JFSiller commented Mar 8, 2024 •

edited

ashishdhingra commented Mar 12, 2024

julian-dimitroff commented Mar 13, 2024

JFSiller commented Mar 15, 2024

julian-dimitroff commented Mar 17, 2024

julian-dimitroff commented Mar 21, 2024

ashishdhingra commented Mar 21, 2024 •

edited

ashishdhingra commented Mar 21, 2024 •

edited

julian-dimitroff commented Mar 22, 2024

ashishdhingra commented Mar 22, 2024

julian-dimitroff commented Mar 25, 2024

TransferUtility.UploadAsync() throws error: "content would exceed Content-Length" when update to .NET8 #3201

TransferUtility.UploadAsync() throws error: "content would exceed Content-Length" when update to .NET8 #3201

Comments

julian-dimitroff commented Feb 27, 2024

Describe the bug

Expected Behavior

Current Behavior

Reproduction Steps

Possible Solution

Additional Information/Context

AWS .NET SDK and/or Package version used

Targeted .NET Platform

Operating System and version

bhoradc commented Feb 27, 2024

julian-dimitroff commented Feb 28, 2024

julian-dimitroff commented Feb 28, 2024

acorbin-vdm commented Mar 4, 2024

laifharwood commented Mar 5, 2024

laifharwood commented Mar 5, 2024

bhoradc commented Mar 5, 2024

JFSiller commented Mar 8, 2024 • edited

ashishdhingra commented Mar 12, 2024

julian-dimitroff commented Mar 13, 2024

JFSiller commented Mar 15, 2024

julian-dimitroff commented Mar 17, 2024

julian-dimitroff commented Mar 21, 2024

ashishdhingra commented Mar 21, 2024 • edited

ashishdhingra commented Mar 21, 2024 • edited

julian-dimitroff commented Mar 22, 2024

ashishdhingra commented Mar 22, 2024

julian-dimitroff commented Mar 25, 2024

JFSiller commented Mar 8, 2024 •

edited

ashishdhingra commented Mar 21, 2024 •

edited

ashishdhingra commented Mar 21, 2024 •

edited