OpenRead(path) disposed Stream immediately

[momcina08]

i have this piece of code:
try
{
_sftpClient = new SftpClient(host, port, username, password);
_sftpClient.Connect();
return _sftpClient.OpenRead(path);
}
catch (Exception ex)
{
_logger.LogError(ex, "Error occured while connecting to sftp server.");
throw;
}

i checked the parameters, i open the File with FileZilla, but the SftpFileStream is unreadable after i exit this method. Even when i tried to forward the Stream to a different method it throws me " Seek operation failed"

[momcina08]

also if i have this code:
var sftpClient = new SftpClient(host, port, username, password);
sftpClient.Connect();
var stream = sftpClient.OpenRead(path);

var x = stream.CanSeek;
var y = stream.Position;
var z = stream.Length;

return stream;

i get exception if i try to read the length.

[Rob-Hague]

OpenRead opens a file handle on the remote side, it is not doing any downloading so seeking and getting the length of the stream do not make a lot of sense.

If you want a seekable stream you could use DownloadFile or read the stream into memory with e.g. OpenRead(..).CopyTo(myStream)

[momcina08]

But what if i want to pass that stream to a different method and use StreamReader on it? I always get that stream is not seekable, i dont want to persist it in memory or download it, only to read and process it.

[Rob-Hague]

I don't think StreamReader does any seeking, so there must be something else going on. You would need to show more code.

[momcina08]

in the next method i do this:

StreamReader streamReader = new(request.Stream); //request.Stream is the SftpFileStream
var reader = new JsonTextReader(streamReader);

reader.Read(); //this throws the " Seek operation failed" and that file handle is closed

[Rob-Hague]

Please post the full code and the full exception with stack trace. It's difficult to deduce anything from small snippets of code

[momcina08]

namespace MyNamespace.Handlers.Interfaces;

using Microsoft.Extensions.Logging;
using Renci.SshNet;
using System.IO;

public interface ISftpStreamConnector
{
Task GetStream(string path, string username, string password, int port, string host);
Task CloseStream();
}

public class SftpStreamConnector : ISftpStreamConnector
{
private readonly ILogger _logger;
private SftpClient _sftpClient;

public SftpStreamConnector(ILogger<SftpStreamConnector> logger)
{
    _logger = logger;
}

public async Task<Stream> GetStream(string path, string username, string password, int port, string host)
{
    try
    {
        _sftpClient = new SftpClient(host, port, username, password);
        _sftpClient.Connect();
        var stream = _sftpClient.OpenRead(path);
        return stream;
    }
    catch (Exception ex)
    {
        _logger.LogError(ex, "Error occured while connecting to sftp server.");
        throw new StreamProtocolException("Failed to obtain stream from sftp server.");
    }
}

public async Task CloseStream()
{
    _sftpClient.Disconnect();
    _sftpClient.Dispose();
}

}

////

namespace MyNamespace.Handlers;

using MyNamespace.Domain;
using MyNamespace.Exceptions;
using MyNamespace.Handlers.Interfaces;
using MyNamespace.Requests;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
using System;
using System.IO.Compression;
using System.Text;

public interface IJsonToNdJsonHandler
{
Task Handle(string path, string username, string password, int port, string host);
}

public class JsonToNdJsonHandler : IJsonToNdJsonHandler
{
private readonly ILogger _logger;
private readonly ISftpStreamConnector _sftpStreamConnector;

public JsonToNdJsonHandler(
    ILogger<JsonToNdJsonHandler> logger,
   ISftpStreamConnector sftpStreamConnector)
{
    _logger = logger;
    _sftpStreamConnector= sftpStreamConnector;
}

public async Task<StreamFeedResult> Handle(string path, string username, string password, int port, string host)
{
   var stream = await _sftpStreamConnector.GetStream(path, username, password, port, host);
   StreamReader streamReader = new(stream);
    var reader = new JsonTextReader(request.StreamReader);

    try
    {
        reader.Read();

        .....
    }
    catch (Exception ex)
    {
        _logger.LogError(ex, $"Failed to parse json file. {ex.Message}");
        throw;
    }
    finally
    {
        request.StreamReader.Close();
    }
}

}

////errors
Seek operation failed.
at Renci.SshNet.Sftp.SftpFileStream.get_Length()

[Rob-Hague]

////errors
Seek operation failed.
at Renci.SshNet.Sftp.SftpFileStream.get_Length()

Surely there are more lines in this stack trace?

[momcina08]

Yes, but they are all trace of my outer classes up until this invocation, nothing relevant. Please, give it a try and you'll see.

[Rob-Hague]

I'm sorry, I'm happy to try to help you diagnose the problem but I'm not going to spend my free time diagnosing it for you.

I don't see anything obviously wrong with your code. I suggest you start by tracking down what is calling Length on the stream. Is it your code, SSH.NET code, Newtonsoft.JSON or something else? Exactly which class and method?

If you find anything interesting, feel free to report back and we can see what to do from there.

[Soulusions]

I've run into a very similar issue so I'll start off by using this issue rather than open yet another one.

app.MapGet("/{*path}", async (CancellationToken token, string path) =>
{
    using var client = new SftpClient(connectionInfo);

    await client.ConnectAsync(token);

    if (!client.Exists(path))
    {
        return Results.NotFound("File not found");
    }

    var attr = client.GetAttributes(path);
    if (attr.IsRegularFile)
    {
        using (var stream = await client.OpenAsync(path, FileMode.Open, FileAccess.Read, token))
        {
            return Results.Stream(stream, "application/octet-stream", path.Split("/").Last());
        }
    }

    return Results.NotFound("File not found");
});

Trying to actually access the endpoint though, I get the following dumped:

ObjectDisposedException: Cannot access a disposed object.
Object name: 'Renci.SshNet.Sftp.SftpFileStream'.

Stack:
Renci.SshNet.Sftp.SftpFileStream.CheckSessionIsOpen()
Renci.SshNet.Sftp.SftpFileStream.ReadAsync(byte[] buffer, int offset, int count, CancellationToken cancellationToken)
System.Threading.Tasks.ValueTask<TResult>.get_Result()
System.Runtime.CompilerServices.ValueTaskAwaiter<TResult>.GetResult()
Microsoft.AspNetCore.Http.StreamCopyOperationInternal.CopyToAsync(Stream source, Stream destination, Nullable<long> count, int bufferSize, CancellationToken cancel)
Microsoft.AspNetCore.Internal.FileResultHelper.WriteFileAsync(HttpContext context, Stream fileStream, RangeItemHeaderValue range, long rangeLength)
Microsoft.AspNetCore.Http.HttpResults.FileStreamHttpResult.ExecuteAsync(HttpContext httpContext)
Microsoft.AspNetCore.Http.HttpResults.FileStreamHttpResult.ExecuteAsync(HttpContext httpContext)

If you need any more info, feel free to @ me, though in this case there really isn't much to this program other than this.
I've also tried not using the handler's cancellationToken thinking that was simply cancelling too early but that didn't work either so I'm kinda out of ideas.

[Rob-Hague]

@Soulusions your stream is being disposed immediately upon return but it is only read from after your request handler delegate is executed. See the code for Results.Stream:

https://github.com/dotnet/aspnetcore/blob/32b772a470e3c7dc147d3d124c32199bb3e7b6bc/src/Http/Http.Results/src/Results.cs#L403
https://github.com/dotnet/aspnetcore/blob/32b772a470e3c7dc147d3d124c32199bb3e7b6bc/src/Http/Http.Results/src/TypedResults.cs#L403
https://github.com/dotnet/aspnetcore/blob/32b772a470e3c7dc147d3d124c32199bb3e7b6bc/src/Http/Http.Results/src/FileStreamHttpResult.cs#L113

You probably need to pull the file to a local MemoryStream, or don't tie the SSH session to the API request. I don't work with ASP.NET much

[momcina08]

OpenRead or OpenAsync lose their purpose if we cant do anything with the stream outside of the method. Stream purpose is to not inflate the memory but processing on the fly for example. If i pull it into MemoryStream i am overloading my machine RAM with couple of gigabytes potentially. So yeah, not the greatest thing.

[Rob-Hague]

It is the same with System.IO.File.OpenRead. The stream is just a handle to the file. Where are you expecting the bytes to go if you are closing the handle before using it?