Implicit conversion from byte[] to ReadOnlyMemory<byte>

[danielmarbach]

Hi

Some of our models for history reasons are stored with properties that have a byte[] type. As an example

    public class FailedAuditImport
    {
        public string Id { get; set; }
        public FailedTransportMessage Message { get; set; }
    }

    public class FailedTransportMessage
    {
        public string Id { get; set; }
        public Dictionary<string, string> Headers { get; set; }
        public ReadOnlyMemory<byte> Body { get; set; }
    }
    public class FailedAuditImportIndex : AbstractIndexCreationTask<FailedAuditImport>
    {
        public FailedAuditImportIndex()
        {
            Map = docs =>
                from cc in docs
                select new FailedAuditImport
                {
                    Id = cc.Id,
                    Message = cc.Message
                };
        }
    }

now the newly ingested data comes in as ReadOnlyMemory<byte> and we are effectively forced to take a defensive copy before storing by doing a ToArray. Would it be possible to safely switch from byte[] to ReadOnlyMemory<byte> under the assumption the underlying buffers stay around until the ravenclient has stored the documents without causing re-indexing or some other troubles?

[ayende]

You are looking at the code that is running on the database side. And the types you use aren't actually identical to what is compiled. In this case, if you'll look at the index, the output would be something like:

In other words, it elided the actual types.
Note that by default, you need to add a converter to ReadOnlyMemory<byte> to support proper serialization on the client side, otherwise, you'll get:

In order to solve that, you can use:

var serializerConventions = (NewtonsoftJsonSerializationConventions)store.Conventions.Serialization;
serializerConventions.CustomizeJsonSerializer += serializer =>
{
    serializer.Converters.Add(new ByteArrayConverter());
};


store.Initialize();

With the converter being defined as:

public class ByteArrayConverter : JsonConverter
{
    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        if (value == null)
        {
            writer.WriteNull();
            return;
        }

        ReadOnlyMemory<byte> data = (ReadOnlyMemory<byte>)value;

        // Convert the ReadOnlyMemory<byte> to a base64 string
        string base64String = Convert.ToBase64String(data.Span);

        writer.WriteValue(base64String);
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        var s = reader.ReadAsString();
        return new ReadOnlyMemory<byte>(Convert.FromBase64String(s));
    }

    public override bool CanConvert(Type objectType)
    {
        return objectType == typeof(ReadOnlyMemory<byte>);
    }
}

So far, that is pure client side work, and note that this isn't efficient in terms of allocations. I assume that this part you already have and I'm including that just so we can talk in the same manner.

using (var s = store.OpenSession())
{
    s.Store(new FailedAuditImport
    {
        Message = new FailedTransportMessage
        {
            Body = Encoding.UTF8.GetBytes("Hello World"),
            Headers = new Dictionary<string, string>
            {
                ["From"] = "Daniel"
            }
        }
    });
    s.SaveChanges();
}

This would be the output of this code:

Now, what would be the index output in this case?

I'm not sure what is the meaning behind this, though.
What exactly do you want to index, and what sort of queries do you even want to perform on the actual message binary data?

[danielmarbach]

What exactly do you want to index, and what sort of queries do you even want to perform on the actual message binary data?

I was planning to dig deeper there because based on my initial research it is not necessary and we probably should change that index

Thanks for the rest of the answer.