Orleans integration

[Rohansi]

How soon can we expect to get this on NuGet with a separate package to use it with Orleans? I've been putting off switching from the default one because the others are a bit too invasive for my liking.

[ReubenBond]

Thanks for the interest, @Rohansi! I still have some work to do before this serializer can be used as the primary Orleans serializer.

Currently I'm working on a networking layer rewrite for Orleans and as a part of that I'm adding in an interface for message serialization which could allow us to swap in entirely different serializers like this one or Hyperion or protobuf-net.

Regarding "the others are a bit too invasive", what do you mean by that? Some serializers require some markup in the form of attributes on types (to specify field ids and other serialization options). Hagar also requires some markup: currently fields/properties must be marked with [Id(x)] in order to be eligible for serialization.

Hagar also currently has some limitations: it requires fields/properties to be accessible and mutable. This limitation can (and should) be lifted, but for the initial development phase I considered it a fine sacrifice.

[Rohansi]

Good to hear!

I believe when I tried Bond it generated new classes/structs for the schema but I would like for it to use my own. I suppose I could compromise with this one (if it sticks a partial modifier on the generated type).

I'm using a lot of inheritance but protobuf-net requires derived types to explicitly be listed out in an attribute. I suppose there's a good reason for it but it's still a bother.

Hyperion looks interesting, and I didn't give it a try, but it seems version tolerance isn't exactly guaranteed when using it.

I'm just looking for something that can handle upgrades, inheritance, and outputs to something reasonably small. I'm not really in a rush to switch but if you have any ideas for the current serializers I'm willing to give them a try.

For constructors it'd be nice to have something like [JsonConstructor] in Json.NET eventually, but very understandable to not have yet.

Thank you for the help!

[ReubenBond]

I'm using a lot of inheritance but protobuf-net requires derived types to explicitly be listed out in an attribute. I suppose there's a good reason for it but it's still a bother.

Got it. Hagar takes inspiration from Bond for how it encodes inheritance (encoding base fields before subclass fields with a delimiter between each level in the hierarchy). Protocol Buffers is a lowest-common-denominator wire protocol which restricts what Marc can do in terms of supporting .NET's relatively rich type system.

Hyperion looks interesting, and I didn't give it a try, but it seems version tolerance isn't exactly guaranteed when using it.

Yes, AFAICT Hyperion is not actually version tolerant and the (partially implemented?) strategy used to try to support version tolerance (a kind of version cookie) is not tenable in my opinion. Other than version tolerance, though, Hyperion looks like a good serializer.

I'm just looking for something that can handle upgrades, inheritance, and outputs to something reasonably small. I'm not really in a rush to switch but if you have any ideas for the current serializers I'm willing to give them a try.

I don't have any good suggestions at the moment

For constructors it'd be nice to have something like [JsonConstructor] in Json.NET eventually, but very understandable to not have yet.

Good idea. I opened #44 to track it

[stephenlautier]

Is it possible to use Hagar similar to the below yet?

public class OrleansSerializer : ISerializer
{
    private readonly SerializationManager _serializationManager;

    public OrleansSerializer(SerializationManager serializationManager)
    {
        _serializationManager = serializationManager;
    }

    public byte[] Serialize(object raw, Type type)
        => _serializationManager.SerializeToByteArray(raw);

    public object Deserialize(byte[] serializedData, Type type)
        => _serializationManager.Deserialize(type, new BinaryTokenStreamReader(serializedData));
}

Basically I would like to try this to serialize/deserialize PubSub state of Orleans. Thanks!

[ReubenBond]

@stephenlautier yes, it's similar to that now. You need to configure Hagar during startup (.AddHagar(config)) and inject the Hagar.Serializer type, which has (hopefully) user-friendly methods for different source/target types.

[stephenlautier]

Thanks, I gave it a quick go now though not sure if its correct (as im not making use of the type either)

public class HagarOrleansSerializer : ISerializer
{
    private readonly Serializer _serializer;

    public HagarOrleansSerializer(Hagar.Serializer serializer)
    {
        _serializer = serializer;
    }

    public byte[] Serialize(object raw, Type type)
        => _serializer.SerializeToArray(raw);

    public object Deserialize(byte[] serializedData, Type type)
        => _serializer.Deserialize<object>(new ReadOnlySpan<byte>(serializedData));
}

I'm getting the following error:

"Could not find a codec for type Orleans.GrainState`1[Orleans.Streams.PubSubGrainState]."

Can you use is it ala JSON serializer contractless? e.g. without implementing custom serializer/attribute

Any help would be appreciated, maybe we change our store serializer to this :)

[ReubenBond]

Yes, you would need to implement/generate serializers for your types. Hagar is very intentionally not a magic serializer.

You can add a surrogate type, or (assuming you could change the source) add the [Hagar.GenerateSerializer] and add [Hagar.Id(0)] etc to the fields which need to be serialized.

You can also directly implement IFieldCodec<T> or IGeneralizedCodec, but be careful with those options - the raw codecs must follow the correct protocol.

[stephenlautier]

So since I need to serialize Orleans PubSubState the mentioned surrogate type is the way to go, right?

Though not too sure if its possible then what I'd like to achieve, as I want to serialize PubSubGrainState (which is internal in Orleans.Streams); I think to do so you need access to the type, or am I wrong?

If so do you have an example which I can follow?

[ReubenBond]

Hagar is probably not the ideal serializer for that, since it's mostly intended for your own types, and is certainly intended for types you can access from your own code (it's all AOT, except for things which cannot be (ISerializable support)).

Hyperion might be more suitable for that, but it's not version-tolerant (which is probably fine).

[stephenlautier]

@ReubenBond thanks for your help. I will look into that! 🙇

[Rohansi]

@ReubenBond Any update on this? There's been quite a few changes in Orleans where you now need to implement IExternalSerializer and it can support multiple serializers. I tried implementing that interface but can't figure out how to implement IsSupportedType properly - for example, I want it to only return true if there is a codegen'd serializer for the given type.

[ReubenBond]

@Rohansi I've been working on integrating the two. For IsSupportedType, you can implement that in terms of CodecProvider.TryGetCodec(type, out _), since that will tell you whether or not Hagar is able to serialize a type.

[Rohansi]

Cool, that's what I had used too. I figured it wasn't correct when I saw it throwing an exception when serializing some exceptions. I'll need to give it a try again to see what exactly is failing to serialize.