Support custom deserialization

[vicero]

I'd like to use my own optimized JSON deserializer against Algolia, avoiding the performance hit from reflection and garbage collection in Newtonsoft.

One possible solution to this would be to allow injection of HttpTransport into the SearchClient constructors. HttpTransport would need to go from internal to public as a consequence.

[Ant-hem]

Hi @vicero

Thanks for the feedback. This is definitely something I wanted to address in the next version of the library since .NET Core 3 will be shipped with a new (and performant) std serializer. (related to #652)

Regarding the implementation, I was more thinking of only injecting the serializer and not the whole transport layer because the latter is responsible for the retry strategy (which is one of the most important part of the library). I will design a simple interface for the serializer. I'll keep you updated on this issue when we come up with something.

However, in the mean time you can still improve a bit the performance of Newtonsoft with Manual Serialization - Using a reader or writer directly skips any of the overhead from a serializer, such as reflection. You could use it on your POCOs.

Hope this helps.

[vicero]

However, in the mean time you can still improve a bit the performance of Newtonsoft with Manual Serialization - Using a reader or writer directly skips any of the overhead from a serializer, such as reflection

Indeed, I tried this (but with a JsonConverter attribute) but there is still overhead according to the profiler.

I cannot set the contract resolver in the Algolia C# client because the JsonConfig is internal.

[Ant-hem]

What kind of performance issues are you facing? Memory/speed or both?
Are you facing this issue at Indexing time or at Search time?

[vicero]

Speed, you can see in the profiler image above that even with the JsonConverter attribute, newtonsoft is spending a lot of time doing reflection and garbage collection

This is happening on search, just deserializing the result of the http response.

[Ant-hem]

Alright. I'll work on the injection of the serializer in the coming days.

In the meantime, another work-around could be to reduce the size of the response with the responseFields and attributesToRetrieve parameters.

If the tip above didn't work. Would it be possible to have the full dump of the profiler? (If it does not contain any sensitive information indeed). You could share it through our support page mentioning this thread. If you could share a reproducible code snippets + application and index name it would be perfect as well!

[vicero]

Sorry for the late response, but I noticed some movement on deserialization!

In the meantime, another work-around could be to reduce the size of the response with the responseFields and attributesToRetrieve parameters.

I initially had the same idea, however, unfortunately I'm not able to minimize either of those.

I came across this recent article from October 2019 about serializer benchmarks in dotnet 3.0 to hopefully share why I am interested in injecting my own serializer into the algolia client:

https://michaelscodingspot.com/the-battle-of-c-to-json-serializers-in-net-core-3/

Even the latest standard serializer from dotnet is not the most performant, in some cases it's even worse than Newtonsoft.

Hopefully injection of the serializer is still on the horizon, we continue to see spikes in our response times grow under load and tailoring deserialization is the only solution to this.

[morganleroi]

Hello !

We just released a new version of the client (Alpha) on Nuget.

This version now use System.Text.Json as Serializer. If needed, we could also allow customer to inject their own logic if removing Newtonsoft did not solve the issue.

If you have time, we need customer feedback about this new version. It's still not GA so you may not want to use it in production.

There is no more deps on Newtonsoft, all Algolia APIs are now available (Insights, Recommend, Query Perso, ...) and all models and routes are synced.

To help you migrate, there is a Migration guide.

Happy to help in your migration process !