Expose ability to customize JSON serialization

[christopheranderson]

Removed public ObjectMapper references which let users customize JSON serialization.

This was due to Azure/azure-cosmosdb-java#153 need a better option.

[dkroehan]

After reading the following blog post: https://devblogs.microsoft.com/cosmosdb/difference-between-null-and-undefined/ I was wondering how to achieve this with the Java SDK.
The default ObjectMapper is configured to always serialize null fields.

I was stumbling across this when adding a ttl field to my entity class. I want to set a custom TTL when updating a particular Cosmos DB item, the majority of items will never need a custom TTL and can simply use the container TTL. Therefore, the field would be null.

But if I set the ttl field to null I get the following error:

The input ttl 'null' is invalid. Ensure to provide a nonzero positive integer less than or equal to '2147483647', or '-1' which means never expire.

The documentation here states that setting ttl to null works, although not quite sure if null has the same meaning there.

[neiser]

@kushagraThapar I'd like to see some solution here, as this really limits some advanced use cases of the SDK. So maybe you can move it out of the "parking lot" again and assign it a higher prio?

For example, you cannot use Kotlin classes out-of-the-box as https://github.com/FasterXML/jackson-module-kotlin is not registered in the static ObjectMapper hidden in the SDK. This forces you to annotate Kotlin classes manually as a very ugly workaround to get deserialization of immutable data classes working:

data class Email @JsonCreator constructor(
    @param:JsonProperty("email") val email: String // note the redundant specification of "email"
)

Or do some hack like calling Utils.getSimpleObjectMapper().registerModule(new KotlinModule()) on that static ObjectMapper instance at an early point of the application lifecycle.

For @dkroehan a workaround would be to use @get:JsonInclude(JsonInclude.Include.NON_NULL) (Kotlin) on the ttl class field.

[kushagraThapar]

@xinlian12 - thoughts on this issue ?

[alzimmermsft]

I don't know if this is still being actively looked into, but there is an example of this in azure-search-documents. There is an interface hierarchy in azure-core with ObjectSerializer as the base interface with a sub-interface JsonSerializer, and an experimental interface AvroSerializer in azure-core-experimental, which allows for a customer designed serializer to be passed into an API call or service client. For azure-search-documents the service can return JSON documents which can have a few different handlings on the service client:

• Document is retrieved with no specified Java type, a generic JSON Map<String, Object> call SearchDocument is returned.
• A document type is specified but JsonSerializer wasn't configured in SearchClientBuilder, JacksonAdapter (or effectively JacksonAdapter) is used to deserialize the JSON document to the type.
• A JsonSerializer was specified in SearchClientBuilder and a document type is specified, the passed JsonSerializer is used to deserialize the customer's document.

There are two implementation packages for JsonSerializer in azure-core-serializer-json-gson and azure-core-serializer-json-jackson which a customer could use out of the box to pass their GSON or Jackson serializer to handle their JSON document type. Customers can also roll their own implementation if they choose to and there is also the ability to use service provider interfaces to load their own implementation from the classpath similar to what is seen with HttpClient implementations.

[neiser]

@alzimmermsft Without having looked into details, am I getting it right that using a custom JsonSerializer is currently only possible with SearchClientBuilder and you're mentioning it here to achieve such a customization for users of the SDK? Or is this actually a general solution and that issue can be closed?

[alzimmermsft]

@alzimmermsft Without having looked into details, am I getting it right that using a custom JsonSerializer is currently only possible with SearchClientBuilder and you're mentioning it here to achieve such a customization for users of the SDK? Or is this actually a general solution and that issue can be closed?

It would be the former, where some libraries offer support for JsonSerializer. I was mentioning it here as a possible solution going forward for the Cosmos libraries, so this issue should remain open.

As additional context SearchClient added support for JsonSerializer after initial release and was able to add support without changing previous runtime behavior by falling back to the internal serialization used by Azure SDKs if JsonSerializer wasn't provided.

[mmcdermottMS]

Bumping this thread. I have run into an issue where the Cosmos SDK failed to serialize a LocalDate within a repository query (resulting in a missing jsr310 module exception). I tried the hack that neiser mentioned above which partially resolved things, but would rather let the customer specify their own serializer as they can with the ACS SDK as mentioned by alzimmermsft. Happy to open a new issue to track, if that makes sense.

[kushagraThapar]

@mmcdermottMS - FYI, I have recently added the jsr310 module to Java SDK. You should be able to leverage it in the latest release of azure-cosmos Java V4 SDK.

[petercooperjr-spls]

Hi, I think this is a related-enough issue to post it here: I'm struggling using GeoJson points in Spring Data for Cosmos. It looks like there's a class designed for just plugging in and using, com.azure.core.models.GeoPoint, but the ObjectMapper set up for serializing for Spring Data defined in com.azure.spring.data.cosmos.core.convert.ObjectMapperFactory doesn't have the serializers for it configured. (In addition to other weird choices like not turning off SerializationFeature.WRITE_DATES_AS_TIMESTAMPS even though ISO-format is the preferred format for storing in Cosmos, but that's a side note.) There's a quite comprehensive separate com.azure.core.implementation.jackson.ObjectMapperFactory that looks to be used internally, and which includes adding a GeoJsonSerializer & GeoJsonDeserializer, but all of those classes are package-local and so can't be easily used when defining my own cosmosObjectMapper.

I wrote a terrible hack using reflection to call the GeoJsonSerializer.getModule() anyway to use while defining my cosmosObjectMapper (though I don't know if I actually want to check it in), but it seems like having the SDK expose better options for customizing is really the way to go here. Thanks.

I'm using spring-cloud-azure-starter-data-cosmos 4.4.1 with Java 11.

[kushagraThapar]

@petercooperjr-spls - I think it would be better to create a separate issue for the problem you are facing. We can better investigate it that way.

[Xaseron]

Another hack would be to use your own serializer and persist the json tree.

        val jsonTree = myObjectMapper.valueToTree<JsonNode>(myKtDataClass)
        val itemResponse = cosmosContainer.createItem(jsonTree)

[kushagraThapar]

Just wanted to update this issue, we are building functionality to customize JSON serializability - #38997

Apologize for the delay in this development of this feature.

[github-actions]

Hi @christopheranderson, we deeply appreciate your input into this project. Regrettably, this issue has remained unresolved for over 2 years and inactive for 30 days, leading us to the decision to close it. We've implemented this policy to maintain the relevance of our issue queue and facilitate easier navigation for new contributors. If you still believe this topic requires attention, please feel free to create a new issue, referencing this one. Thank you for your understanding and ongoing support.

[kushagraThapar]

This has been done. Implementation details in this PR - #38997