Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migration from Kryo 4 to Kryo 5 #796

Open
theigl opened this issue Nov 25, 2020 · 2 comments
Open

Migration from Kryo 4 to Kryo 5 #796

theigl opened this issue Nov 25, 2020 · 2 comments
Assignees

Comments

@theigl
Copy link
Collaborator

theigl commented Nov 25, 2020

In most cases, Kryo 5 and Kryo 4 are not serialization compatible.

If you need to migrate persistent data, the best approach is to have both Kryo 4 and Kryo 5 on the classpath, read old data with Kryo 4 and write it with Kryo 5.

For more information, see https://github.com/EsotericSoftware/kryo/wiki/Migration-to-v5

@theigl theigl self-assigned this Nov 25, 2020
@theigl theigl pinned this issue Nov 25, 2020
@nicknezis
Copy link

Might be a good idea to show the change in FieldSerializer configuration. It seems that FieldSerializerConfig needs to be passed into the constructor instead of FieldSerializer itself exposing setter methods to configure after creation. Not sure if that's a general change that should be mentioned?

@kapilgarg1996
Copy link

kapilgarg1996 commented Feb 6, 2024

Hi @theigl , Posting our discussion here.

In the migration guide, you mentioned to use v4 and v5 together.

if (kryo4Version)
  return kryo4.readObject(input4, ArrayList.class);
else
  return kryo5.readObject(input5, ArrayList.class);

Here, how do we determine the value of kryo4Version ? I thought of the following approach and want your opinion on it.

  1. Add a wrapper around kryo4 and kryo5 and add methods for registration, read, write.
  2. Add a custom class resolver for kryo5.
    a. In writeClass method, add a custom byte with some known value to indicate that this output is written by v5
    b. In readClass, read the byte. and call super.readClass()
  3. Add a custom method isVersion5(Input)​ to return true/false based on the byte value.
  4. Before calling kryo.readClassAndObject(), call isVersion5()​ and use that in the above if/else condition

After some time, all of the data will eventually be v5 only. Does this approach seem fine ? Or do you have some better approach to do this migration ?

Our use case doesn't have any persisted data serialized by kryo, its all transient data present in Queues for short term. Also we have more than 100 classes registered with Kryo so we can't modify every class.

@theigl you mentioned in the mail

It really depends. Your idea is one possibility if your data is stored somewhere (e.g. a DB or a queue) without any wrapper.
If you have some wrapper around Kryo-serialized data that contains metadata (creationDate etc) it makes sense to put the info there or use the creationDate directly to determine the Kryo version.

We actually don't have wrappers around serialised data.

Are there any edgecases I might be missing by adding an extra byte to the output ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

3 participants