Migrate entities from RealmObject to IRealmObject

[softlion]

Hello, I switched to IRealmObject in an existing app that was using RealmObject.

When I run the app, it triggers a migration.

It looks like it's loosing everything.

I switched some fields from "string" to "string?" after migrating to IRealmObject.
What is the correct way to migrate, keep the old object definition using RealmObject and use it to read from old realm and transfer to the new during the migration ?

[papafe]

Hi @softlion,

If you are just migrating to IRealmObject and you didn't change the schema, then there should be no migration involved.

Regarding strings:

• with RealmObject you could specify that the string was never supposed to be null ("non-nullable") by using the [Required] attribute;
• with IRealmObject if the nullability context is enabled, you can just use the nullability annotation to specify that (string for non-nullable, string? for nullable). If the nullability context is not enabled, then you need to use the [Required] attribute as before?

Would it possible for you to post here some of the classes before/after? So that we can take a look and understand what is happening.

[softlion]

if the nullability context is enable
This is enabled by default, no ?

In the original definition, no field were required.
So I added string? to make them optional in the new definition, except for the key field.

So that does change the schema as the key is now non nullable.
I thought the migration would automatically copy the content of the field of the existing table.
It looks like it does not, and think it is a completely new table.

I will create a unit test tomorrow to verify what exactly happens.

After:

    public partial class DbUserInfo : IRealmObject
    {
        public string? DisplayName { get; set; }
        public string Login { get; set; }
        public string? Password { get; set; }
        
        public string? Authorization { get; set; }
    }

Before:

    public class DbUserInfo : RealmObject
    {
        public string DisplayName { get; set; }
        public string Login { get; set; }
        public string Password { get; set; }
        
        public string Authorization { get; set; }
    }

Migration code: does nothing.

Result after migration: no more data in the DbUserInfo table.

[papafe]

Ok, I see your point.

I think that the issue here is that in the previous schema you had Login as nullable, while in the new schema it is non-nullable. This triggers a migration, but it should not remove the whole content from the database. The only thing happening now is that the Login variable should be empty.

How are you checking that there is no more data in the table?

[softlion]

Can I ask you why login will be empty ?

[softlion]

Ok I created a unit test, and obviously login is empty in the migration callback for NewRealm. I don't understand why, as the field goes from string to string?; but ok.

So how to handle this migration gracefully ?

It looks like I can't read the value of Login in the old realm in the migration callback.

[softlion]

Here the unit test using xUnit.

 [Fact]
        public async Task TestMigrateUserInfo()
        {
            var path = Path.GetTempFileName();
            try
            {
                {
                    using var oldDb = Realm.GetInstance(new RealmConfiguration(path)
                    {
                        SchemaVersion = 1,
                        Schema = new RealmSchema.Builder().Add(typeof(OldDbUserInfo))
                    });
                    oldDb.Error += (sender, args) => output.WriteLine(args.Exception.ToLongString());

                    oldDb.Write(() => { oldDb.Add(new OldDbUserInfo { Login = "a@a.com", Authorization = "ok" }); });
                    var allItems = oldDb.All<OldDbUserInfo>().ToList();
                    Assert.Single(allItems);
                    Assert.Equal("a@a.com", allItems[0].Login);
                    Assert.Equal("ok", allItems[0].Authorization);
                }

                using var db = Realm.GetInstance(new RealmConfiguration(path)
                {
                    SchemaVersion = 2,
                    Schema = new RealmSchema.Builder().Add(typeof(DbUserInfo)),
                    MigrationCallback = (migration, version) =>
                    {
                        //var allItems = migration.OldRealm.All<DbUserInfo>().ToList(); //Unable to get OldDbUserInfo nor DbUserInfo
                        var newItems = migration.NewRealm.All<DbUserInfo>().ToList();
                        var i = 0;
                    } 
                });

                output.WriteLine("waiting initial notification");
            }
            finally
            {
                if(File.Exists(path))
                    File.Delete(path);
            }
        }

    [MapTo("DbUserInfo")]
    public class OldDbUserInfo : RealmObject
    {
        public string Login { get; set; }
        public string Authorization { get; set; }
    }

    public class DbUserInfo : IRealmObject
    {
        public string Login { get; set; }
        public string? Authorization { get; set; }
    }

[softlion]

I was able to migrate using the dynamic api. Will try it on iOS, not sure it will work as dynamic may not be available on iOS.

                        var oldItem = migration.OldRealm.DynamicApi.All("DbUserInfo").FirstOrDefault();
                        Assert.NotNull(oldItem);
                        if (oldItem != null)
                        {
                            var oldLogin = (string?)oldItem.Login;
                            Assert.NotNull(oldItem.Login);
                            Assert.Equal("a@a.com", oldItem.Login);

                            var newItem = migration.NewRealm.All<DbUserInfo>().First();
                            Assert.Equal("", newItem.Login);
                            newItem.Login = oldLogin!;
                        }

[papafe]

The reason why it's nulla it's because it goes from being nullable to non-nullable, so we can't really say what should happen if you have a null string in one of your objects, for instance.
And yes, you're correct, in this case you would need a migration method to handle this gracefully, you can see more info about it in the docs.

Regarding iOS, you can actually use dynamic, but you can't do any kind of code generation (that are not doing behind the scenes), so you should be fine.

[softlion]

Well what should happen is loose no user data.

So switching a null into a string.empty and keeping everything else looks so obvious and make it so easy to use Realm that I did not even imagine it could be something different that would require a mildly complicated custom code.

[nirinchev]

While this sounds reasonable, it may not be what the developer intended and doing an automatic migration like that has the potential for information loss. Let's say you have objects with both empty string and null values for a field. You accidentally remove the nullability annotation for the field and we do an automatic migration where we set all null values to "". You realize your mistake and you re-add the nullability annotation. At this point, there's no way to differentiate between objects that had null and objects that had "" prior to the migration. While this may be 1% or even 0.1% of the cases, we prefer to err on the side of safety than to do something magical under the hood that has the potential to put the developer in a tough spot.

I guess there's an argument to be made that we could do migration if we detect that there are no null values for that field, but that, again, is risky as it makes it possible for things to work just fine during development but start crashing in production.

[papafe]

Just to add to what @nirinchev said, you are actually not losing any data. You are still able to access the previous value of the properties in the migration callback.

[softlion]

I understand your point. About the issue in production you are talking about, you know that if I reported that with so much weight, it’s because I discovered this issue in production and was upset.That has been probably hidden during development.

about the percentage of chance that it could happen, It looks like it’s 99% chance vs 1%.

But ok if you prefer to protect the 1%. I would have asked the community to vote. But it’s your decision.

The migration callback does give the old data but only with the Dynamic version. It’s very tricky to understand that. There is a caveat with dynamic, you can’t know if a property exists or not. You must access it and catch the exception.

[papafe]

I can understand that it could be tricky to figure out how to work with migrations, but I hope that the documentation does a good job at explaining how to deal with it.
Regarding the dynamic part, it is build like that exactly for the reason that generally developers don't have access to the old schema, so it's not possible to have a strongly typed API.