Ability to rename collections and fields

[alrick]

Hey there !

It would be great to have the ability to rename fields of a collection.

In the meantime, what's the procedure to rename them from the database ?

1. Rename the field in the correspondent table
2. Rename the field in the directus_fields table

Any other places I should consider checking ?

Thanks !

[benhaynes]

If you go through all the system collection field name references you should be ok:

• directus_collection_presets.view_query — Deep nested
• directus_fields.field
• directus_permissions.read_field_blacklist
• directus_permissions.write_field_blacklist
• directus_relations.field_many
• directus_relations.field_one
• directus_relations.junction_field
• directus_revisions.data
• directus_revisions.delta

We'll be adding a feature to automate this in an upcoming version...

[rokdd]

This list is not really updated.. it is a good start but there are some more fields.. even the problem is that the schema of knex need to migrated afterwards and reload the whole app (or is there any workaround?). I am just trying to write some piece of code to replace it in templates and all the other occurences :-(

[rijkvanzanten]

even the problem is that the schema of knex need to migrated afterwards and reload the whole app (or is there any workaround?)

I'm confused, what do you mean by "schema of knex need to be migrated"?

[rokdd]

When I rename the field, I need to rename it in all collections and finally I also need it to rename the database column (with knex)?

[rijkvanzanten]

Oh gotcha. Yes, the database table / column has to be renamed to. How you do that is up to you (doesn't necessarily have to be through Knex, though that's the thing that Directus already uses, so it's easier to reuse that than try to roll something custom)

[rokdd]

Exactly.. but up is done when directus startup, else the queries are failing. So what I did now, that I setup the docker container to restart and kill the app after I rename the field :-)

[manuelwehrli]

It would be good to also include the ability to rename collections and the underlying database tables.

[alexey-yarmosh]

What I do to rename the collections along with the associated tables:

• generate the latest schema using directus@latest schema snapshot;
• in the generated yaml file copy the collections, fields and relations of the initial table with the name of a new collection;
• create the clone collection using directus@latest schema apply;
• copy data using migration INSERT INTO new_table (col1, col2, ...) SELECT col1, col2, ... FROM initial_table;
• update permissions table:

knex('directus_permissions').where('collection', 'initial_table').update({
  collection: 'new_table',
})

• remove the initial_table either by editing the yaml file or on UI.

[mikolajkazmierczak]

Hi @alexey-yarmosh !

Does this really work for you without any side effects? I am looking for a temporary solution to this issue until it's in the official version.

I was thinking about creating a script that would automatically rename a collection, and if your approach works well I'd really appreciate your help.

Kudos!

[alexey-yarmosh]

Hey, creating and SQL script renaming all references of the original name to the new name may be an easier option if you want to automate it.

I was doing it manually so leaving as many operations to directus cli was safer for me.

[xgenvn]

I think we need an option to disable autosave the changes until we are done designing all related fields. The autosaving is helpful for multiple developers trying to modeling, but I think it's uncommon. I'm quite curious about the reason behind autosaving the schema though.

[benhaynes]

If I remember correctly, the schema changes interrelate, so they need to be done live, and not staged for saving later.

[rijkvanzanten]

I think we need an option to disable autosave the changes until we are done designing all related fields.

Are you running into any particular issues because of it autosaving?

[xgenvn]

@rijkvanzanten when designing a schema, more than often I'd like to change the schema name, field name, and field type. With the current schema editor, I need to remove and add again which is counter-intuitive. Also, I hope there is a similar way to support a code-first solution, which can produce JSON schema then import to Directus (perhaps it's already there but I missed it).

[rijkvanzanten]

Also, I hope there is a similar way to support a code-first solution, which can produce JSON schema then import to Directus (perhaps it's already there but I missed it).

Funny that you mention it, I just opened a PR to add this in 😄

As for your request, sounds like we should focus on supporting field renames rather than trying to stage all the changes (the difficulty with that is that fields can rely on other fields relationally across collections, which is near impossible to stage ahead of time and save as a single batch)

[xgenvn]

@rijkvanzanten I understand the complexity, there is no way to support what I hope for in near future (especially crossing collection relationship modeling). Field rename and field type changing lively is quite sufficient in most use cases imo.

[colemahatmccreary]

Any update on enabling field renames? Also (sort of related), the ability to update a record's PK and have the change cascade across relationships? @rijkvanzanten

Would make refactoring 100x simpler!

[benhaynes]

Unfortunately, we can not provide estimates on features or fixes. As of now, the only way to get an accurate timeline or release date is to sponsor this task. We recognize that this may be an important feature/fix, but we are a small open-source organization with a lot to triage and complete. If you can't sponsor this ticket, then the next best thing is to increase its priority by giving it a 👍 . Or contribute a pull-request... after all, this is OSS!

---

This is an automated response.

[GioKaaja]

Hi all! This feature is extremely needed by our team given the necessity to remodel Directus fields. Based on this, we reached out to Directus support team through Driscol and asked for a quotation to get this done. Price is very limited (below $3k) and I'd like to know if anyone is interested in joining to share the bill for getting this done; TBH dont know if this message respects new ideas' rules (in that case, sorry!!) and I'd like to deep dive with all the interested people and also Directus support team! Cheers

[GioKaaja]

I take @benhaynes heart reaction as a yes :) if anyone is interested, please reply here below this message!

[benhaynes]

I'll almost always throw $100 towards any feature that passes the 80/20 rule, and would be really happy to see this one added. I'd sponsor more, but it's not really "sponsoring" when it comes from internal team members. 😉

[sebescudie]

Would be interested as well!

[carwei]

@GioKaaja I'm willing to join in. How do we get this done?

[Christilut]

I'm building a Directus instance that maps to fields in a website template, so we can easily deploy multiple websites based on Directus content, this works really well.

By far, the biggest issue I've had with this so far is that I can't rename fields. Sometimes naming changes or there was a typo or another reason but then I have to delete that field and re-create it which is a super big hassle, not to mention it can only be done before content has been added since it would lead to data loss otherwise.

I understand it's not so simple to rename a field because of internally named fields but I think for advanced users who are building something more than a basic content site, this is a bit of a problem.

The workaround now is to write custom API calls that download content & field metadata, delete the field, re-create the field and then add the content again. This gets increasingly complex if there are relationships to this data.

Just dropping my situation about it here.

[rijkvanzanten]

Yeah, makes total sense! And don't get me wrong, we do want to support renaming fields/collections eventually, just hard to prioritize this one with other (sponsored) work going on 🙂

Groetjes 👋🏻

[rokdd]

Well I wrote some code to rename "simple" field by a custom API. But I also noticed that it is more complicated as it looks like. There are settings like views, the revision data or the title templates of fields / collections / .. (which I do not cover with my code)

[drennvinn]

I understand the complexity of this feature but it would clearly be great if this feature could be implemented!
We all agree that naming is a headache in our devs because we often have to make changes (often because of customers 😅)

[benhaynes]

It's not the hardest feature to implement, nor the easiest. Maybe we can find a company to sponsor it to jump start building it?

[beasteers]

Since this has been a recurring problem, I thought I'd document how I've had to deal with it.

I've had some issues with modifying schemas outside of directus where I add or remove columns using SQL and it can completely break directus's snapshot imports (e.g. when column configurations exist inside directus, but the actual column doesn't or is different). So I'd like to have a set of known safe ways to do these changes. (I'm biased towards postgres, but hopefully these apply across dbs)

The main downside with this is that it's manual and you lose any field configuration and all that, but at least you keep the data.

Changing a column data type

Example: converting my_table.some_column JSON to STRING (e.g. multi-dropdown to single dropdown)

-- pgadmin: copy the column
alter table my_table add column tmp varchar;
-- your data type conversion could happen here
-- TODO - jsonb to csv conversion
update my_table SET tmp = some_column;

-- directus: now we can delete the column in directus

-- back to pgadmin: rename the column back to the old one
alter table deployments rename column tmp to some_column;

-- directus: go configure the field correctly

Renaming a column

renaming my_table.old_column to new_column

-- pgadmin: copy the column
alter table my_table add column new_column varchar;
update my_table SET new_column = old_column;

-- directus: now we can delete the column in directus and re-configure the new field

These feel a bit obvious after writing them out, but I wasn't sure about it until I did some trial and error so hopefully this will help someone.

Renaming a collection

... 🥸

converting jsonb to csv string (when changing data type)

-- when assigning the tmp column, you can convert to csv like this
update my_table SET tmp = array_to_string(
    ARRAY(select * from jsonb_array_elements_text(some_column::jsonb)),
    ',');

-- verify
select tmp from my_table;

If you have additional ways of retaining/migrating schema changes it would be cool to document and build upon them here until we can work out a canonical way of doing it. Selfishly as well, I hope that if we can flesh out the workflows here, it would be easier for any one of us to put out a PR.

I guess I'll leave some open questions:

• what use cases do these not cover, and how do you get around them?
• is doing things this way okay? do you have better ways of doing these?
• if you think "we should edit all those directus tables instead!" let's draft those queries 👏🥳

[rokdd]

Well I did it with JS-SDK:

       replace_variables: function replace_variables(objSource, objReplacer) {
            var pattern = this.replace_variables.pattern;
            var path = this.replace_variables.path;
            if (typeof objSource === "object") {
                if (objSource === null) return null;

                if (objSource instanceof Array) {
                    for (var i = 0; i < objSource.length; i++) {
                        objSource[i] = this.replace_variables(objSource[i], objReplacer);
                    }
                } else {
                    for (var property in objSource) {
                        let property_new = property.replace(pattern, function (finded, varName) {
                            return varName in objReplacer ? objReplacer[varName] : finded;
                        });
                        objSource[property] = this.replace_variables(objSource[property], objReplacer);
                        if (property_new != property) {
                            objSource[property_new] = objSource[property]
                            delete objSource[property]
                        }
                    }
                }
                return objSource;
            }

            if (typeof objSource === "string") {
                //console.log(objSource, pattern)
                return objSource.replace(pattern, function (finded, varName) {
                    return varName in objReplacer ? objReplacer[varName] : finded;
                });

            }
            return objSource;
        },
        exists_field: async function (req, collection, name) {
            let fs = await new services.FieldsService({ schema: req.schema, accountability: req.accountability });
            try {
                await fs.readOne(collection, name)
            }
            catch
            {
                return false
            }
            return true
        },
        rename_field: async function (req, collection, name_old, name_new) {
            let transformers = [
                {
                    /*
                          |id |bookmark|user                                |role|collection             |search|layout|layout_query                                                                              |layout_options                                            |refresh_interval|filter|icon            |color|
            |---|--------|------------------------------------|----|-----------------------|------|------|------------------------------------------------------------------------------------------|----------------------------------------------------------|----------------|------|----------------|-----|
            |4  |        |9a99f6c7-2727-4c92-aa50-f5b0810ca926|    |objecttypes_objecttypes|      |      |{"tabular":{"page":1,"fields":["id","objecttypes_id.name","related_objecttypes_id.name"]}}|{"tabular":{"widths":{"objecttypes_id":211.111083984375}}}|                |      |bookmark_outline|     |
          */
                    collection: "directus_presets",
                    filter: { collection: collection },
                    tests: {},
                    fields: {
                        "layout_query": {
                            json_path: ["*", "fields"],
                            method: "replace_as_name",

                        },
                        "layout_options": {
                            json_path: ["*", "widths"],
                            method: "replace_as_name"
                        },
                        //connecte fields not working here.. then it is with dot
                        "filter": {
                            json_path: ["*"],
                            method: "replace_as_name"
                        }
                    }
                }, {
                    /* FOR EING TABLES
              */
                    collection: "directus_presets",
                    filter: {},
                    tests: {},
                    fields: {
                        "layout_query": {
                            json_path: ["*", "fields"],
                            method: "replace_as_compl",

                        },
                        "layout_options": {
                            json_path: ["*", "widths"],
                            method: "replace_as_compl"
                        },
                        //connecte fields not working here.. then it is with dot
                        "filter": {
                            json_path: ["*"],
                            method: "replace_as_compl"
                        }
                    }
                }, {
                    collection: "directus_fields",
                    filter: { collection: collection, field: name_old },
                    tests: { "rows_count__min": 1, "rows_count__max": 1 },
                    fields: {
                        "field": { method: "update" }
                    }
                },
                {
                    collection: "directus_relations",
                    filter: { many_collection: collection, many_field: name_old },
                    tests: {},
                    fields: {
                        "many_field": { method: "update" }
                    }
                },
                {
                    collection: "directus_relations",
                    filter: { one_collection: collection, one_field: name_old },
                    tests: {},
                    fields: {
                        "one_field": { method: "update" }
                    }
                },
                {
                    collection: "directus_relations",
                    filter: { one_collection: collection, junction_field: name_old },
                    tests: {},
                    fields: {
                        "junction_field": { method: "update" }
                    }
                },
                {
                    collection: "directus_permissions",
                    filter: { collection: collection },
                    tests: {},
                    fields: {
                        "permissions": {
                            json_path: ["*", "widths"],
                            method: "replace_as_name"
                        },
                        "validation": {
                            json_path: ["*", "widths"],
                            method: "replace_as_name"
                        },
                        "presets": {
                            json_path: ["*", "widths"],
                            method: "replace_as_name"
                        },
                        "fields": {
                            json_path: ["*", "widths"],
                            method: "replace_as_name"
                        },
                        "display_options": {
                            /*      "options": {
                   "template": "{{related_objecttypes_id.name}}"*/
                            json_path: ["*", "template"],
                            pattern: "\\{\\{(" + collection + "\\." + name_old + ")\\}\\}",
                            method: "replace_custom"
                        },
                    }
                },
                {
                    collection: "directus_revisions",
                    filter: { collection: collection },
                    tests: {},
                    fields: {
                        "data": {
                            json_path: ["*"],
                            method: "replace_as_name"
                        },
                        "delta": {
                            json_path: ["*"],
                            method: "replace_as_name"
                        },

                    }
                },


            ]



            for (const [iter, table] of Object.entries(transformers)) {
                logger.debug("Map " + table.collection + "")
                let items = await new ItemsService(table.collection, { schema: req.schema, accountability: null });
                let rows = await items.readByQuery({ filter: table.filter })
                console.log(table.filter)
                if (table.tests.hasOwnProperty("rows_count__max") && table.tests["rows_count__max"] < rows.length) {
                    logger.error("More rows as expected in " + table.collection)
                    return false
                }

                if (table.tests.hasOwnProperty("rows_count__min") && table.tests["rows_count__min"] > rows.length) {
                    logger.error("Less rows as expected in " + table.collection)
                    return false
                }
                //END TESTS
                if (rows.length > 0) {
                    for (let [row_id, row] of Object.entries(rows)) {
                        for (const [field, field_config] of Object.entries(table.fields)) {
                            logger.debug(table.collection + " " + field)
                            if (field_config.hasOwnProperty("json_path")) {
                                if (field_config.method == "replace_as_name")
                                    this.replace_variables.pattern = new RegExp("^(" + name_old + ")$", "gi")
                                else if (field_config.method == "replace_as_attr2")
                                    this.replace_variables.pattern = new RegExp("\\b(" + name_old + ")\\.", "gi")
                                else if (field_config.method == "replace_as_attr_placeholder")
                                    this.replace_variables.pattern = new RegExp("\\b(" + collection + "\\." + name_old + ")", "gi")
                                else if (field_config.method == "replace_as_compl")
                                    this.replace_variables.pattern = new RegExp("((\\." + collection + "\\." + name_old + "\\b)|(\\b" + collection + "\\." + name_old + "\\b))", "gi")
                                else if (field_config.hasOwnProperty("pattern"))
                                    this.replace_variables.pattern = new RegExp(field_config.pattern, "gi")
                                row[field] = this.replace_variables(row[field], { [name_old]: name_new })
                                console.log(field, JSON.stringify(row[field]))
                            }
                            else if (field_config.method == "update") {
                                row[field] = name_new
                            }
                        }
                        console.log(row)
                        items.updateOne(row.id, row)
                    }

                }
                else
                    console.error("No rows for " + table.collection)
            }
            await database.schema.table(collection, function (table) {
                table.renameColumn(name_old, name_new)
            })
            /*
            TODO: tempaltes?
            TODO: translations?
                */
        }
    }

[beasteers]

Super cool thanks for sharing! 💕 What has been your experience with this? What cases does it work for and are there caveats/cases to watch out for? (I see templates and translations are outstanding issues).

also out of curiosity, do you run this from an extension?

[rokdd]

Super cool thanks for sharing! 💕 What has been your experience with this? What cases does it work for and are there caveats/cases to watch out for? (I see templates and translations are outstanding issues).
Yes the problem is that the templating titles you can setup in the relationships need to be replaced as well . And thats kind of triggy because the field names might the same between collections and then you have to make sure to replace only the one you really want to change.

also out of curiosity, do you run this from an extension?
Yes
Another limitation is that you need to restart directus after renaming, because else it has some problems with knex. I did not tried to disable the cache of the schema.

[rijkvanzanten]

Heya! Thanks for opening this feature request!

This feature request has received over 15 votes from the community. This means we'll move this feature request to the Under Review state!

The Core team will schedule a meeting to review this request as soon as possible. The discussion will then be approved or denied. You may or may not be invited to join this meeting with the core team.

For more information, see our Feature Request Process.

[vwasteels]

Hello Directus team !
Do you have any update on this feature ?
Thanks !!

[Natetronn]

Very much looking forward to this one!

[rijkvanzanten]

Spent an hour or so thinking through this on today's request review live stream session trying to shape this up into a proper RFC! Quite a lot to update across the database on change, so some questions around migration and adoption strategy still. In the works!

Here's a live link to my our in-works RFC; i'll publish that here for archiving as well when it's done 🙂

https://directus.notion.site/Rename-Collections-Fields-3427b5e21d914e4abbdae53c48169f01?pvs=4

[rijkvanzanten]

Realistically the only way to support this natively is to start relying on UUIDs for collections/fields instead of the database key. Trying to update the name in like 30 places every time you want to change the key feels very error prone

[Natetronn]

Realistically the only way to support this natively is to start relying on UUIDs for collections/fields instead of the database key. Trying to update the name in like 30 places every time you want to change the key feels very error prone

Sorry to make light, but I couldn't resist lol

Anyway, if us developers default to using UUIDs on our own in the future, how would that "lighten our load"? Can you elaborate on that it a bit further, por favor? How would it look compared to the ~20 some odd changes it takes now?

Or is this something that would have to be done by default at some point down the road to work?

[rijkvanzanten]

Or is this something that would have to be done by default at some point down the road to work?

It's really just this! If we use UUIDs in the system tables (eg in those ~20 places) instead of keys, and just store the key with the UUID in directus_collections/fields, we only have to change it in that one place when you want to change the key of the collection.

[Natetronn]

Is it fair to assume the bulk of the change/issues aren't so much about moving to UUIDs for two collections and is more about migrations and backwards compatibility? Are there other reason why people might want to use keys instead of UUID, at this point (for those two collections or any collections, for that matter?) Are all databases capable of the handling this change? (I've only ever used SQLite, postgres and MySQL so far). Is it more about forcing people with external databases, ones not setup/built with directus, into making changes to their system before being able to make the move to Directus? I'm just curious what you all are up against here.

[ricardovigatti]

UUID is a great idea.

[carwei]

Guys, I need a AI/ML model that detect the trained words and renamed the words used the trained destination words.
What model i can use for it

I don't know if this is spam or just wildly off-topic.

[msharshath]

For suppose i have some set of data like
A
B
C
D
E

I want to rename the above data as
A to F
B to G
C to H
D to I
E to J

The data is present in CSV file.
So, I need an AI/ML model that reads the data from the file and it should rename as per the above conditions.
So, For that case what model is to used.
As far i know, i should use Supervised ML Learning. But I am not sure which model should pick from the ML model.

[carwei]

Just in case you are not trolling: If think you need ML to solve something this simple it’s probably a good time to ask ChatGTP or just good old Google. It’s not even a Directus related problem.
https://letmegooglethat.com/?q=excel+conditional+replace+formula

[vitalijalbu]

I also want this feature, renaming collections also would be fine

[silasabbott]

For anyone here that is only concerned about what the Directus user sees and not names in the underlying database, you can use field name translations and collection naming translations to change the name throughout the Directus Studio. It doesn't change anything for the underlying database or the API, but it's better than nothing!

[vitalijalbu]

I would like this feature too…👀 Il giorno lun 22 gen 2024 alle 22:15 Rijk van Zanten < ***@***.***> ha scritto:

…

Or is this something that would have to be done by default at some point down the road to work? It's really just this! If we use UUIDs in the system tables (eg in those ~20 places) instead of keys, and just store the key with the UUID in directus_collections/fields, we only have to change it in that one place when you want to change the key of the collection. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[j-vitali]

I always lost data when renaming them 👎

[mikolajkazmierczak]

That's curious because it would seem obvious that if you just replace every occurence of, say, collection-name in the db to new-collection-name would take care of it. But maybe I'm missing something.