import19 cannot import a moin1.9 wiki farm

[ClaasTeichmann]

I tried to import my running moin1.9 wikifarm (14 wikis and 18GB of data alltogether). I followed the the instructions from the website (https://moin-20.readthedocs.io/en/latest/admin/upgrade.html) and from the code.
I couldn't find any documentation for importing a wikifarm (sorry if I overlooked something!).

Present setup (moin1.9):

File/directory structure:

./farm/farmconfig.py
./farm/wiki1.py
./farm/wiki2.py
./farm/ ....
./farm/common/user/
./farm/wiki1/data/pages/
./farm/wiki1/data/cache/
./farm/wiki1/ ....
./farm/wiki2/data/pages/
./farm/wiki2/data/cache/
./farm/wiki2/ ....

(expected) Future setup

I understood that former wikis of a wikifarm translate into namespaces of a moin2 wiki (please correct me if I am wrong here).
So I would expect the following structure:

./farm/wikiconfig.py
./farm/index
./farm/preview
./farm/sql
./farm/data
./farm/data/default
./farm/data/help-common
./farm/data/help-en
./farm/data/userprofiles
./farm/data/users
./farm/data/wiki1
./farm/data/wiki2
./farm/data/ ...

Problem

It seems that I can convert each of my wikis one by one, but the pages are assigned to the default namespace and not the individual ones, which I already configured in the wikiconfig.py

I can only import one wiki of the wikifarm at once:

moin import19 -d ./moin19_wikifarm/wiki1/data

Suggested solution

1. The import19 could detect the wikifarm automatically and assign the namespaces accordingly:

moin import19 -d ./moin19_wikifarm

2. There could be the option to which namespace the wiki should be converted:

moin import19 --namespace=wiki1 -d ./moin19_wikifarm/wiki1/data
moin import19 --namespace=wiki2 -d ./moin19_wikifarm/wiki2/data
...

I am happy to write the instructions or do further testing if this helps the moin2 development.

[UlrichB22]

I think solution 2 should be the easiest. I will check it out, but it will take some time.
Perhaps you can continue in the meantime with one of your largest or most important wikis to test whether there are any showstoppers. Thanks.

[UlrichB22]

#1609 adds a new parameter --namespace <target_namespace> to the moin import19 command.

To explain behavior of import19 lets assume following pages in moin 1.9:

• user01 # this is identical to an existing user
• user02 # user does not exist
• item01
• item02
• wiki01
• wiki01/subitem01

When you configure a custom namespace wiki01 in your wikiconfig.py and don't specify the new parameter you will get (old behavior):

• users/user01 # moved to users namespace
• user02 # user does not exist, goes to namespace default
• item01
• item02
• wiki01 # moved to wiki01 namespace, URL does not change
• wiki01/subitem01 # moved to wiki01 namespace, URL does not change

When you configure custom namespaces wiki01 and wiki02 in your wikiconfig.py and run moin import19 -d <data_dir> --namespace wiki02 the new structure will be

• users/user01 # moved to users namespace
• wiki02/user02 # user does not exist, goes to target_namespace
• wiki02/item01 # moved to target_namespace
• wiki02/item02 # moved to target_namespace
• wiki01 # moved to wiki01 namespace, URL does not change
• wiki01/subitem01 # moved to wiki01 namespace, URL does not change

Does this meet your expectations? Please also check the links for items inside the new namespace.

[ClaasTeichmann]

@UlrichB22 Sorry for my later reply! I am now back to this topic.
So far it looks good.
I would run the command line for each wiki in the wikifarm:
wiki01->namespace01
wiki02->namespace02
etc.

Is this how it should be? Or would it be better to create a seperate VM with a seperate instance of a wiki, if the EditorGroup contains many different people? I try to directly transfer my wikifarm to namespaces.
There are a few issues, though, which I might better list in the #1609

[UlrichB22]

There is no wiki farm support in moin2.

I would run the command line for each wiki in the wikifarm:
wiki01->namespace01
wiki02->namespace02
etc.

Yes, that's the idea behind the change in #1609, it allows you to load wiki data into different namespaces.

The users and their configurations are stored in the userprofiles namespace in moin2. They are generally available in all namespaces, but you can restrict access to individual namespaces with ACL and groups.

The group configuration for wikigroups is done using items in the default namespace, see https://moin-20.readthedocs.io/en/latest/admin/configure.html#groups. I have never tested creating wikigroups in a custom namespace.

You have to decide whether you want to use different wikis on individual VMs or namespaces. There are several pros and cons.

[UlrichB22]

Regarding the issues in #1609:

Missing directories

I tested following workflow

moin create-instance

Add namespace wiki01 to wikiconfig.py

moin index-create

As a result following directories have been created:

wiki/data/wiki01/data
wiki/data/wiki01/meta

User directories

Using a link as a workaround is a good idea.

Groups

Using individual "EditorGroups" for each namespace is a valid custom configuration.

Missing Link

I need some time to check this, likely this is a bug.

EditorGroup doesn't exist

There is no default item named "EditorGroup". For security reasons, some changes have been made to the standard wikiconfig.py regarding the ACL defaults in the last months. Please check whether you have write permissions in the default namespace.

[ClaasTeichmann]

Thanks for your comments @UlrichB22 ! :-)

My replies to the different topics:

Wikifarm

Yes, I understood that there is no farm in moin2. I was just wondering how to best continue working with the different wikis. As the account-sharing is nice, I think I would go for one wiki with the different namespaces (this is also less admin effort, I hope).

Missing directories

You are right! moin index-create did the trick. I just overlooked this command in the instructions.

User directories

The link works fine. For others, it would maybe be good to describe this potential caveat and how to circumvent it!? If useful, I could also do it (usually, I would create a Wiki-page for that ;-) ).

There are two other things concerning users: I had users without an email-address: import19 worked fine, but the user-browser in moin2 only showed a generic error-message. I found the reason in the uwsgi-log file. Removing the users without email-addresses (some kind of spam) in moin19 and importing again, worked nicely.

Groups

I would create all groups in the default namespace (as suggested and common practice) and not in the individual ones.
If the groups don't exist (those configured in wikiconfig.py), the Groupsbrowser doesn't work and also results in a generic error message (uwsgi-log file shows the reason).

Missing Link

I assume that this is a bug in the import19 skript. It seems that all subpages attachment are not imported correctly into the namespace. I am happy to test again, if you have an updated version.

EditorGroup doesn't exist

I think you were right about the permissions. After I tried again, it worked :-)

Users permissions

I think I discovered a new problem: The imported users do not have the permissions they should have according to the fact that they are part of, e.g., Wiki01EditorGroup (which I defined as default in the wikiconfig.py for this specific namespace):

    'wiki-one': dict(before='ClaasTeichmann:read,write,create,destroy,admin AdminGroup:read,write,create,destroy,admin',
        default='WikiOneEditorGroup:read,write,create All:',
        after='',
        hierarchic=False, ),

But I'll check this further. Probably it is a configuration mistake. I followed these instructions:
https://moin-20.readthedocs.io/en/latest/admin/configure.html#group-backend-configuration