[FC-0049] Handle tags when importing/exporting courses #34356

ChrisChV · 2024-03-12T17:54:46Z

Description

Export: Add tags.csv when exporting a course.
Import: Read tags.csv and creates object_tags depending on the scenario
Changes to use object_tag._export_id
Update export/import tags separator to ;

Supporting information

Github issue, closes [Tagging] Handle tags when importing/exporting courses modular-learning#182
Depends on: [FC- 0049] feat: Features to enable import/export courses openedx-learning#172
Internal ticket: FAL-3624

Testing instructions

Run make requirements on cms shell.
Run make migrate on cms shell.
Make sure you have Taxonomy/Tags data setup locally and verify that the command runs well: https://github.com/open-craft/taxonomy-sample-data/
Create a new/empty course.
Export empty course. Verify that tags.csv is not included in the export files.
Export a Sample Taxonomy Course. Verify that tags.csv is included in the export files. Open and verify the content of the file.

Scenario 1:

Import the new generated export files into the empty course. Verify that the tags are visible.

Scenario 2:

Open tags.csv and add tags that doesn't exist on a taxonomy (FlatTaxonomy in this example). Save.
Import with the edited tags.csv. Verify that the new tags are not visible.
Go to Taxonomy list page. Export FlatTaxonomy as csv and add the new tags. Import.
Verify that the new tag is visible on the course.

Scenario 3:

Open tags.csv and add tags that doesn't exist on a taxonomy that doesn't exist. You need to add a new column with a new export_id. Save.
Import with the edited tags.csv. Verify that the new tags are not visible.
Create a csv file to import a new taxonomy with the new tags
Go to Taxonomy list page. Import the new taxonomy with the same export_id added on tags.csv.
Open Manage organizations menu of the new taxonomy and add it to all organizations.
Verify that the new taxonomy and tags are visible on the course.

Scenario 4: openedx/modular-learning#204

Other information

Merge and bump version: [FC- 0049] feat: Features to enable import/export courses openedx-learning#172

openedx-webhooks · 2024-03-12T17:54:50Z

Thanks for the pull request, @ChrisChV! Please note that it may take us up to several weeks or months to complete a review and merge your PR.

Feel free to add as much of the following information to the ticket as you can:

supporting documentation
Open edX discussion forum threads
timeline information ("this must be merged by XX date", and why that is)
partner information ("this is a course on edx.org")
any other information that can help Product understand the context for the PR

All technical communication about the code itself will be done via the GitHub pull request interface. As a reminder, our process documentation is here.

Please let us know once your PR is ready for our review and all tests are green.

ChrisChV · 2024-03-26T19:20:19Z

openedx/core/djangoapps/content/search/documents.py

@@ -170,10 +170,10 @@ def _tags_for_content_object(object_id: UsageKey | LearningContextKey) -> dict:
    }
    for obj_tag in all_tags:
        # Add the taxonomy name:
-        if obj_tag.name not in result[Fields.tags_taxonomy]:
-            result[Fields.tags_taxonomy].append(obj_tag.name)
+        if obj_tag.taxonomy.name not in result[Fields.tags_taxonomy]:


@bradenmacdonald On openedx/openedx-learning#172 _name has been changed to _export_id, but here I have kept the taxonomy name. That is ok?

Yes, that's right. Here should be the taxonomy name only. Thanks :)

requirements/edx/kernel.in

ChrisChV · 2024-03-26T19:46:53Z

@rpenido @bradenmacdonald About the Scenario 4:

Tags will be imported and visible in Studio

In the tag drawer, by default taxonomies that are not part of the course organization are not displayed. Would this change on import? What I have in mind is to add a field to the ObjectTag: imported, if it is True and meets the conditions of this scenario, it would be displayed in the tag drawer. What do you think?

bradenmacdonald · 2024-03-26T20:35:07Z

@ChrisChV

Scenario 4: Tags in the OLX match a Taxonomy (by Export ID) but the current user does not have permission to view/use that taxonomy [because it's linked to a different org]:
Tags will be imported and visible in Studio. Course author will be able to delete them if they want. Course author will not be able to add additional tags from the same taxonomy though, because they don't have permission.

In the tag drawer, by default taxonomies that are not part of the course organization are not displayed. Would this change on import? What I have in mind is to add a field to the ObjectTag: imported, if it is True and meets the conditions of this scenario, it would be displayed in the tag drawer. What do you think?

No, you don't need to add an imported field. It shouldn't matter whether the tag was imported or not. Basically, any time that there are tags from a taxonomy that the user doesn't have permission to use, it should behave like this (whether imported or not): "Course author will be able to delete them if they want. Course author will not be able to add additional tags from the same taxonomy though, because they don't have permission."

CC @jmakowski1123 to confirm.

ChrisChV · 2024-03-27T17:52:55Z

"Course author will be able to delete them if they want. Course author will not be able to add additional tags from the same taxonomy though, because they don't have permission."

Taxonomies/tags that course author don't have permission to view are currently not displayed. Can't delete. I would have to check if the taxonomy has tags and show it

rpenido · 2024-03-27T20:08:54Z

Export empty course. Verify that tags.csv is not included in the export files.

Hi @ChrisChV ! While testing, the "empty" course has a language tag (auto-tagged). I removed the tag and exported the course again. I got a tags.csv with the following content:

"Name","Type","ID","-1-languages"
"Empty Course","course","course-v1:org1+empty1+001",""

I don't think this is an issue, but it differs from your test instructions (actually, I think this is good because the user can add new tags even if the course has none applied).

Besides that, everything works flawlessly. Good work, @ChrisChV! When you have Scenario 4, let me know so I can review it again.

bradenmacdonald · 2024-03-28T18:44:54Z

Taxonomies/tags that course author don't have permission to view are currently not displayed. Can't delete. I would have to check if the taxonomy has tags and show it

If you want, we can just open a ticket for this on https://github.com/openedx/modular-learning/issues/ and tackle it later.

ChrisChV · 2024-03-28T19:22:27Z

If you want, we can just open a ticket for this on https://github.com/openedx/modular-learning/issues/ and tackle it later.

@bradenmacdonald @rpenido I created open-craft#649 and openedx/frontend-app-course-authoring#926 as a proof of concept of the Scenario 4. I am missing the tests and testing instructions. I will continue it if you agree with the solution 😀

When you add a new tags in a course, the previous tags are not exported How repodruce: - Import a course - Add new tags to the course or blocks - Export a course

bradenmacdonald · 2024-04-01T20:59:55Z

@ChrisChV Is there some way we could do it without changing the taxonomy permissions or retrieving "all taxonomies"? Perhaps just list all the objecttags and if any don't match the "allowed" taxonomies, we can display the rest under a heading like "Other Tags"

ChrisChV · 2024-04-02T19:20:59Z

@bradenmacdonald I have realized that scenario 4 needs more discussion. I created openedx/modular-learning#204 and I continued the discussion there. This PR works and can be merged without Scenario 4 CC @rpenido

bradenmacdonald · 2024-04-03T20:48:18Z

@rpenido Can you please review this? Then I can do a CC review + merge. We can handle "scenario 4" separately.

rpenido

LGTM @ChrisChV ! 👍

I tested this using the Testing Instructions
I read through the code
~~I checked for accessibility issues~~
Includes documentation

This is ready for CC review @bradenmacdonald

openedx/core/djangoapps/content_tagging/api.py

openedx/core/djangoapps/content_tagging/helpers/test_objecttag_export_helpers.py

rpenido · 2024-04-03T21:52:36Z

xmodule/modulestore/xml_exporter.py

+        tags_count = get_object_tag_counts(block_id_pattern)
+        course_tags_count = get_object_tag_counts(courselike_key_str)
+
+        if tags_count or course_tags_count:


I found the cause of having an empty tags.csv in my tests. At the auto-tagging handler, we tag the course (course-v1:...) and the "course block". The latter is not shown anywhere but probably appears in this count.

Could we create a small task to fix this @bradenmacdonald?

@rpenido Sure, thanks!

bradenmacdonald

Thanks, looks good! Just a few minor things and then I'll approve and merge.

bradenmacdonald · 2024-04-05T16:21:17Z

cms/djangoapps/contentstore/management/commands/tests/test_export_olx.py

@@ -61,7 +63,7 @@ def create_dummy_course(self, store_type):
        )
        return course.id

-    def check_export_file(self, tar_file, course_key):
+    def check_export_file(self, tar_file, course_key, withTags=False):


Please use snake case: with_tags

Updated here

bradenmacdonald · 2024-04-05T16:22:30Z

cms/djangoapps/contentstore/views/import_export.py

@@ -343,6 +344,11 @@ def export_handler(request, course_key_string):
    requested_format = request.GET.get('_accept', request.META.get('HTTP_ACCEPT', 'text/html'))

    if request.method == 'POST':
+        if not has_view_object_tags_access(request.user, str(course_key)):
+            raise PermissionDenied(
+                "You do not have permission to view object tags for this object_id."


We're already checking if not has_course_author_access(request.user, course_key): - isn't that enough?

I'm just slightly worried that this will cause some issue where exports start to fail for people who aren't even using tags at all in their course, so I want to play it safe here.

If you want to keep it in, we should just disable the tags.csv file when the user doesn't have permission, rather than making the whole export fail.

We're already checking if not has_course_author_access(request.user, course_key): - isn't that enough?

I think you are right, I will follow this. 👍

If you want to keep it in, we should just disable the tags.csv file when the user doesn't have permission, rather than making the whole export fail.

I think do this will do the code more complex

Yeah, just take it out for now. The existing check for has_course_author_access is sufficient. If you're allowed to export the course, you're allowed to see all the tags.

Updated here

xmodule/modulestore/xml_importer.py

ChrisChV · 2024-04-08T17:12:38Z

@bradenmacdonald I fixed the nits. It's ready for merge

bradenmacdonald · 2024-04-08T17:38:02Z

@ChrisChV There is just a minor pylint issue - please fix :)

openedx-webhooks · 2024-04-08T18:39:49Z

@ChrisChV 🎉 Your pull request was merged! Please take a moment to answer a two question survey so we can improve your experience in the future.

edx-pipeline-bot · 2024-04-08T20:01:12Z

2U Release Notice: This PR has been deployed to the edX staging environment in preparation for a release to production.

edx-pipeline-bot · 2024-04-08T20:18:34Z

2U Release Notice: This PR has been deployed to the edX production environment.

edx-pipeline-bot · 2024-04-08T20:18:34Z

2U Release Notice: This PR has been deployed to the edX production environment.

openedx-webhooks added the open-source-contribution PR author is not from Axim or 2U label Mar 12, 2024

ChrisChV marked this pull request as draft March 12, 2024 17:54

ChrisChV force-pushed the chris/FAL-3624-import-export-courses branch 4 times, most recently from 3305ef2 to adb81c5 Compare March 14, 2024 22:38

ChrisChV added 4 commits March 19, 2024 13:50

refactor: Extract generate tags on csv to a function api

de3decb

feat: Add tags.csv to export tarball

8056ea4

feat: Verify tag count to generate tags.csv

0c74ae2

feat: Add new fields to csv exporter

82dab62

ChrisChV force-pushed the chris/FAL-3624-import-export-courses branch from adb81c5 to 82dab62 Compare March 19, 2024 18:50

test: Fix tests and types on csv exporting

2f931b3

ChrisChV mentioned this pull request Mar 25, 2024

[FC- 0049] feat: Features to enable import/export courses openedx/openedx-learning#172

Merged

ChrisChV added 4 commits March 26, 2024 11:49

feat: Import tags on course and blocks

07db758

Merge branch 'master' into chris/FAL-3624-import-export-courses

2ee29e5

test: Fixing tests and adding test for import

f7e3260

fix: Tests and lint

abf7b06

ChrisChV marked this pull request as ready for review March 26, 2024 19:16

style: on comments

97571af

ChrisChV commented Mar 26, 2024

View reviewed changes

requirements/edx/kernel.in Outdated Show resolved Hide resolved

style: Nit on comment

9bba25c

Merge branch 'master' into chris/FAL-3624-import-export-courses

d143b4f

ChrisChV added 2 commits March 27, 2024 17:15

refactor: Change export/import tags separator to ';'

476cf95

Merge branch 'master' into chris/FAL-3624-import-export-courses

1a3fbb6

ChrisChV added 4 commits April 1, 2024 11:36

chore: bump version

70d1d64

fix: Bug in get_all_object_tags

31eab30

When you add a new tags in a course, the previous tags are not exported How repodruce: - Import a course - Add new tags to the course or blocks - Export a course

Merge branch 'master' into chris/FAL-3624-import-export-courses

297e212

style: Nits

4534de0

ChrisChV mentioned this pull request Apr 2, 2024

[Tagging] Handle import course when the current user does not have permission openedx/modular-learning#204

Closed

chore: Fix merge conflicts

1a8b8b6

style: Fix types checks

5435985

rpenido approved these changes Apr 3, 2024

View reviewed changes

ChrisChV added 3 commits April 4, 2024 14:12

Merge branch 'master' into chris/FAL-3624-import-export-courses

fdf7bc7

refactor: Move test_objecttag_export_helpers.py to test directory

8e8a595

fix: Fix tests

bda8da2

bradenmacdonald reviewed Apr 5, 2024

View reviewed changes

ChrisChV added 2 commits April 8, 2024 12:10

style: Nits on tests

a271fde

Merge branch 'master' into chris/FAL-3624-import-export-courses

90d6572

bradenmacdonald approved these changes Apr 8, 2024

View reviewed changes

style: Nit on pylint

c80f6f7

bradenmacdonald merged commit ddb407a into openedx:master Apr 8, 2024
67 checks passed

GlugovGrGlib pushed a commit to raccoongang/edx-platform that referenced this pull request Apr 12, 2024

feat: handle tags when importing/exporting courses (openedx#34356)

723171d

KyryloKireiev pushed a commit to raccoongang/edx-platform that referenced this pull request Apr 24, 2024

feat: handle tags when importing/exporting courses (openedx#34356)

7426ab6

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FC-0049] Handle tags when importing/exporting courses #34356

[FC-0049] Handle tags when importing/exporting courses #34356

ChrisChV commented Mar 12, 2024 •

edited

openedx-webhooks commented Mar 12, 2024 •

edited

ChrisChV Mar 26, 2024

bradenmacdonald Mar 26, 2024

ChrisChV commented Mar 26, 2024

bradenmacdonald commented Mar 26, 2024 •

edited

ChrisChV commented Mar 27, 2024

rpenido commented Mar 27, 2024 •

edited

bradenmacdonald commented Mar 28, 2024

ChrisChV commented Mar 28, 2024

bradenmacdonald commented Apr 1, 2024

ChrisChV commented Apr 2, 2024

bradenmacdonald commented Apr 3, 2024

rpenido left a comment

rpenido Apr 3, 2024

bradenmacdonald Apr 4, 2024

bradenmacdonald left a comment

bradenmacdonald Apr 5, 2024

ChrisChV Apr 8, 2024

bradenmacdonald Apr 5, 2024

ChrisChV Apr 8, 2024

bradenmacdonald Apr 8, 2024

ChrisChV Apr 8, 2024

ChrisChV commented Apr 8, 2024

bradenmacdonald commented Apr 8, 2024

openedx-webhooks commented Apr 8, 2024

edx-pipeline-bot commented Apr 8, 2024

edx-pipeline-bot commented Apr 8, 2024

edx-pipeline-bot commented Apr 8, 2024

[FC-0049] Handle tags when importing/exporting courses #34356

[FC-0049] Handle tags when importing/exporting courses #34356

Conversation

ChrisChV commented Mar 12, 2024 • edited

Description

Supporting information

Testing instructions

Other information

openedx-webhooks commented Mar 12, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ChrisChV commented Mar 26, 2024

bradenmacdonald commented Mar 26, 2024 • edited

ChrisChV commented Mar 27, 2024

rpenido commented Mar 27, 2024 • edited

bradenmacdonald commented Mar 28, 2024

ChrisChV commented Mar 28, 2024

bradenmacdonald commented Apr 1, 2024

ChrisChV commented Apr 2, 2024

bradenmacdonald commented Apr 3, 2024

rpenido left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bradenmacdonald left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ChrisChV commented Apr 8, 2024

bradenmacdonald commented Apr 8, 2024

openedx-webhooks commented Apr 8, 2024

edx-pipeline-bot commented Apr 8, 2024

edx-pipeline-bot commented Apr 8, 2024

edx-pipeline-bot commented Apr 8, 2024

ChrisChV commented Mar 12, 2024 •

edited

openedx-webhooks commented Mar 12, 2024 •

edited

bradenmacdonald commented Mar 26, 2024 •

edited

rpenido commented Mar 27, 2024 •

edited