Musing on feedback, and multiple submissions & mutiple feedback

[perllaghu]

I'm interested if this is just a perceived problem in my head, or a real problem others have seen - the Assignments page only ever links to the last feedback received.

What's my problem

• student submits multiple times (and the notebook could well vary, as they go back and do more work or refine their answers)
• instructors collect multiple times (they don't always collect at the due date), but will always collect the last submitted record from a student (even if the student has submitted more that once)
• grading is only ever for the last collected submission
• instructors may release feedback multiple times, but the feedback is always for the last collected submission
• students collect the last released feedback, but will only collect the last released feedback (even if that means they miss one)

My problem is that the UI provides a link for every submission, and that link always goes to the latest feedback

What's the code doing

• The lister looks here for submissions here:

        if self.inbound:
            regexp = r".*/(?P<course_id>.*)/inbound/(?P<student_id>[^+]*)\+(?P<assignment_id>[^+]*)\+(?P<timestamp>[^+]*)(?P<random_string>\+.*)?"

• For each notebook in each submitted assignment, the lister looks for two things:

1. For already fetched feedback:

                local_feedback_dir = os.path.join(
                    assignment_dir, 'feedback', info['timestamp'])
                local_feedback_path = os.path.join(
                    local_feedback_dir, '{0}.html'.format(nb_info['notebook_id']))

.... note that this is separated into info['timestamp'] folders

2. For feedback in the exchange:

                exchange_feedback_path = os.path.join(
                    self.root, info['course_id'], 'feedback', '{0}.html'.format(nb_hash))

.... which will always be the latest generated feedback

So, in theory, we should know all the feedback we have, and the latest available feedback.

Reading through https://nbgrader.readthedocs.io/en/stable/exchange/exchange_api.html#released-submitted - we currently don't include timestamps in any of the data we create.

Suggestion

If the data-object that the exchange plugin passes to the client, lets add a timestamp field.

• assignment["timestamp"] is the timestamp of the submission was handled by the server (the (?P<timestamp>[^+]*) bit of the path), and will be slightly different (by time-zones & milliseconds) to the timestamp in the timestamp.txt file - but fine for this sorting

            {
                "assignment_id":"tree",
                "student_id":1,
                "course_id":"cool_course_id",
                "status":"submitted",
                "path":"path/to/./inbound/student_1+tree+2020-10-24 16:08:38.781505",
                "notebooks":[
                   {
                      "notebook_id":"Lab 1",
                      "has_exchange_feedback":true,
                      "feedback_updated":false,
                      "has_local_feedback":true,
                   },
                   {
                      "notebook_id":"Lab 2",
                      "has_exchange_feedback":true,
                      "feedback_updated":false,
                      "has_local_feedback":true,
                   },
                ],
                "timestamp":"2020-10-24 16:08:38.781505",
                "has_local_feedback":true,
                "has_exchange_feedback":true,
                "feedback_updated":false,
                "local_feedback_path":"cool_course_id/tree/feedback/2020-11-03 15:17:23.357641"
             },

I think this will allow us to match a piece of feedback to the submission to applies to

Thoughts?
(I've not actually tried any of this yet....)