Add good enough ID system

[bain3]

PRONOTE only returns its temporary object id. Since its id is temporary we cannot compare objects from different sessions. It would be great to add comparisons with this too.

[Bapt5]

@bain3 Do you think it's a good idea to create an id with the concatenation of some attributes. The inconvenant is that if for example the title changes, the id will change.

[bain3]

Making it from attributes is probably a bad idea... ids are not supposed to change. I thought I could reverse engineer it, but haven't done much progress. There isn't a clear structure to the changing ids. If I take the hex string as a number the first three or four bits don't change (but you can see that from the hex itself), that's what I got so far... Also I think they have something to do with the session number but that's just a guess.

[Bapt5]

I try many things to find how the ID system work, but I didn't find anything convincing. I will try to contact the owners of the other Pronote wrapper repos to find out if they have found anything.

[vingerha]

Providing some feedback on this topic from what I experienced the past months.
My setup collects everything in a database with which I eventually hope to provide historical views too.
Without having a unique identifier, I was 'forced' to create unique keys per table and this proved to be 'hell' for lessons at least even with the number field, supposedly the last being the truth (not!) there seems to be any combination possible.
Cases:

1. normal entry > cancelled > new entry
2. normal entry > changed > cancelled
3. normal entry > cancelled > changed > new entry > changed .... and remains changed
4. normal entry > cancelled > changed > new entry > changed ... but still cancelled
   esp. the last two are (for me) impossible to resolve... I was hoping to use the date/timestamp but in this case it won't work.
   If I collect the lessons fresh daily then it is fine and in case 3. where I now have 3 lines, it would only extract 1
   ... but, I wanted the history too as to check on absence (the labelling used by the school is so-to-speak very odd where teachers are sick and then the lesson gets marked 'class absent')

So not a solution at all, just my findings and apparently the behavior of the school's pronote-admin person plays a big role here too

[bain3]

I've made some limited progress on reverse engineering the IDs. It's not much, but I hope I'm getting somewhere. All the information I've found is in this gist: https://gist.github.com/bain3/953f377c0b96b83721c84f34d551d5d2

EDIT: So at this point we could try and implement some equality checks. They probably won't be perfect, but could be ok. We would need to store a reference object that is the same for all objects throughout all sessions. With some XOR magic we should be able to tell if they have the same id...

[bain3]

Ok, after some tests it looks very promising! A month of lessons does not have any collisions, neither do all grades from a period, or the periods themselves. Now its just a matter of finding a reference object to get something with the same key.

[vingerha]

I am not sure about your test data and possibly it is good but... just to give you an idea (after my entry above)...
You see 3 x 3 entries for the same lesson.
Pronote app shows:
For the first three : the last one, so cancelled.
For the second three: the first one , modified
For the last three: either first or second one, modified
So I am no longer sure what value the 'number' has, in these 3 cases I cannot bake cake :)

[vingerha]

Of course I am happy to commence testing but the school year has come to an end

[bain3]

A little update: I don't remember exactly what the problem was :D but it didn't work in practice. I tried to implement a comparable object class, so all the other data classes could inherit from it. I think the equations that I came up with in the gist don't work accross object types. All in all, the solution wasn't good enough.

[vingerha]

My (!) problem is predominantly with lessons. I collect data in a table and require to identify unique records, i.e. which lesson was imported yesterday and how does it compare to the same lesson imported just now.
I donot know how/why pronote itself let's this happen but from the school of my son I got all (a lot) the variety's possible and no way to identify which record presented the true status of the lesson... in the end I gave up and am now removing existing today/future lessons before loading new ones

[bain3]

Version 2023 completely changed the ID system. They now look something like this: 112#3U1I7QZJsAXGJ4Gq_V3kMG1PpTmOQoojn321aonvMBE. They unfortunately still change between sessions...

[vingerha]

Replace one not-good solution with another .... looks different, but isn't. My problem remains with not being able to detect which record has the 'real' value irrespectively of the ID. As in my son's school they could (!) first add the new situation, then remove the old lesson, resulting in 3 sometimes 4 records for the same slot.... or they first remobe it and then add the new one, then correct it. ...no clue what is the 'truth'. I continue to suspect this is a way to hide absentism

[Vexcited]

Version 2023 completely changed the ID system. They now look something like this: 112#3U1I7QZJsAXGJ4Gq_V3kMG1PpTmOQoojn321aonvMBE. They unfortunately still change between sessions...

Seems like the number before the # is some type ?
Even between sessions they're still unchanged.

Like on ListeCours, it's always 31#...;
On the content, the lesson's name is always 82#... and the lesson's classroom is always 138#...

Those are some examples I extracted from my DevTools in the demo instance, would be interesting to know if it's the same for everyone I guess.

Even though I have no idea what the characters after the # are referring to...

[bain3]

Seems like the number before the # is some type ? Even between sessions they're still unchanged.

Yes. In fact the types were there even before V2023. We just didn't notice them (well I did notice that part didn't change, but I didn't think it was a type).

New period: 112#Get6H47sO9UvOoqrgvR_Zp1c5Npl_Fj-naiP5OlXlhM
Old period: 112B9795153F959
New lesson: 31#FudPFxTJric0-0mO3QXX6xOqF38qYtJh0dakMq6nw_E
Old lesson: 319E0379E5AA50

I've spent most of today trying to reverse engineer the new IDs. I've looked into the server DB and it seems that they just use sequential IDs internally. I'm starting to think the new ID is somehow encrypted. Suspiciously, it is 32 bytes long, which is 2*AES block size. That would mean the paintext is longer that 16 bytes though... which is improbable (or these are two separate encrypted messages, or one part is an IV, we can never know...).

I've also tried scanning the server process' memory for an ID string and the byte sequence it represents with no results.