feat/HIT26_GDPR-anonymity-of-beneficiaries #23

GuilhermeGabriel · 2024-01-15T18:52:38Z

Description

To stay in conformity of GDPR, we need to deleted some data after some long time that users don't use it:

Add field (lastLogin) to user model that would be updated with the date on every new login.
Deleting a user (using cron):
Finding the staff record related to the user (users question with their userID selected).
Hiding the info both on the user and their staff record

email: [RANDOM STRING]@anonymus-oort.com
nome: [ANONYMOUS]
...

For the beneficiaries:
Trigger when: The last aid to the family was given more than 18 months ago
Than: Anonymize all members of that family

Useful links

Please insert link to ticket: https://oortcloud.atlassian.net/jira/software/projects/OORT/boards/5?selectedIssue=HIT-26

Type of change

Please delete options that are not relevant.

New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Changing the records in database to past, running the script and check if it anonymize it.

Screenshots

before

after

Checklist:

( * == Mandatory )

More explanation

https://www.loom.com/share/05a716d61b9744faaf51fb304c21d1e5?sid=f87cf896-582a-4f76-93ae-8ceed801b145

matheus-relief

@GuilhermeGabriel I refactored a bit the code, and addressed most of the comments I left already. Just review the logic for the beneficiary case please

matheus-relief · 2024-01-21T03:36:43Z

migrations/1705411074596-add-last-login.ts

+import { User } from '@models';
+import { startDatabaseForMigration } from '../src/utils/migrations/database.helper';
+
+/**
+ *  Update lastLogin field.
+ */
+export const up = async () => {
+  await startDatabaseForMigration();
+
+  const users = await User.find({});
+  for (const user of users) {
+    // Create field lastLogin with value of modifiedAt
+    user.lastLogin = user.modifiedAt;
+
+    // Save user
+    await user.save();
+  }
+};
+
+/**
+ * Sample function of down migration
+ *
+ * @returns just migrate data.
+ */
+export const down = async () => {
+  /*
+      Code you downgrade script here!
+   */
+};


I changed my mind, since we're only using this on the cron job, we can just do this same logic there and skip the migration entirely

matheus-relief · 2024-01-21T03:37:45Z

src/index.ts

+      // Find the resource with name staff
+      const staff = await Resource.findOne({ name: 'Staff' });
+
+      // Find all the records of staff with the users found above
+      const usersStaffRecords = await Record.find({
+        resource: staff._id,


We can just pass the id directly and skip one DB call

matheus-relief · 2024-01-21T03:38:27Z

src/index.ts

+        user.username = `${Math.random()
+          .toString(36)
+          .substring(2, 15)}@anonymus-oort.com`;


Changed my mind about this too, instead of a random string, we can just use the mongo object id

matheus-relief · 2024-01-21T03:40:24Z

src/index.ts

+
+        await user.save();


Instead of doing operations inside loops, use Users.bulkSave(usersToDelete) to save a bunch of DB calls

matheus-relief · 2024-01-21T03:41:11Z

src/index.ts

+        staffRecord._createdBy = new User({
+          name: 'ANONYMOUS',
+          username: `${Math.random()
+            .toString(36)
+            .substring(2, 15)}@anonymus-oort.com`,
+        });


We don't need to do this, we are already hiding the information of the original user

matheus-relief · 2024-01-21T05:01:57Z

src/index.ts

+      // Find all the records of Ais was given more than 18 months ago
+      const AidRecords = await Record.find({
+        resource: Aid._id,
+        createdAt: {
+          $lt: new Date(Date.now() - 18 * 30 * 24 * 60 * 60 * 1000),
+        }, // 18 months ago
+      });
+
+      // Anonymize all members of that family


This logic is incorrect, like this it would erase all beneficiaries that got an aid longer than 18 months ago, when in fact we want to erase those who have NOT gotten an aid in the last 18 months.

I added a new commit to fix that!

matheus-relief · 2024-01-30T03:56:09Z