Issue with finding values with encrypted data

[Amraneze]

I have encrypted a column email but I can't find user that have the same email because it gives a different encryption. Is there a way to bypass that ?

I tried something like this

But without a chance.

[generalpiston]

Take a peek at this test case: https://github.com/generalpiston/typeorm-encrypted/blob/master/test/find-operator.test.ts#L41

[Amraneze]

I used Equal but I'm getting the same result.

[generalpiston]

@Amraneze So, the way it works is it encrypts the field and checks it against the internal value. If the secrets / keys have changed, then it won't work. Unfortunately, without access to your data set, I can't really figure it out for you. The tests we have for the find operator are passing. So, I suspect your key changed or the data doesn't exist.

[Amraneze]

@generalpiston if I use a static IV then it works fine. But, it's not a good practice to use a static one. The secret is set in environment variable.

[generalpiston]

@Amraneze hey that's a great point. Perhaps the IV should be different for each record?

[Amraneze]

@generalpiston I will just use a different IV for each entity. I can create a test project if you want to reproduce the issue.

PS: I also enabled encryption on the database.

[generalpiston]

@Amraneze I don't think we need a test project. Every thing you said tracks. I'm going to think about the security implications of using the same IV a bit and get back to this.

[razr22]

Any update on this? I'm not very keen on using the same IV or IV's for each entity.

[Amraneze]

@razr22 I'm using the same IV. it's better to generate IV for each entity but it you will have a lot of IV in your env variables. @generalpiston should we close this issue ?

[generalpiston]

@Amraneze @razr22 having a different IV is interesting since it would reduce the cardinality of sets open to attack. I think to do this, we'd have to do one of the following:

1. Use another table.
2. Use another column in the same table.
3. Prefix the IV to encrypted value.

In some cases, the IV may be a counter. In others, it should be a uniformly randomly generated value. This would vary based on the encryption strategy.

This is kind of a big task... I won't have time to get to this for a while. @Amraneze or @razr22 would either of you feel comfortable implementing it?

Here's how I'd do it:

• Implement 3 from the above list for a very specific, commonly used encryption scheme (ie. AES CBC).
• Fallback to common IV.

I'm happy to add whoever implements this to the list of collaborators and create a list of contributors and list their name.

[coderdan]

I've just started using this lib and came across this issue. Please do not use a fixed IV with AES. While there are some very specific cases that it will be OK, it will almost always completely break the encryption. See https://crypto.stackexchange.com/questions/26790/how-bad-it-is-using-the-same-iv-twice-with-aes-gcm

Queryable encryption is generally not possible with correct, randomised encryption. You can use the encryption in deterministic mode which is more secure but if that is the case, the IV should be generated using something like the HMAC of the value (not fixed).

[generalpiston]

@coderdan thanks for the insight. Are you recommending we use randomly generated IVs?

[mpalmer]

Yes, you should absolutely use IVs generated from a cryptographically-secure random number generator for each record, and store them with the ciphertext.

[coderdan]

@coderdan thanks for the insight. Are you recommending we use randomly generated IVs?

IVs must never repeat. If the same message is encrypted more than once with the same IV you can literally decrypt the message by XOR'ing the ciphertexts.

A cryptographically strong randomly generated IV is the best solution. A counter can also be used but you need to make sure that it never resets (or wraps around) and can't be manipulated so much less preferred IMHO. Users of this library likely won't know the nuances of the issue so best to abstract IV management away and just generate the IV randomly internally.

That actually happens now if you don't pass an IV to the transformer so a simple fix here could just be to remove the ability to manually set an IV.

[generalpiston]

@coderdan I guess with GCM mode, the nonce can't repeat. In CBC mode, a repeat isn't catastrophic, but it's slower. We should probably allow the iv function to be an async function.

[coderdan]

With CBC it is less bad, yes. Still a fixed IV will result in deterministic encryption and potential chosen plaintext attacks. Allowing the IV to be an async function would be OK so long as the risks are well documented :)

As you can see I get very passionate about this!

[bilalanees98]

@generalpiston is there a solution for this? sort of stuck on it.
I have an OTP entity with an encrypted property: email. This property is encrypted using aes-256-gcm.

When I try to find an entity against encrypted email, I keep running into the issue mentioned above:
Lets assume the email is example@gmail.com

console.log('otps: ', await otpRepository.find({ where: { email: Equal(email) }, order: { createdAt: 'DESC' } }));

The email value in my DB is: Wj0UcaorNUvIeG2cOccTJC9kRZA9GQoz5MXGAPpAp0vFC/MOQSms+ExwlnWEUYRhfdHBNVOIi/FAuw==

But the sql query generated is as follows:

`Otp`.`id` AS `Otp_id`, `Otp`.`code` AS `Otp_code`, `Otp`.`email` AS `Otp_email`, `Otp`.`created_at` AS `Otp_created_at` FROM `otps` `Otp` WHERE (`Otp`.`email` = ?) ORDER BY `Otp`.`created_at` DESC -- PARAMETERS: ["BIXVKw7RZXTRkYB/rNQL5nY9qkudMUShrL56QakpM4I4YWuCL7pd9m7h2T6m29z3Z2OTwSryMYcMJg=="]

for the same email the encrypted values of the email are completely different and I am unable to find an entry against said email.

Is there a workaround for this?