Encrypting with Aes256Gcm, public key certificate and CmsEnvelopedDataGenerator throws error - 2.2.1 decryption fix broke encryption

[jumajeric]

Hello everybody,
I am trying to encrypt a string using Aes256Gcm, but the encryption fails on generation with this message:

**System.ArgumentException: 'illegal object in GetInstance: Org.BouncyCastle.Asn1.DerOctetString (Parameter 'obj')'**
This exception was originally thrown at this call stack:
    Org.BouncyCastle.Asn1.Asn1Sequence.GetInstance(object) in Asn1Sequence.cs
    Org.BouncyCastle.Asn1.Cms.GcmParameters.GetInstance(object) in GcmParameters.cs
    Org.BouncyCastle.Security.ParameterUtilities.GetCipherParameters(string, Org.BouncyCastle.Crypto.ICipherParameters, Org.BouncyCastle.Asn1.Asn1Object) in ParameterUtilities.cs
    Org.BouncyCastle.Cms.CmsEnvelopedGenerator.GetAlgorithmIdentifier(string, Org.BouncyCastle.Crypto.Parameters.KeyParameter, Org.BouncyCastle.Asn1.Asn1Encodable, out Org.BouncyCastle.Crypto.ICipherParameters) in CMSEnvelopedGenerator.cs
    Org.BouncyCastle.Cms.CmsEnvelopedDataGenerator.Generate(Org.BouncyCastle.Cms.CmsProcessable, string, Org.BouncyCastle.Crypto.CipherKeyGenerator) in CMSEnvelopedDataGenerator.cs
    Org.BouncyCastle.Cms.CmsEnvelopedDataGenerator.Generate(Org.BouncyCastle.Cms.CmsProcessable, string) in CMSEnvelopedDataGenerator.cs
    AB.Common.Bll.ASiCEComponent.RsaEncryptWithPublic(byte[], Org.BouncyCastle.X509.X509Certificate) in ASiCEComponent.cs
    AB.Common.Bll.ASiCEComponent.GenerateASiCContainer(System.Collections.Generic.List<System.IO.Stream>, System.Collections.Generic.List<string>, string, string, string, bool) in ASiCEComponent.cs

It seems that the object can't be converted to Asn1Sequence ... why?

Using some other algorithm (non Aes for example DesEde3Cbc) works, so I don't know where to look for the actual error. The public key is loaded from string using X509Certificate2 and then converted to BC X509 (new Org.BouncyCastle.X509.X509CertificateParser().ReadCertificate(certificate.GetRawCertData())). The encryption method looks like this:

private static Stream RsaEncryptWithPublic(byte[] data, Org.BouncyCastle.X509.X509Certificate cert)
{
    var gen = new CmsEnvelopedDataGenerator();
    gen.AddKeyTransRecipient(cert);
    var message = new CmsProcessableByteArray(data);
    var envelopedData = gen.Generate(message, NistObjectIdentifiers.IdAes256Gcm.Id); // <- this is the line that fails
    var encryptedData = envelopedData.GetEncoded();
    return new MemoryStream(encryptedData);
}

The same code in IKVM and Java encrypts the data - but having many (really a lot) exceptions it fails on our production servers. Even creating a Java service using gRPC didn't solve the problem (the Java equivalent code also works) ... just added a whole bunch of additional problems for our production environment.

So what am I missing here, is the certificate not correct, but why does it work in IKVM and Java case? Or is Aes256Gcm algorithm still not supported on C# BC? Any help with this would be appreciated.

EDIT:
It seems that in the old version of the library (1.9.0 - Portable.BouncyCastle) the encryption works ... but in this version the decryption doesn't work if the original data was encrypted in Java BC. So if I'm right, the fix for decryption in 2.2.1 broke the encryption for Aes256Gcm ... fun stuff.

EDIT 2:
After trying some other algorithms it seems that Aes###Cbc works, all others fail.

(edit: @cipherboy / Feb 13 - edited for formatting)

[thasimon]

Hello!
I needed to find a fix for this problem. It turns out the actual problem arrises in the GenerateAsn1Parameters functions. It currently for GCM uses a DerOctet-string of the IV-size, but when the parameters were rewritten to fix decryption it now expects a DerSequence with the nonce of the algorithm.

Since the GenerateAsn1Parameters is virtual you can override it in the CmsEnvelopedDataGenerator. The code below assumes DefaultLenIV of 12:

public class AesGcmCompatibleCmsEnvelopedDataGenerator : CmsEnvelopedDataGenerator {
    protected override Asn1Encodable GenerateAsn1Parameters(string encryptionOid, byte[] encKeyBytes)
    {
        if (encryptionOid == Aes128Gcm || encryptionOid == Aes256Gcm) {
            var random = new SecureRandom(); 
            var nonce = SecureRandom.GetNextBytes(random, 12); 
            return new DerSequence(new DerOctetString(nonce)); 
        }
        
        return base.GenerateAsn1Parameters(encryptionOid, encKeyBytes);
    }
}

This rewrites the parameters for AES GCM to what it actually expects, and it has been tested to work across C# and Java encryption/decryption. In your sample above you would replace CmsEnvelopedDataGeneartor with the AesGcm compatible version.

@peterdettman Is it possible to implement this in the library so the workaround is no longer needed? The best place for it might be ParameterUtils, GetCipherParameters is already updated, but there is missing logic in the GenerateParameters method