The New web3.js – Technology Preview 3

tp3 (2024-04-25)

The next version of the @solana/web3.js Technology Preview brings a major change to how signed transactions are represented, in response to user feedback.

To install the third Technology Preview:

npm install --save @solana/web3.js@tp3

Most notably, all *Transaction*() helpers have been renamed to *TransactionMessage*() to reflect what is actually being built when you build a transaction: the transaction message.

Before

const tx = pipe(
    createTransaction({ version: 0 }),
    tx => addTransactionFeePayer(payerAddress, tx),
    /* ... */
);

After

const txMessage = pipe(
    createTransactionMessage({ version: 0 }),
    m => addTransactionMessageFeePayer(payerAddress, m),
    /* ... */
);

We've introduced a new type to represent signed and partially signed messages. This type encapsulates the bytes of a transaction message – however they were serialized – and the ordered map of signer addresses to signatures. Reducing a transaction message to just those two things after the first signature is applied will make it harder for a subsequent signer to invalidate the existing signatures by _re_serializing the transaction message in such a way that the bytes or the order of signer addresses changes.

Try a demo of Technology Preview 3 in your browser at CodeSandbox.

Changelog since Technology Preview 2

• #2434 31916ae Thanks @lorisleiva! - Renamed mapCodec to transformCodec

• #2411 2e5af9f Thanks @lorisleiva! - Renamed fixCodec to fixCodecSize

• #2352 125fc15 Thanks @steveluscher! - SubtleCrypto assertion methods that can make their assertions synchronously are now synchronous, for performance.

• #2329 478443f Thanks @luu-alex! - createKeyPairFromBytes() now validates that the public key imported is the one that would be derived from the private key imported

• #2383 ce1be3f Thanks @lorisleiva! - getScalarEnumCodec is now called getEnumCodec

• #2382 7e86583 Thanks @lorisleiva! - getDataEnumCodec is now called getDiscriminatedUnionCodec

• #2397 a548de2 Thanks @lorisleiva! - Added a new addCodecSizePrefix primitive

  const codec = addCodecSizePrefix(getBase58Codec(), getU32Codec());
  
  codec.encode("hello world");
  // 0x0b00000068656c6c6f20776f726c64
  //   |       └-- Our encoded base-58 string.
  //   └-- Our encoded u32 size prefix.

• #2419 89f399d Thanks @lorisleiva! - Added new addCodecSentinel primitive

  The addCodecSentinel function provides a new way of delimiting the size of a codec. It allows us to add a sentinel to the end of the encoded data and to read until that sentinel is found when decoding. It accepts any codec and a Uint8Array sentinel responsible for delimiting the encoded data.

  const codec = addCodecSentinel(getUtf8Codec(), new Uint8Array([255, 255]));
  codec.encode("hello");
  // 0x68656c6c6fffff
  //   |        └-- Our sentinel.
  //   └-- Our encoded string.

• #2400 ebb03cd Thanks @lorisleiva! - Added new containsBytes and getConstantCodec helpers

  The containsBytes helper checks if a Uint8Array contains another Uint8Array at a given offset.

  containsBytes(new Uint8Array([1, 2, 3, 4]), new Uint8Array([2, 3]), 1); // true
  containsBytes(new Uint8Array([1, 2, 3, 4]), new Uint8Array([2, 3]), 2); // false

  The getConstantCodec function accepts any Uint8Array and returns a Codec<void>. When encoding, it will set the provided Uint8Array as-is. When decoding, it will assert that the next bytes contain the provided Uint8Array and move the offset forward.

  const codec = getConstantCodec(new Uint8Array([1, 2, 3]));
  
  codec.encode(undefined); // 0x010203
  codec.decode(new Uint8Array([1, 2, 3])); // undefined
  codec.decode(new Uint8Array([1, 2, 4])); // Throws an error.

• #2344 deb7b80 Thanks @lorisleiva! - Improve getTupleCodec type inferences and performance

  The tuple codec now infers its encoded/decoded type from the provided codec array and uses the new DrainOuterGeneric helper to reduce the number of TypeScript instantiations.

• #2322 6dcf548 Thanks @lorisleiva! - Use DrainOuterGeneric helper on codec type mappings

  This significantly reduces the number of TypeScript instantiations on object mappings,
  which increases TypeScript performance and prevents "Type instantiation is excessively deep and possibly infinite" errors.

• #2381 49a764c Thanks @lorisleiva! - DataEnum codecs can now use numbers or symbols as discriminator values

  const codec = getDataEnumCodec([
    [1, getStructCodec([[["one", u32]]])][
      (2, getStructCodec([[["two", u32]]]))
    ],
  ]);
  
  codec.encode({ __kind: 1, one: 42 });
  codec.encode({ __kind: 2, two: 42 });

  This means you can also use enum values as discriminators, like so:

  enum Event {
    Click,
    KeyPress,
  }
  const codec = getDataEnumCodec([
    [
      Event.Click,
      getStructCodec([
        [
          ["x", u32],
          ["y", u32],
        ],
      ]),
    ],
    [Event.KeyPress, getStructCodec([[["key", u32]]])],
  ]);
  
  codec.encode({ __kind: Event.Click, x: 1, y: 2 });
  codec.encode({ __kind: Event.KeyPress, key: 3 });

• #2430 82cf07f Thanks @lorisleiva! - Added useValuesAsDiscriminators option to getEnumCodec

  When dealing with numerical enums that have explicit values, you may now use the useValuesAsDiscriminators option to encode the value of the enum variant instead of its index.

  enum Numbers {
    One,
    Five = 5,
    Six,
    Nine = 9,
  }
  
  const codec = getEnumCodec(Numbers, { useValuesAsDiscriminators: true });
  codec.encode(Direction.One); // 0x00
  codec.encode(Direction.Five); // 0x05
  codec.encode(Direction.Six); // 0x06
  codec.encode(Direction.Nine); // 0x09

  Note that when using the useValuesAsDiscriminators option on an enum that contains a lexical value, an error will be thrown.

  enum Lexical {
    One,
    Two = "two",
  }
  getEnumCodec(Lexical, { useValuesAsDiscriminators: true }); // Throws an error.

• #2398 bef9604 Thanks @lorisleiva! - Added a new getUnionCodec helper that can be used to encode/decode any TypeScript union.

  const codec: Codec<number | boolean> = getUnionCodec(
    [getU16Codec(), getBooleanCodec()],
    (value) => (typeof value === "number" ? 0 : 1),
    (bytes, offset) => (bytes.slice(offset).length > 1 ? 0 : 1),
  );
  
  codec.encode(42); // 0x2a00
  codec.encode(true); // 0x01

• #2401 919c736 Thanks @lorisleiva! - Added new getHiddenPrefixCodec and getHiddenSuffixCodec helpers

  These functions allow us to respectively prepend or append a list of hidden Codec<void> to a given codec. When encoding, the hidden codecs will be encoded before or after the main codec and the offset will be moved accordingly. When decoding, the hidden codecs will be decoded but only the result of the main codec will be returned. This is particularly helpful when creating data structures that include constant values that should not be included in the final type.

  const codec: Codec<number> = getHiddenPrefixCodec(getU16Codec(), [
    getConstantCodec(new Uint8Array([1, 2, 3])),
    getConstantCodec(new Uint8Array([4, 5, 6])),
  ]);
  
  codec.encode(42);
  // 0x0102030405062a00
  //   |     |     └-- Our main u16 codec (value = 42).
  //   |     └-- Our second hidden prefix codec.
  //   └-- Our first hidden prefix codec.
  
  codec.decode(new Uint8Array([1, 2, 3, 4, 5, 6, 42, 0])); // 42

• #2433 2d48c09 Thanks @lorisleiva! - The getBooleanCodec function now accepts variable-size number codecs

• #2394 288029a Thanks @lorisleiva! - Added a new getLiteralUnionCodec

  const codec = getLiteralUnionCodec(["left", "right", "up", "down"]);
  // ^? FixedSizeCodec<"left" | "right" | "up" | "down">
  
  const bytes = codec.encode("left"); // 0x00
  const value = codec.decode(bytes); // 'left'

• #2410 4ae78f5 Thanks @lorisleiva! - Added new getZeroableNullableCodec and getZeroableOptionCodec functions

  These functions rely on a zero value to represent None or null values as opposed to using a boolean prefix.

  const codec = getZeroableNullableCodec(getU16Codec());
  codec.encode(42); // 0x2a00
  codec.encode(null); // 0x0000
  codec.decode(new Uint8Array([42, 0])); // 42
  codec.encode(new Uint8Array([0, 0])); // null

  Both functions can also be provided with a custom definition of the zero value using the zeroValue option.

  const codec = getZeroableNullableCodec(getU16Codec(), {
    zeroValue: new Uint8Array([255, 255]),
  });
  codec.encode(42); // 0x2a00
  codec.encode(null); // 0xfffff
  codec.encode(new Uint8Array([0, 0])); // 0
  codec.decode(new Uint8Array([42, 0])); // 42
  codec.decode(new Uint8Array([255, 255])); // null

• #2380 bf029dd Thanks @lorisleiva! - DataEnum codecs now support custom discriminator properties

  const codec = getDataEnumCodec(
    [
      [
        "click",
        getStructCodec([
          [
            ["x", u32],
            ["y", u32],
          ],
        ]),
      ],
      ["keyPress", getStructCodec([[["key", u32]]])],
    ],
    { discriminator: "event" },
  );
  
  codec.encode({ event: "click", x: 1, y: 2 });
  codec.encode({ event: "keyPress", key: 3 });

• #2414 ff4aff6 Thanks @lorisleiva! - Used capitalised variant names for Endian enum

  This makes the enum more consistent with other enums in the library.

  // Before.
  Endian.BIG;
  Endian.LITTLE;
  
  // After.
  Endian.Big;
  Endian.Little;

• #2376 9370133 Thanks @steveluscher! - Fixed a bug that prevented the production error decoder from decoding negative error codes

• #2358 2d54650 Thanks @steveluscher! - The encoded SolanaError context that is thrown in production is now base64-encoded for compatibility with more terminal shells

• #2502 5ed19c6 Thanks @steveluscher! - Added TypeScript types to @solana/fast-stable-stringify

• #2491 2040f96 Thanks @lorisleiva! - Remove program types and resolveTransactionError helper

• #2490 1672346 Thanks @lorisleiva! - Add isProgramError helper function to @solana/programs

• #2504 18d6b56 Thanks @steveluscher! - Replaced fast-stable-stringify with our fork

• #2415 c801637 Thanks @steveluscher! - Improve transaction sending reliability for those who skip preflight (simulation) when calling sendTransaction

• #2553 af9fa3b Thanks @buffalojoec! - Changes createRecentSignatureConfirmationPromiseFactory to enforce rpc and rpcSubscriptions to have matching clusters, changing the function signature to accept an object rather than two parameters.

• #2554 0b02de1 Thanks @buffalojoec! - Changes createNonceInvalidationPromiseFactory to enforce rpc and rpcSubscriptions to have matching clusters, changing the function signature to accept an object rather than two parameters.

• #2550 54d68c4 Thanks @mcintyre94! - Refactor transactions, to separate constructing transaction messages from signing/sending compiled transactions

  A transaction message contains a transaction version and an array of transaction instructions. It may also have a fee payer and a lifetime. Transaction messages can be built up incrementally, for example by adding instructions or a fee payer.

  Transactions represent a compiled transaction message (serialized to an immutable byte array) and a map of signatures for each required signer of the transaction message. These signatures are only valid for the byte array stored in the transaction. Transactions can be signed by updating this map of signatures, and when they have a valid signature for all required signers they can be landed on the network.

  Note that this change essentially splits the previous @solana/transactions API in two, with functionality for creating/modifying transaction messages moved to @solana/transaction-messages.

• #2413 002cc38 Thanks @lorisleiva! - Removed getStringCodec in favour of fixCodecSize and addCodecSizePrefix

  The getStringCodec function now always returns a VariableSizeCodec that uses as many bytes as necessary to encode and/or decode strings. In order to fix or prefix the size of a getStringCodec, you may now use the fixCodecSize or prefixCodecSide accordingly. Here are some examples:

  // Before.
  getStringCodec({ size: "variable" }); // Variable.
  getStringCodec({ encoding: getUtf8Codec(), size: "variable" }); // Variable (equivalent).
  getStringCodec({ size: 5 }); // Fixed.
  getStringCodec({ encoding: getUtf8Codec(), size: 5 }); // Fixed (equivalent).
  getStringCodec(); // Prefixed.
  getStringCodec({ encoding: getUtf8Codec(), size: getU32Codec() }); // Prefixed (equivalent).
  
  // After.
  getUtf8Codec(); // Variable.
  fixCodecSize(getUtf8Codec(), 5); // Fixed.
  addCodecSizePrefix(getUtf8Codec(), getU32Codec()); // Prefixed.

• #2412 e3e82d9 Thanks @lorisleiva! - Removed the size option of getBytesCodec

  The getBytesCodec function now always returns a VariableSizeCodec that uses as many bytes as necessary to encode and/or decode byte arrays. In order to fix or prefix the size of a getBytesCodec, you may now use the fixCodecSize or prefixCodecSide accordingly. Here are some examples:

  // Before.
  getBytesCodec(); // Variable.
  getBytesCodec({ size: 5 }); // Fixed.
  getBytesCodec({ size: getU16Codec() }); // Prefixed.
  
  // After.
  getBytesCodec(); // Variable.
  fixCodecSize(getBytesCodec(), 5); // Fixed.
  addCodecSizePrefix(getBytesCodec(), getU16Codec()); // Prefixed.