how to create UInt8Array in AssemblyScript and return it to caller

[msbasanth]

Hi,

I am using AssemblyScript to convert my RLE decode code written in TypeScript to WebAssembly.
This is what I did,

1. This is my decodeRLE code which is in my index.ts file which I am trying to convert to wasm.

export function decodeRLE(encodedBuffer: Uint8Array): Uint8Array {
    var decodedBuffer = [];
    var encoderDataLength = encodedBuffer.byteLength;
    var pos = 8;
    let isRip: boolean;
    let isRun: boolean;
    let ripCount: number;
    let runCount: number;
    while (pos < encoderDataLength)
    {
      var cmd: number = getInt8(encodedBuffer[pos]);
      isRip= cmd<0;
      isRun= cmd>0;
      if(cmd<0){ // rip
        cmd=Math.abs(cmd);
        pos+=1;
       var posIndex = pos;
        for (var i = 0; i < cmd; i++)
        {
          var myvalue = encodedBuffer[posIndex];
          decodedBuffer.push(myvalue);
          posIndex++;
        }
        pos+=cmd;
      }
      else // run
      {
        cmd +=2;
        var v =encodedBuffer[pos + 1];
        if(v != 0)
        {
          for (var j = 0; j < cmd; j++) {
            decodedBuffer.push(v)
          };
        }
        else
        {
          for (var j = 0; j < cmd; j++) {
            decodedBuffer.push(0)
          };
        }
        pos+=2;
      }
    };
    return decodedBuffer;
}

2. Errors:
   ERROR AS100: Operation not supported.
   var decodedBuffer = [];
   in assembly/index.ts(97,24)

ERROR TS2339: Property 'push' does not exist on type 'i32'.
decodedBuffer.push(myvalue);
in assembly/index.ts(116,24)

ERROR AS200: Conversion from type 'f64' to 'i32' requires an explicit cast.
pos+=cmd;
in assembly/index.ts(119,13)

I would like to know how we can create a array (UInt8Array) in AssemblyScript? I aso want to return the created array back to consumers of my WASM. So that I can move the decode code to WASM.

Thanks
Basanth

[MaxGraey]

@msbasanth This code wouldn't valid for pure Typescript. Uint8Array hasn't push method because can't extensible, also it create differently. So, you could replace Uint8Array to Array<u8>.
Other issues:

• decodedBuffer = []; should be declare like decodedBuffer: u8[] = [];.
• boolean to bool.
• number to i32 or u32.

[msbasanth]

I modified accordingly @MaxGraey, now I have one warning,

WARNING AS102: User-defined: "Calling 'memory.allocate' requires a memory manager to be present."
WARNING("Calling 'memory.allocate' requires a memory manager to be present.");
 in ~lib/memory.ts(42,4)

How should I make memory manager available? Does this warning will create any runtime problem?
I have memory added while instantiating WebAssembly.

const memory = new WebAssembly.Memory({ initial: ((byteSize + 0xffff) & ~0xffff) >>> 16 });

[MaxGraey]

No, you just need import one of memory allocators.
For tests you can use arena for example. You should include it once in main file:

import "allocator/arena";

[msbasanth]

Yes @MaxGraey after adding memory allocator that warning also gone. Now I could generate the d.ts, wasm and .map files from my AssemblyScript.

my optimized.d.ts look like this:
declare module ASModule {
function decodeRLE(encodedBuffer: u32): u32; // Array is not in the signature because of which I am not able to return an array from the AssemblyScript function.
}

This is my function signature:
export function decodeRLE(encodedBuffer: Array<u8>): Array<u8>
I also tried with,
export function decodeRLE(encodedBuffer: u8[]): u8[]

In both these cases asbuild generated 'function decodeRLE(encodedBuffer: u32): u32' in optimize.d.ts.

Whether I am missing or need to do for correcting the function signature.

[dcodeIO]

When returning an array from WASM to JS, what you'll get on the JS side is the pointer (offset in memory), to a structure with the following layout. To retreive the values on the JS side, you'd read the .buffer property to obtain the backing ArrayBuffer and read the values from there.

[msbasanth]

Thanks for the quick reply @dcodeIO

the returned pointer to the array (number it showed is 16) I tried reading buffer but it is undefined.

const uintArray = convertToUint8Array(imageresponse.Buffer);
const array = Array.from(uintArray);
const decodedBuffer = this.exports.decodeRLE(array);
const arrayBuffer = decodedBuffer.buffer; // Here buffer is undefined.

These are the two AS functions I am trying to access,

export function decodeRLE(encodedBuffer: u8[]): u8[] {
    var decodedBuffer: u8[] = [];
    ....................................
    return decodedBuffer;
}

export function show_array(): Float64Array {
  let array: Float64Array = new Float64Array(5);
  array[0] = 99.88;
  array[1] = 88.77;
  array[2] = 77.66;
  array[3] = 66.55;
  array[4] = 55.44;
  return array;
}

When I tried to access .buffer from JS, it is undefined.

[dcodeIO]

.buffer is in WASM memory, not a property. Essentially, you'd first read the i32 at the pointer value (that's .buffer), which is again a pointer to the backing ArrayBuffer. At the ArrayBuffer pointer, you'd skip 8 bytes (.byteLength and 0), to reach the start of the array's elements in memory. Depending on the type of the array, the elements are then i32, f32 or f64 etc., in your case 8 bytes each f64. (see: Memory layout)

[msbasanth]

Thanks for the explanation, still unable to read the .buffer from the pointer value.

This is my WebAssembly fetch code:

        // Compute the size of and instantiate the module's memory
        const memory = new WebAssembly.Memory({ initial: ((80 + 0xffff) & ~0xffff) >>> 16 });
        // Fetch and instantiate the module
        fetch('assemblies/optimized.wasm')
            .then(response => response.arrayBuffer())
            .then(buffer => WebAssembly.instantiate(buffer, {
                env: {
                    memory,
                    abort: function () { }
                },
                config: {
                    BGR_ALIVE: 1, // little endian, LSB must be set
                    BGR_DEAD: 1, // little endian, LSB must not be set
                    BIT_ROT: 1
                },
                JSMath: Math
            }))
            .then(module => {
                this.exports = module.instance.exports;
                const decodedBufferPtr = this.exports.decodeRLE(inputArray);

My observations:

1. module.F64 is undefined same with module.I32. Even exports doesn't have F64 or I32
2. module.instance.exports.memory.buffer shows the array created inside WASM but unable to fetch from there.

How can I make F64 available for module?

[kyr0]

Well, if I'm not totally off, your initial memory is set to 1.

((80 + 0xffff) & ~0xffff) === 65536
65536 >>> 16 === 1

And I don't see why you need such a bit-dancing calc for the memory :)

[kyr0]

So, as I was in a total lack of better understanding, I asked on the WebAssembly design repo. And heureka... here we go!! :)))

WebAssembly/design#1231

[binji]

Well, if I'm not totally off, your initial memory is set to 1.

That's OK, the initial memory is defined in terms of 64KiB pages. That calculation is rounding 80 bytes up to the number of pages required.

[kyr0]

@msbasanth

Example code (using current impl. by @dcodeIO):

https://github.com/kyr0/assemblyscript-js-wasm-interop-example

[msbasanth]

@kyr0 the sample worked fine which showed how we can share an array to WASM and get the array modified. This simplified the passing arrays between JS and WASM.

The same sample I am trying to run in browser (Angular) application. Instead of using loader.instantiateBuffer I used loader.instantiateStreaming.

async anAsyncMethod() {
    const imports = {};
    const {instance} = await loader.instantiateStreaming(fetch('assets/optimized.wasm'), imports);
    console.log('WASM succeeded' + instance);
    const calcNums = new Int32Array([1, 2, 3, 4, 5, 0x7fffffff]);
    console.log('Input array data to be summed:', calcNums);
    const ptr = instance.newArray(calcNums);
  }
ngAfterViewInit() {
    this.anAsyncMethod();
}

I am getting this error:

Uncaught (in promise): RangeError: WebAssembly.Instance is disallowed on the main thread, if the buffer size is larger than 4KB. Use WebAssembly.instantiate.
RangeError: WebAssembly.Instance is disallowed on the main thread, if the buffer size is larger than 4KB. Use WebAssembly.instantiate.

Is it the right way I am using loader.instantiateStreaming?

Regards
Basanth

[dcodeIO]

Seems there's a limitation on new WebAssembly.Instance, which the loader is using after a compileStreaming in its implementation of instantiateStreaming. Should probably update the loader to use WebAssembly.instantiateStreaming all the way down.

[binji]

Seems there's a limitation on new WebAssembly.Instance

Just to be clear, this is a Chrome-only restriction, AFAIK other browsers allow it. But to be safe, it's best to use WebAssembly.instantiate or WebAssembly.instantiateStreaming which return promises instead of resolving synchronously.

[msbasanth]

Thanks @binji and @dcodeIO

There is a limitation on WebAssembly.Instance for Chrome. I switched to firefox for my testing and I could see below code worked fine.

const loader = require('../../../assemblyscript/lib/loader');
declare var WebAssembly;


const memory = new WebAssembly.Memory({ initial: ((100 + 0xffff) & ~0xffff) >>> 16 });
        const imports = {
            env: {
                memory,
                abort: function () { }
            }
        };
        loader.instantiateStreaming(fetch('assets/RleAssemblyScript_optimized.wasm'), imports).then(module => {
            console.log('WASM succeeded.');  // "3"
            const decodedPtr = module.newArray(Uint8Array, length);
            const ptr = module.newArray(bufferToDecode);
            module.decodeRLE(ptr, decodedPtr);
            var decodedBuffer = module.getArray(Uint8Array, decodedPtr);
            module.freeArray(ptr);
            module.freeArray(decodePtr);
  });