.Net SDK/PDK - Host Function multiple arguments documentation/strings

[Simonl9l]

Whist that we get that this early days, for comprehensive docs, do you have any examples of how to set up a host function to have multiple arguments?

On poking around the .net sdk, we see this https://github.com/extism/dotnet-sdk/blob/11b9e4933477679f90f7bf11d1b97baa20f324c8/src/Extism.Sdk/HostFunction.cs#L17

It seem that today there is only support for three parameters, via the FromMethod static factory functions. It seem that one can't extend that to add additional parameters (on basis that PDK supports it) by replicating/extending this construct with more template arguments, as the ToExtismType helper function is private.

However on the basis that for now it supports 3 arguments as string's that with such a setup and that the provided callback signature would need to say, specify the type of each as long, how does the PDK side of thing know these are in fact string's?

We're kind of inferring this due to the readme per:

HostFunction.FromMethod("kv_read", IntPtr.Zero, (CurrentPlugin plugin, long keyOffset) =>
{
     var key = plugin.ReadString(keyOffset);
     ....
}

On the basis this is known, do each this long contain the keyOffset per argument from which one would need to plugin.ReadString(keyOffset) for each string?

How does the PDK side of things know the are in fact supposed to be strings arguments ? Is there an attribute that one need to provide that helps describe this ?

On the basis that all arguments are of the same type, is there a variation of FromMethod that can take a params (docs) argument as an array of such arguments that can then easily be stuffed in the the inputTypes/outputTypes of HostFunction ?

Ideally our use case (in part) we'd need to support some way to represent a set of Key/Value pairs, 4x strings, and two long integers as arguments to the host function.

We guess there could be a set of Host Functions that pass each part, and one to initiate the final request - but this would not be ideal!?

What role does the userdata parameter play ?

Also per other issues/pr's, it would be good there as an FromMethodAsync that encapsulated async/await behaviors and cancellation tokens.

We're somewhat fascinated as to what y'alls are putting together here, and are also doing a bunch of .Net/Rust FFI inter-op ourselves so understand some of the challenges, we figure Extism could form a major part of our extensibility model!

[mhmd-azeez]

Hello @Simonl9l thank you for your interest, I will try to answer your questions

It seem that today there is only support for three parameters, via the FromMethod static factory functions

The FromMethod overloads are a convenience layer on top of the HostFunction constructor. You can have as many parameters as you want if you use the raw HostFunctions constructor. I only wrote FromMethod for 3 params as I wasn't sure if more params are necessary (we have to stop somewhere 😅)

On the basis this is known, do each this long contain the keyOffset per argument from which one would need to plugin.ReadString(keyOffset) for each string?

Yes. keyOffset is basically a pointer to a memory location in the wasm memory space.

How does the PDK side of things know the are in fact supposed to be strings arguments ? Is there an attribute that one need to provide that helps describe this ?

The type of each parameter is a contract between the Host and the Plugin (Guest). So the Host decides what each parameters type is. It's up to the Host devs to provide documentation for the Plugin developers. Unfortunately I can't find a way to make this easier, as wasm functions only support these data types: i32, i64, f32, f64 and some types that are not very well supported (v128, funcref, externref). Extism uses i64 as pointers for memory addresses. So a i64 parameter can either be a literal i64 number, or a pointer to a memory location. This depends on the agreement of the Host and Plugin.

On the basis that all arguments are of the same type, is there a variation of FromMethod that can take a params (docs) argument as an array of such arguments that can then easily be stuffed in the the inputTypes/outputTypes of HostFunction ?

We don't have a params variation because the callback params will no longer be strongly typed (i.e. you'll get object as parameters). This defeats the purpose of it being a convenience layer

Ideally our use case (in part) we'd need to support some way to represent a set of Key/Value pairs, 4x strings, and two long integers as arguments to the host function.
We guess there could be a set of Host Functions that pass each part, and one to initiate the final request - but this would not be ideal!?

One other way is to serialize the whole payload into json (or some other format) and pass it in as one string from the PDK side, and then deserialize it in the Host Function. This way you can pass a complex object between your plugin and host.

But, if you prefer having different parameters, then you can use the HostFunction constructor for this purpose:

using var hf = new HostFunction(
    "my_function",
    new[] { ExtismValType.PTR, ExtismValType.PTR, ExtismValType.PTR, ExtismValType.PTR, ExtismValType.I64, ExtismValType.I64 },
    new[] { ExtismValType.PTR }, // assuming you return a string
    IntPtr.Zero,
    (CurrentPlugin plugin, Span<ExtismVal> inputs, Span<ExtismVal> outputs) =>
    {
        // read inputs
        var key1 = plugin.ReadString(inputs[0].v.ptr);
        var value1 = plugin.ReadString(inputs[1].v.ptr);

        var key2 = plugin.ReadString(inputs[2].v.ptr);
        var value2 = plugin.ReadString(inputs[3].v.ptr);

        var num1 = inputs[4].v.i64;
        var num2 = inputs[5].v.i64;

        // TODO: do stuff here

        // return a string to the plugin
        outputs[0].v.ptr = plugin.WriteString(key1 + value1);
    });

What role does the userdata parameter play ?

You can get the InPtr of a .NET object and pass it in the Host Function. And then you can get it back in the callback through CurrentPlugin.UserData. But this is usually not necessary because it's easier to just use closures (i.e. just let the callback to capture the variable)

Also per other issues/pr's, it would be good there as an FromMethodAsync that encapsulated async/await behaviors and cancellation tokens.

Support for async functions in wasm is not very good. We will add support for FromMethodAsync as soon as the underlying runtime supports it. Because this must work for all guest languages.

I hope this answers your questions. Please let me know if any of the points isn't clear

[Simonl9l]

Thanks again!

To confirm, per plugin.ReadBytes, where for I get the expected Span<byte> from given a Span<ExtismVal> on basis that I need an argument value to be of type object?

[Simonl9l]

@mhmd-azeez - an additional question, per your reply:

Is there support for host functions overloading where the same function name can have different signatures?

Yes, this should work

[DllImport("extism", EntryPoint = "MyMethod")]
static extern void MyMethod"(ulong arg1Offset, ulong arg2Offset, ulong arg3Offset, ulong arg4Offset, long arg5Offset, long arg6Offset);

[DllImport("extism", EntryPoint = "MyMethod")]
static extern void MyMethod"(ulong arg1Offset, ulong arg2Offset, ulong arg3Offset, ulong arg4Offset, long arg5Offset, long arg6Offset);

This results in syntax error: A local function named 'MyMethod' cannot be declared in this scope because it would give a different meaning to 'SetEvent', which is already used in a parent or current scope to denote something else

So to confirm there is no way to do overloading?

[mhmd-azeez]

@Simonl9l from my experience it does need the WASI SDK to be installed, and that's what @maraf is saying:

Not yet. It's something that will most probably change, but at the moment you need to install wasi-sdk manually and set WASI_SDK_PATH to point to it.

[mhmd-azeez]

To confirm, per plugin.ReadBytes, where for I get the expected Span from given a Span on basis that I need an argument value to be of type object?

Each ExtismVal represents a parameter. plugin.ReadBytes needs an address for a memory block. If one of the parameters is a pointer to a memory address, then you can use the Extism.v.ptr or Extism.v.i64 to get the value of the parameter (aka the pointer) and pass it on to plugin.ReadBytes

[mhmd-azeez]

This results in syntax error: A local function named 'MyMethod' cannot be declared in this scope because it would give a different meaning to 'SetEvent', which is already used in a parent or current scope to denote something else
So to confirm there is no way to do overloading?

My bad. Yes, the C# compiler doesn't allow you to do that.