Clarification on bool marshalling with LibraryImport

[sweemer]

Hello, I'm looking at the library design docs for LibraryImport, specifically the part on bool marshalling and I'm confused about a few things. Could somebody help clarify them for me?

We have decided to use System.Runtime.CompilerServices.DisableRuntimeMarshalling to enable our custom value type marshalling support. As a result, when a value type that has a bool field is passed to native code through source-generated marshalling, the bool field is treated as a 1-byte value and is not normalized. Since this default is a little odd and unlikely to be the majority use case, we're going to generally take a stance that all bool marshalling must be explicitly specified via MarshalAs or other mechanisms.

1. According to the Customize parameter marshalling documentation, MarshalAs has no effect when runtime marshalling is disabled. However, the paragraph above says that MarshalAs must be used when runtime marshalling is disabled, which seems to be a contradiction. How can these two statements be reconciled?

2. What about a managed delegate that either takes a bool argument and/or returns a bool value - does the marshalling need to be explicitly specified via MarshalAs when runtime marshalling is disabled? I ask because I'm not seeing any warnings when I pass such a delegate to a P/Invoke function, even if it has a UnmanagedFunctionPointer attribute.

Essentially there should be a table that clarifies the behavior of bool marshalling for all combinations of {runtime marshalling enabled/disabled, P/Invoke with and without source generation, and where used: function argument, return value, member of struct, argument to managed delegate passed to function, etc}.

[huoyaoyuan]

1. According to the Customize parameter marshalling documentation, MarshalAs has no effect when runtime marshalling is disabled.

MarshalAs has no effect on raw DllImport.

However, the paragraph above says that MarshalAs must be used when runtime marshalling is disabled, which seems to be a contradiction.

The source generator reads MarshalAs on LibraryImport declaration, and translate it into appropriate form in generated DllImport.

What about a managed delegate that either takes a bool argument and/or returns a bool value - does the marshalling need to be explicitly specified via MarshalAs when runtime marshalling is disabled? I ask because I'm not seeing any warnings when I pass such a delegate to a P/Invoke function, even if it has a UnmanagedFunctionPointer attribute.

Delegates are not support with runtime marshalling disabled. You must use function pointers. There's a warning CA1420 for it.
Note that runtime marshalling is decided by the assembly that declares the P/Invoke.

Essentially there should be a table that clarifies the behavior of bool marshalling for all combinations of {runtime marshalling enabled/disabled, P/Invoke with and without source generation, and where used: function argument, return value, member of struct, argument to managed delegate passed to function, etc}.

This is not a matrix because many of these are not supported when runtime marshalling is disabled. Instead, the behavior should be clarified case by case.

When runtime marshalling is disabled, bools in raw DllImport will always stay in the representation used in managed world: 1 byte.
The behavior of source generator is easily to inspect by viewing the generated code. Basically, it can manipulate the behavior of top-level argument and return value. Struct members are not manipulated.
The default behavior of runtime marshalling can be find in many old documentations.
I'm not sure about the UnmanagedCallersOnly and function pointer case. But basically, they should always defaults to the raw representation in managed world.

[sweemer]

@huoyaoyuan Thanks very much for your reply. I find it super confusing that MarshalAs has different rules for LibraryImport and DllImport, but your answer has helped to clarify it a bit.

Delegates are not support with runtime marshalling disabled. You must use function pointers. There's a warning CA1420 for it.

Could you clarify what this means? The following code doesn't generate any warnings for me. Are you talking about something else?

using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;

[assembly: DisableRuntimeMarshalling]

namespace DelegateTest
{
    [UnmanagedFunctionPointer(CallingConvention.Cdecl)]
    public delegate void ManagedFunc();

    public partial class NativeFuncs
    {
        [LibraryImport("lib.so")]
        public static partial void func(ManagedFunc managedFunc);
    }

    public class DelegateTestClass
    {
        private void MyManagedFunc()
        {
        }

        public void Func()
        {
            NativeFuncs.func(MyManagedFunc);
        }
    }
}

[huoyaoyuan]

The following code provides a CA1420 warning when using .NET 8 SDK:

using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;

[assembly: DisableRuntimeMarshalling]

namespace CSPlayground
{
    [UnmanagedFunctionPointer(CallingConvention.Cdecl)]
    internal delegate void FPMethod(int a, bool b);

    internal class Program
    {
        static void Main(string[] args)
        {
            Foo((a, b) => { });
        }

        [DllImport("Foo")]
        private static extern void Foo(FPMethod m);
    }
}

When running, it throws MarshalDirectiveException says that marshalling is not enabled, instead of DllNotFoundException.

[sweemer]

OK, so again it's just a difference in behavior between DllImport and LibraryImport. I still think that the documentation needs to more clearly explain the difference between these two. Anyway thanks for helping me understand that the rules are different for those two attributes.

[sweemer]

@huoyaoyuan I think you have answered my first question. Can I ask you to clarify the second question a bit more?

Let's say I have the following managed delegate that returns bool and a native func that takes it as an argument:

[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
public delegate bool ManagedFunc();

[LibraryImport("lib.so")]
public static partial void func(ManagedFunc managedFunc);

I understand that LibraryImport will convert the delegate to a function pointer using GetFunctionPointerForDelegate. However what I still don't understand is what will the bool marshalling rules be with runtime marshalling disabled - is it always 1 byte? What is the exact native type for it? And why isn't MarshalAs required on the bool return type like it is in other contexts with runtime marshalling disabled?

[jkoritzinsky]

Yes, when runtime marshalling is disabled, the runtime will only generate a simple thunk that handles the calling convention and getting the delegate from the function pointer/the function pointer from the delegate.

[sweemer]

And why isn't MarshalAs required on the bool return type like it is in other contexts with runtime marshalling disabled?

@jkoritzinsky Do you have any thoughts on this part? It would seem to me that either MarshalAs should be required in all contexts when using LibraryImport with runtime marshalling disabled, or it should not be required anywhere and always use the default rule.

[huoyaoyuan]

And why isn't MarshalAs required on the bool return type like it is in other contexts with runtime marshalling disabled?

MarshalAs has totally no effect on raw DllImport when runtime marshalling is disabled. It is somehow intentional since the behavior of legacy marshalling isn't that effective.

[sweemer]

@huoyaoyuan I'm talking about LibraryImport, not DllImport, and have edited my previous comment to make that more clear. I understand that MarshalAs has no effect on DllImport, but it does have an effect on LibraryImport and indeed is required for bool arguments and return values. I'm asking why those contexts and contexts in which managed delegates are passed via LibraryImport follow different rules.

[jkoritzinsky]

Do you have any thoughts on this part? It would seem to me that either MarshalAs should be required in all contexts when using LibraryImport with runtime marshalling disabled, or it should not be required anywhere and always use the default rule.

We decided to require MarshalAs for LibraryImport for a few reasons: The previous default for DllImport was Windows-centric and unintuitive. The new behavior with disabled runtime marshalling was uncommonly needed, different from the DllImport default, and uncommonly desired.

As a result, we decided to follow the same pattern we did with string marshalling: For .NET types with multiple equally valid and intuitive unmanaged representations, the user must select the type on their own.

Delegate types are not LibraryImport-style, they're still based on the runtime support (LibraryImport is source-generator based), so they don't follow these rules. There's not even a way to detect in the delegate marshalling that it's in a LibraryImport context vs a DllImport context.

[sweemer]

@jkoritzinsky Thanks for the explanation, that's helpful. I have a couple more questions just to make sure that I understand fully the design decisions.

As a result, we decided to follow the same pattern we did with string marshalling: For .NET types with multiple equally valid and intuitive unmanaged representations, the user must select the type on their own.

You're talking about LibraryImportAttribute.StringMarshalling right? If the pattern is the same for both bool and string, then just wondering why there's no LibraryImportAttribute.BoolMarshalling field too. It's probably not a big deal but that would maybe make it more clear that it's a rule linked to LibraryImport and not to marshalling in general.

Delegate types are not LibraryImport-style, they're still based on the runtime support (LibraryImport is source-generator based), so they don't follow these rules. There's not even a way to detect in the delegate marshalling that it's in a LibraryImport context vs a DllImport context.

As far as I understand, the marshalling rule for delegates with a bool argument or return type is 4-byte with runtime marshalling enabled and 1-byte with runtime marshalling disabled. Is that correct? If so then any reason why you decided not to require the user to specify the marshalling rule like for LibraryImport? There may be some users who stick with MarshalAs.Bool on LibraryImport but don't realize that their delegates are basically always MarshalAs.I1.

[jkoritzinsky]

As a result, we decided to follow the same pattern we did with string marshalling: For .NET types with multiple equally valid and intuitive unmanaged representations, the user must select the type on their own.

You're talking about LibraryImportAttribute.StringMarshalling right? If the pattern is the same for both bool and string, then just wondering why there's no LibraryImportAttribute.BoolMarshalling field too. It's probably not a big deal but that would maybe make it more clear that it's a rule linked to LibraryImport and not to marshalling in general.

We added the StringMarshalling property because strings are an extremely common type to use, whereas bools are significantly less common. Even for StringMarshalling, we only added two extremely common marshallers; not every built-in string marshaller is available there.

Delegate types are not LibraryImport-style, they're still based on the runtime support (LibraryImport is source-generator based), so they don't follow these rules. There's not even a way to detect in the delegate marshalling that it's in a LibraryImport context vs a DllImport context.

As far as I understand, the marshalling rule for delegates with a bool argument or return type is 4-byte with runtime marshalling enabled and 1-byte with runtime marshalling disabled. Is that correct? If so then any reason why you decided not to require the user to specify the marshalling rule like for LibraryImport? There may be some users who stick with MarshalAs.Bool on LibraryImport but don't realize that their delegates are basically always MarshalAs.I1.

We can't change the delegate marshalling rules for built-in interop due to back-compat issues, and a big part of DisableRuntimeMarshalling is that MarshalAs has no effect as the non-blittable marshalling logic is ignored for scenarios that use built-in interop marshalling. We plan on introducing source-generated marshalling for the delegate scenario at some point, but we haven't had time (we have issues open for it). In said source-generated delegate marshalling, we'll require MarshalAs explicitly to specify which marshalling rules are used for bools (like LibraryImport).

[sweemer]

Thanks again for the explanation. Last question now, I promise. How would source-generated delegate marshalling look different from what is already possible, namely passing a delegate as an argument to a LibraryImport function? I tried looking for the open issues but couldn't find them.

[jkoritzinsky]

Source-generated delegate marshalling would require a special attribute on the delegate type to signal that source-generation should be done. Then, like what we did for source-generated-COM interop, we'd enable all interop source generators to marshal a delegate with said attribute such that the source-generated code will be used instead of the built-in marshalling logic.