[RFC FS-1060 - Nullable Reference Types] discussion

[cartermp]

Discussion for https://github.com/fsharp/fslang-design/blob/master/RFCs/FS-1060-nullable-reference-types.md

The biggest user-facing change in the RFC is the discovery that syntax like this:

let len (str: string | null) = ...

Would be a breaking change. For example, this code is valid F# today:

let f2 (_ : string | null) = 1

And there are likely other variants of this that people could conceivably have in their code.

Thus, we are currently thinking that we will match the C# syntax:

let len (str: string?) = ...

Since it is not a source-breaking change.

[dsyme]

Would be a breaking change. For example, this code is valid F# today:

There are other examples, e.g. compare

type X = A of string | B | C

and

type X = A of string | null | C

Here the reduction to perform on | is very non-obvious in the context of Yacc LALR parsing

Requiring parenthesization may be feasible but definitely fiddly to implement

type X = A of (string | null) | C

[isaacabraham]

I don't think that this would be immediately obviously to users what the parentheses are for here, either.

[dsyme]

Other options might be

let len (str: Nullable<string>) = ... // for reference types compoiled as `string`, for value types Nullable<_>
let len (str: string or null) = ... 
let len (str: string || null) = ... 

[jamessdixon]

Do option types "None" still map to Null in C# world? And therefore Some t maps to this new Non Nullable? That would be a breaking change in many places in my code base

[cartermp]

@jamessdixon You can read about this a bit here: https://github.com/fsharp/fslang-design/blob/master/RFCs/FS-1060-nullable-reference-types.md#interaction-with-usenullastruevalue

In short, None will still map to null, and the only change we may consider doing is warning on Some null in F# code. But existing C# code that interops with F# options must still compile as it does today.

[JordanMarr]

My feedback:

I really like the "?" notation because it is concise, and I'm already used to it from C#:
let len (str: string?) = ...

Unimportant side question: would we would have to open System to use the ? notation?

If below is also valid, then does that mean we could also substitute T? anywhere we would otherwise write Nullable<T>? That would be a very welcome side benefit from this IMO.
let len (str: Nullable<string>) = ...

I really like the twist that the emitted warnings can also be tuned to emit as errors. I would definitely want that to be the case all the time, but I can understand why it might not be the default. Nice to have the option though.

[cartermp]

@JordanMarr

Would we would have to open System to use the ? notation?

No, it would have to be available without opening that namespace.

If below is also valid, then does that mean we could also substitute T? anywhere we would otherwise write Nullable<T>?

Perhaps. We're considering how to rationalize nullable reference types with nullable value types (Nullable<'T>) and if syntax and analysis can equally be applied there. Note that Nullable<string> does not compile because the 'T is restricted to structs.

[cartermp]

@dsyme We'll also need to consider nullable operators: https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/symbol-and-operator-reference/nullable-operators

I think I'd discourage their usage, but it could feel weird that they didn't work with nullable reference types.

[dsyme]

@cartermp Nullable operators are primarily present for query expressions. Indeed Nullable in C# is primarily there for nulls in databases and LINQ

It may well raise lots of questions about nullability and database queries and database ORM mappers. And type providers too

[cartermp]

Yeah, I'm not keen on them being used outside of that context (lacking any data, my impression is they're rarely used already). But it'd be good to keep track of nonetheless.

[dsyme]

A note on inference subtleties...

In current F#, code such as

let f x = match x with null -> 1 | _ -> 2

places a T: null constraint on the inferred type. e.g.

> let f x = match x with null -> 1 | _ -> 2;;
val f : x:'a -> int when 'a : null

That is, in the absence of other information F# 4.5 infers a definition-site annotation on the type variable itself.

If the input type is known to have a use-site nullable annotation, then it seems reasonable to allow this:

let f (x: string?) = match x with null -> 1 | _ -> 2;;
val f : x:string? -> int

This is not so bad and is likely the behaviour we will expect, but it requires type annotations in a way that is unusual for F# code: adding the type annotation fundamentally changes what the code is about (from definition-side nullability to use-site nullability). This is somewhat consistent with how units of measure work in in F#.

Note that in a fully fledged version of a use-site nullness feature you would expect

> let f x = match x with null -> 1 | _ -> 2;;
val f : x:'a? -> int

However this would be a breaking change and in any case nullable annotation code can't be generic like this. If we did allow it (eg. in inlined code) then a type annotation would be needed:

> let inline f (x: 'T?) = match x with null -> 1 | _ -> 2;;
val f : x:'T? -> int

Indeed this should be allowed (it is compilable once 'T is known to be either a struct or a not-struct)

> let  f (x: 'T? when 'T : not struct) = match x with null -> 1 | _ -> 2;;
val f : x:'T? -> int when 'T : not struct

> let  f (x: 'T? when 'T : struct) = match x with null -> 1 | _ -> 2;;
val f : x:'T? -> int when 'T : struct

[dsyme]

A note on inference... In the prototype the following gives a warning because obj is used as the default for the otherwise-uninferred type of null

    let y0 = isNull null 

I suppose we should use obj? if the unconstrained type inference variable has a 'T : null constraint as is does in this case.

In contrast the following does not give a warning:

    let y0 = isNull (null: obj?)

[dsyme]

A note on extern declarations.... The default interpretation of extern declaration syntax in F# should perhaps be nullable. Or we must allow the explicit declaration of nullable pointers

From CompilerLocationUtils.fs with nullable checking on:

    [<DllImport("Advapi32.dll", CharSet = CharSet.Unicode, BestFitMapping = false)>]
    extern uint32 RegQueryValueExW(UIntPtr _hKey, string _lpValueName, uint32 _lpReserved, uint32 & _lpType, IntPtr _lpData, int & _lpchData);

Here is RegQueryValueExW accepting nullable strings or not?

[dsyme]

I'm currently working on compiling FSharp.Build with null checking on.

A note on box and unbox and the obj type. There is the question about whether these have their signatures updated to this:

        val unbox : value:obj? -> 'T
        val box : value:'T -> obj?
        val tryUnbox : value:obj? -> 'T option

However I actually feel that for most F# programming the type obj should always be interperted as including nulls. This type is used when reflection, casting etc. is used. Given we've thrown away type information it seems to be wrong to be trying to track nullness.

[dsyme]

Now compiling up FSharp.Compiler.Private using nullness checking prototype. Just looking at the first few messages.

Here's one example of a place where we need to consider how we will update FSharp.COre:

    type StructuredFormatDisplayAttribute =
        inherit Attribute
        new : value:string-> StructuredFormatDisplayAttribute
        member Value: string

SHould Value here be string or string? or oblivious. In the current prototype it is string i.e. non-null

[7sharp9]

Mixing optional params and nullable params with the ? in member definitions is going to lead to a lot of confusion, as in the optional syntax:

            ProvidedProperty(
                name, 
                propertyType, 
                getterCode = (fun args -> Expr.FieldGet(args.[0], field)),
                ?setterCode = blah

[dsyme]

In what context?

In error messages, e.g. "Nullness warning: Go direct to Tony Hoare, if you pass go collect £1B"

[cartermp]

I think I would prefer not having any kind of text-based classification of the warning here. I'd much prefer the message itself just contain the relevant information, such as a type mismatch. But I'll give this a feel with a prototype.

[dsyme]

I think I would prefer not having any kind of text-based classification of the warning here. I'd much prefer the message itself just contain the relevant information, such as a type mismatch. But I'll give this a feel with a prototype.

Yes it's partly for my own purposes right now, the /checknulls flag applies to about 5 different warnings, all labelled with that. I'm not sure if we're going to want to keep /checknulls or not at the moment it's really useful.

[boeremak]

I don't know if this is still a thing, but I just saw the talk Philip gave to NDC about this. In the talk the first question he gets is why not treat them as Option values instead of building a whole new thing.

He answers that it will be a lot of work, especially in the interop layer, and I can see that. But I just wanted to say that it really feels like a wasted opportunity to build a new thing when we already have the Option type that, for all intents and purposes, does the same thing.

Regarding the interop code, when handled a null now it will already crash right? There is just a convention that you won't be handled a null, it is not enforced. So if you treat the nullable type as an Option it would be possible to do an Option.bind on all the parameters and when a null is found it will just crash like it would do now. There are probably smarter ways of doing this, I am by no means a great programmer or anything.

As for backwards compat, these types are only in C# 8 right if I understand correctly? So they are only found in the most recent .Net versions, can't we do a version check somewhere.

I am not an expert by any means, just my 2 cents.

[cartermp]

@boeremak Implicitly treating any nullable reference type as an Option<reference-type> would mean massively breaking changes when a package adopts NRT. For example, any string you consume from a library today would now be an Option<string>. I found 14 cases of this a small (100 LoC) file in the F# tooling already; it terrifies me to think how many breaking changes that would cause for a large codebase.

Additionally, the way that the option type is compiled into .NET metadata means that it must be a nullable reference type, since it uses UseNullAsTrueValue to emit the None case as null. C# consumers must check for null. If this were to change, it would be a source breaking change for any C# consumer of F# options.

[boeremak]

@cartermp I did some more reading about the C# rationale and it seems they want to transition to a place where F# already is, namely things can't be null and have to be initialised when you use/instance them. At least that is what I got from reading this: https://devblogs.microsoft.com/dotnet/nullable-reference-types-in-csharp/

Is that correct?

Another thing, seems like quite a change especially if you keep the IL output the same. So it is purely enforced in the tooling? A string? in IL is just string. How are they doing VB.net? Or isn't that happening?

I did some more reading and these are good points:
fsharp/fslang-suggestions#577 (comment)

fsharp/fslang-suggestions#577 (comment)

And is exactly my thinking.

Edit: Another option: why not keep it the way it is, everyone already does null checks ( i hope :) ) when things come from C# and when it comes from F# it can't be null anyway and just forgo this completely? Maybe add some smarts to intellisense to say "this has been compiled in C# 8 so you probably don't need to do null checks on this".

I don't know the feasibility of this though.

One think I would really dislike is for F# to accommodate C# in a way that makes the core language cumbersome to use. I really do like the language.

Edit2: If this has already been discussed, I am sorry I just saw the talk today.

[cartermp]

I suggest reading the Motivation and Principles sections detailed here for a thorough rationale of the feature from an F# and .NET perspective: https://github.com/fsharp/fslang-design/blob/master/RFCs/FS-1060-nullable-reference-types.md#motivation

Much of the rest of the spec is concerned more with details on what needs to be done and what they design will be, but this prelude is the framing for the feature.

[GratianPlume]

Would be a breaking change. For example, this code is valid F# today:

There are other examples, e.g. compare

type X = A of string | B | C

and

type X = A of string | null | C

Here the reduction to perform on | is very non-obvious in the context of Yacc LALR parsing

Requiring parenthesization may be feasible but definitely fiddly to implement

type X = A of (string | null) | C

but I think it more forward-looking

[charlesroddie]

The indications are that F# may not support NRTs at the launch of .Net 5 and that's going to be a large problem for F# as this is the recommended timeline for libraries to adopt NRTs.

The suggestions in the RFC are good, but not all are necessary to support in V1 of this feature. It's important to get this feature to a minimal spec to reduce the effort to get V1 out.

Suggestions for cuts to the RFC / moves to a "Future" section

"Partial alignment of nullable value types and nullable reference types."

This can be cut. Value types don't need a T? syntax for NRTs to be consumable in F#. The fact that generic methods over NRTs are harder to write and require a when 'T : not struct consraint is not a big problem and it's also a problem that C# has to deal with.

Omitted NonNull in pattern matching

It is also proposed that the NonNull can be omitted for variable patterns preceded by an unqualified null pattern.

Having x nonnull in match nrt with | null -> ... | x -> ... is a convenience that can be added later. It's also a little controversial so better to wait to see initial usability feedback. Requiring | null -> ... | NonNull -> ... is OK for V1.

Null ambivalence (obliviousness)

If necessary this can be dropped from V1 since it could add a lot of complexity in implementation. Especially droppable if there is a rollout phase in F# where the feature is opt-in: early adopters can see if obliviousness causes significant difficulty.

Unresolved question: nullability warnings on a per-file basis?

This can easily be scoped out of V1.

Unresolved question: UseNullAsTrueValue

This is an F# flaw and not C#/.Net's responsibiltiy to fix. Have these type exposed as NRTs and have a migration path for Option.

[Happypig375]

I think we should adopt the same syntax as https://github.com/fsharp/fslang-design/blob/main/RFCs/FS-1092-anonymous-type-tagged-unions.md with the required parentheses like (string|null).

[charlesroddie]

string? is established in C# and so in descriptions of NRTs out there.

NRTs are a well thought out feature that has consistent support across .Net languages. Typical codebases will incorporate NRTs for good reasons, by consuming .Net libraries, while the valid use cases of tagged unions are vanishingly small.

With this suggestion, people may think: now I am using string|null here, a well-supported .Net feature mainly used for interop, I am going to use string|int there, without realizing that it is a very different F#-specific feature which completely breaks interop.

[Happypig375]

string? is established in C#

And in F# we wish to promote option over null.

NRTs are a well thought out feature

I'd say that nullable reference types just have a similar level of type safety as anonymous type tagged unions, as nulls can still sneak in from third party sources, or from code that fake non-nullness like IEnumerator.Current.

that has consistent support across .Net languages

Of all .NET languages, only C# supports it at the moment. This also says a lot about interoperability with VB.NET, IronPython, PowerShell, etc.

the valid use cases of tagged unions are vanishingly small

The number of valid use cases for nullable reference types of typical F# code that don't have lots of interop boundaries would be similar to anonymous type-tagged unions (not much). Moreover, type inference diminishes the need for writing type signatures over and over.

string|null would encourage the use of string|int

Using string? just like in C# would also quickly expose the inconsistencies between nullable value types and nullable reference types, especially in generics. The C# team still chose the ? suffix for indicating nullability for reference types though, because of its meaning, rather than implementation. (string|null) vs (string|int) falls in the same bucket here.

[Swoorup]

string|null requires type-level literal support which is a different RFC on its own. But despite, what is stopping it to have the same implementation as Nullable<string>?

[Happypig375]

string|null requires type-level literal support which is a different RFC on its own. But despite, what is stopping it to have the same implementation as Nullable<string>?

Nullable conflates itself with the built-in struct type.

[chkn]

+1 .. I think this is more aligned with the F# philosophy.

It might also eliminate some hand wavy complexity by treating (string | null) as simply an anonymous union. Just like with other union types, the compiler will warn you if you don't handle all cases.

And because anonymous unions are erased to a reference type, the type (int | null) would just be erased to obj, which seems much simpler than dealing with Nullable<_> and generics and the different IL representations.

[bugproof]

One major reason why I'll probably never adopt F# is a wide range of C# libraries that use NRTs to mark "optional" properties. F# uses Option type but it seems useless and counter-intuitive when you consume C# libraries just forcing you to use null again. There's so much pollution/inconsistency you would need to rewrite the entire library in F#. I'd rather rewrite them in Rust instead.

I think it would be interesting if NRTs could be interpreted as option by F#. But I'm not sure if that would have any drawbacks.

[smoothdeveloper]

option in F# works kind of the same as System.Nullable<T> only F# doesn't have the coalescing and type coercion aspects of C#, and it doesn't have the restriction of System.Nullable<T> which only applies to value types.

When F# will support NRT, I was considering it may be preferable to have, later on, a code generation that allows to translate option and ValueOption into NRT annotations in the IL, rather than make NRT "be interpreted as option" which I don't understand fully.

Would you mind to expand on the other reasons for never adopting F#, and which wide ranging C# libraries uses NRT?

Could you expand on your idea for F# to interpret NRT as option?

Thanks.

[bugproof]

I think if someone consumes a C# library that makes use of NRT from F# it would benefit F# users if they were treated as Option<T>. NRTs were added to C# only because there was no way to tell if a property should really be null but it doesn't do much else.

In F# there was always Option<T> type which is and will always be much better than NRT and anything C# team will come up with in the future like required keyword.

Would you mind to expand on the other reasons for never adopting F#

I find that bridge between C# -> F# too convoluted. I'd like to use Option<T> everywhere in my code base but I know it's simply impossible because there will always be some C# lib I will need to use and it will feel just like writing C# again but with a weirder syntax. If there was a way to write code without using null ever again just like Rust but .NET ecosystem that would be nice.

Yes theoretically I could make my wrappers/bindings that hide null usage and map None to null but it's too time consuming and I'd rather focus on writing code than maintaining bindings of different libs.

[cartermp]

See here for the consideration: https://github.com/fsharp/fslang-design/blob/main/RFCs/FS-1060-nullable-reference-types.md#alternative-option-or-valueoption

[bugproof]

It would also be nice if F# types could be consumed from C# without a dependency on FSharp.Core. Maybe if there was a tool based on Mono.Cecil or dnLib that removes the dependency and converts Option<T> to NRT that would be great. I maintain one library that has a lot of data only classes and a lot of optional properties. If I could re-write it in F# and make it usable from both without additional dependencies it would be a win-win for everyone.

Or maybe write a tool that does the opposite? Generate F# compatible packages that convert NRT/required stuff to Option<T>. That would be interesting. Like an F# binding generator tool that could even be extended further.

Like I said the only thing that's stopping me from adopting F# in any way is there's no elegant way to make both of the languages work together in a harmonic way. I think a better tooling is needed to make it easier for people to adopt F# be it C# -> F# binding generator (fsharp-bindgen) or the other way around F# -> C# without FSharp.Core dependency so libraries could be created in F# but still be 100% compatible with C# without additional dependencies..

Even if in practice you can reference C# lib from F# and F# lib from C# the experience is horrible and I'm surprised such tools don't exist.

[En3Tho]

Why dependency on FSCore is such a problem? I don't really see much difference between referencing C# library nugget or F# library nuget. Okay, F# libs have one more dependency. Is that an issue at all?

[7sharp9]

For me it’s never been that much of an issue turning a potential null into F# friendly usage, it’s not very often and when I do encounter it it’s just a Option.ofNull etc. I suppose it depends on the domain?

…

On 21 Jun 2022, at 00:06, bugproof ***@***.***> wrote:  I think if someone consumes a C# library that makes use of NRT from F# it would benefit F# users if they were treated as Option<T>. NRTs were added to C# only because there was no way to tell if a property should really be null but it doesn't do much else. In F# there was always Option<T> type which is and will always be much better than NRT and anything C# team will come up with in the future like required keyword. Would you mind to expand on the other reasons for never adopting F# I find that bridge between C# -> F# too convoluted. I'd like to use Option<T> everywhere in my code base but I know it's simply impossible because there will always be some C# lib I will need to use and it will feel just like writing C# again but with a weirder syntax. If there was a way to write code without using null ever again just like Rust but .NET ecosystem that would be nice. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[7sharp9]

ILMerge does that.

…

Sent from my iPhone

On 25 Jun 2022, at 11:06, bugproof ***@***.***> wrote:  It would also be nice if F# types could be consumed from C# without a dependency on FSharp.Core. Maybe if there was a tool based on Mono.Cecil or dnLib that removes the dependency and converts Option<T> to NRT that would be great. I maintain one library that has a lot of data only classes and a lot of optional properties. If I could re-write it in F# and make it usable from both it would be a win-win for everyone. Or maybe write a tool that does the opposite? Generate F# compatible packages that convert NRT/required stuff to Option<T>. That would be interesting. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[ken-okabe]

To work around the issue, I did the following:

https://github.com/ken-okabe/vanfs?tab=readme-ov-file#nullable

NullableT

type NullableT<'a> =
    | Null
    | T of 'a
    member this.Value
            = match this with
            | Null -> failwith "Value is null"
            | T a -> a

let NullableT a =
    match box a with
    | :? NullableT<'a> as nullable -> nullable
    | _ -> T a

Usage

let nullable1 =
    Null

let nullable2 =
    NullableT "hello"

log nullable1
// Null
log nullable2
// T hello
log nullable2.Value
// hello