[RFC FS-1093 Discussion] - Additional type directed conversions

[dsyme]

Discussion thread for RFC FS-1093 Additional type directed conversions

[Swoorup]

Wonder if this can also be extended to implicitly convert when accessing member interface properties/functions of a concrete type instance?

type IEvent = 
  abstract Name: string

type Leave = 
  | Leave 
  interface IEvent with 
    member x.Name = "Leave"

let evt = Leave
evt.Name // Error: The type 'Leave' does not define the field, constructor or member 'Name'
(evt :> IEvent).Name // fine

Extending it to anonymous type-tagged unions.

type Join = 
  | Join of address: string
  interface IEvent with 
    member x.Name = "Join"

let event2: (Join|Leave) = Join("foo@nsa.gov")
event2.Name

[charlesroddie]

The first example needs a separate language issue because it's not related to the known type information of this RFC isn't related to allowing direct access to interface members. Naturally there would be a lot of issues to resolve around name collisions. The second example is like the first except you would additionally need to make a DU to implement the interface.

[Swoorup]

It is however in the same umbrella of a known type auto upcasting, C# allows this (Not for default interface methods for some reason). With that said, happy for this to be a different RFC, given there appears to be difference in the F# can have interface members with same signature with the exception of different return types (C# does not allow for this to my knowledge).

The second example is of anonymous type tagged union, not a DU type. If all union cases implement the same interface, in theory it is fine to access the shared interface members.

[gusty]

I wonder if this feature would cause breaking changes in complex overload resolution, where automatically upcasts are a problem, and I'm using some tricks (wrappings) to avoid it. Maybe now those tricks would stop working.

Still, I think we need to have something like the opposite of the flexible type operator # to get full control of flexibility (on/off).

[dsyme]

I wonder if this feature would cause breaking changes in complex overload resolution, where automatically upcasts are a problem, and I'm using some tricks (wrappings) to avoid it. Maybe now those tricks would stop working.

Yes, it's possible, the conversions should be a tie-breaker of last-resort. I'll start to develop test cases along those lines.

The rules are not active in SRTP resolution.

[cmeeren]

From the RFC:

TODO: consider the exact range of type-directed conversions to be included, specifically whether numeric widenings and op_Implicit are to be included.

I would very much like op_Implicit to be included. Certain newer .NET APIs, such as those in ASP.NET Core, make frequent use of it. For example, many APIs in Microsoft.Net.Http.Headers make use of StringSegment arguments, where C# devs can just pass a string, but F# devs must explicitly call op_Implicit all over the place (or rely on a custom unary operator, as I do since I grew tired of the noise: CacheControlHeaderValue.Parse(!"no-store") where let inline (!) (x: 'a) : ^b = (^b: (static member op_Implicit: 'a -> ^b) x)).

Some popular 3rd party libraries also use op_Implicit. E.g. MassTransit uses RequestTimeout, which has an op_Implicit conversion from TimeSpan as well as int (milliseconds), seemingly for a simpler API with fewer overloads.

One thing to watch out for is op_Implicit conversions to another type. E.g. StringSegment has conversions to ReadOnlySpan<char> and ReadOnlyMemory<char>. Perhaps that might impact overload resolution in a negative way if op_Implicit is supported.

[dsyme]

I would very much like op_Implicit to be included.

Yes, I think we may need to include this, despite the downsides.

[dsyme]

My main concern about op_Implicit is how to avoid normalising its use in F# coding. I really feel it could be used very badly. One option is to place a warning on op_Implicit declarations in F#, which must be explicitly #nowarn'd.

I also have a nagging concern about allowing even int32 --> int64 widening, and indeed any additional type directed conversions. As a result the feature will include an opt-in warning to warn whenever a TDC is used, at all.

I feel an int32 --> int64 TDC this is something we do need to do for the longevity of F#, because int64 is now really the industry default and writing literals 64L all the time, even when int64 is known, is problematic. And if we're going to do int32 --> int64 we may as well do int32 --> single and int32 --> double.

It is difficult to make this apply only to literals, though that would be an option.

[gusty]

I feel an int32 --> int64 TDC this is something we do need to do for the longevity of F#, because int64 is now really the industry default and writing literals 64L all the time, even when int64 is known, is problematic.

In order to solve that, I think it's better to implement generic numbers rather than "auto-magical" conversions a-la C#. We have a discussion already about it fsharp/fslang-suggestions#737 (comment)

[charlesroddie]

The original language suggestion (automatic casting when type information is known) is narrow and uncontroversial. Whereas op_Implicits, automatic widening, generic numbers, a changed default for int, seem like separate and wide-ranging discussions where the options have very different tradeoffs and the best path is not clear. It seems like a subject for separate fslang-suggestions on op_Implicit and on how to deal with multiple integer types more conveniently.

[dsyme]

This RFC/PR does two things

1. Puts in place a general backwards-compatible mechanism for type directed conversions (and one that works in conjunction with the existing techniques to allow subsumption at some specific places)

2. Selects a particular set of type directed conversions to use. These are currently

   • the existing func --> delegate type directed conversions
   • the existing delegate --> LINQ Expression type directed conversions
   • upcasting
   • int32 --> int64/float32/float64
   • float32 --> float64
   • op_Implicit when both source and destination are nominal.

3. Implements an opt-in warning when any of these are used (outside existing uses of upcasting)

The choices in (2) and (3) are separate to the mechanism in (1). However the question of what conversions are in (2) at least initially is best discussed and trialled in the context of this RFC. So for now we should discuss both numeric widening and op_Implicit in the context of this RFC.

Practically speaking I think it's unlikely we will address numeric widening or op_Implicit except via the mechanism of this RFC.

automatic casting when type information is known is narrow and uncontroversial

To be honest, even this is somewhat controversial for me, or would have been done before - see for example some of the examples in the RFC where extracting tuple expressions to a let without a type annotation give an error. There are already examples of this in F# but they are relatively limited.

That said, it's less controversial than the others. So in order of controversy we have

AutoCasting < Literal widening < Integer widening < Float widening < Int-to-Float widening < op_Implicit widening

On integer widening

it's better to implement generic numbers ...

I understand this position, and we considered it for F# 1.0, though as I think we've discussed elsewhere I do not believe it's feasible to implement generic literals without making a significant breaking change - so I don't actually think it's a starter. The fact that 17 commits to type int immediately (if there is no known type) is very significant for a lot of F# code and there's no real way of escaping that.

It seems like a subject for separate fslang-suggestions on op_Implicit and on how to deal with multiple integer types more conveniently.

I'm fairly certain any discussion about op_Implicit and integer types would boil down to "add additional type directed conversions for op_Implicit and numeric widenings". I honestly don't see any other game in town that wouldn't later trend towards the use of this mechanism when further examples arose.

There are however things we can tune in this RFC, including

1. The exact set of type-directed conversions included
2. Whether warnings are given for these
3. Whether these warnings are opt-in or not
4. Perhaps limiting the use of type-directed conversions to some specific syntactic locations, e.g. arguments.

The intent of this RFC is to give a user experience where:

1. Interop is easier (including interop with some F# libraries, not just C#)

2. Numeric int64/float32/float64 data in tuple, list and array expressions looks nicer

3. Fewer upcasts are needed when programming with types that support subtyping

4. You don't need to be cognisant of the mechanism when coding in F#, though you may inadvertently use the mechanism when coding with numeric data or data supporting subtyping.

5. Inadvertent use of the mechanism should not introduce confusion or bugs.

That is, when you hit a library whose design activates it then "it's just nice". That is it's not really part of the F# programmer's actively used features of the language c.f. like the way the "you can use a lambda for a delegate" is not really an active part of the F# programmer's active feature set, it's just something that makes using particular libraries nice.

[dsyme]

I suspect that rule would result in breaking changes as there are some cases where type information is not complete.

In principle I don't think so, as a type-directed conversion (TDC) can only ever apply in cases where type checking of the overload was definitely going to fail. By preferring overloads that succeeded without TDC we first commit to any existing successful resolution that would have followed for existing code, and then allow new resolutions into the game.

This reasoning should however be carefully checked.

[dsyme]

I think you should add this to the RFC as alternative.

I've listed this in the RFC, thanks

[gusty]

By preferring overloads that succeeded without TDC we first commit to any existing successful resolution that would have followed for existing code, and then allow new resolutions into the game.

I think we discussed that technique once in a private conversation and you told me that something like failed overloads that stop failing might change the entire logic.

I think a scenario could be:

• Overloads for method A contains an overload that involves method B which also has overloads.
• Since B fails resolving another solution is picked.

Now, if B starts resolving, solution for A will be changed.

Hopefully I'm wrong, please correct me in that case.

[dsyme]

I think we discussed that technique once in a private conversation and you told me that something like failed overloads that stop failing might change the entire logic.

I believe this is true if changing overload resolution used for SRTP constraints. However type-directed conversions are not activated during SRTP constraint resolution. The intention is that they only activate if compilation genuinely was about to fail for a particular actual call.

The compat of these changes does need to be looked at very carefully however.

[gusty]

@dsyme I added a complete suggestion for enhancing the NumericLiteral mechanism here fsharp/fslang-suggestions#972

[dsyme]

@gusty @charlesroddie @cmeeren I've added sections to the RFC on motivation, design principles and so on capturing much of this discussion.

I'm particularly interested in examples where genuine cognitive inconsistency (i.e. confusion) or inadvertent bugs would arise in practive if this mechanism is activated, particularly with regard to:

1. examples where implicit numeric widening gets activated in some cases and not others, potentially intruding on the mental model of the programmer.

2. examples where op_implicit kicks in when using existing well-designed C# libraries in inconsistent ways, and again intruding on the mental model of the programmer.

Ideally these examples would use the RFC trial implementation to write example code afresh in an IDE where type hints are available. That is not exercises in potential confusion, but demonstrations of actual confusion.

I actually think genuine examples will be rare. But I'd like to think of what methodology we can use to assess this (beyond enabling it in preview bits).

[charlesroddie]

Example which seems OK in C# but might cause a problem in F#:

type T(x:option<option<unit>>) = class end

// CSharp
var q = new T(FSharpOption<FSharpOption<Unit>>.None);
var r = new T(FSharpOption<Unit>.None); // op_Implicit . Not ambiguous since the type is fully specified.

// fsharp

let s = T(None) // This would be ambiguous if op_Implicit is activated

[DalekBaldwin]

In regard to upcasts, copying my comment from the suggestion issue:

I've noticed that in many cases F# function types already automatically handle argument type contravariance but not return type covariance, presumably because inserting an upcast on a return value could spoil a proper tail call. Should tail calls be addressed in the RFC?

[dsyme]

Yes, definitely, it's a very good point.

Not all upcasts need spoil tailcalls though, only ones involving a representation change (boxing)

I'll add it as a thing i the RFC

[dsyme]

I've revised the RFC to match the implementation in clarifying some technical details.

There are some remaining issues. Specifically the presence of op_Implicit on some important .NET and F# types now gets noticed and activated-as-last-resort, allowing code of a kind not previously allowed in F#, e.g.

> let x : System.Nullable<int> = 3;;
val x : System.Nullable<int> = 3

> let x : option<int> = 3;;
val x : int option = Some 3

> let f (x: option<int>) = x;;
val f : x:int option -> int option

> f 3;;
val it : int option = Some 3

This is, frankly, "surprising" to the F# programmer as things stand today, though there is nothing particularly unsound about it. However I need to consider the full ramifications of this - likely we will disallow it for type option.

[cartermp]

The nullable stuff could be welcome in some ML scenarios where nullable data is used all over the place. The use of Nullable x everywhere does feel like the compiler is being a little pedantic. Perhaps the same could also be said for options, but I haven't felt that as much as the nullable stuff. Don't really have a good reason for that feeling :)

[dsyme]

Yes, the implicit Nullable interop seems valuable.

[dsyme]

Some topics to consider

• Should there be a TDC from int32<measure> to int64<measure> given that int32 auto-widens to int64?
• Should there be a TDC from function types _ -> _ to System.Delegate. This comes up in "minimal APIs" which take System.Delegate as an argument. See https://github.com/halter73/HoudiniPlaygroundFSharp/pull/3/files for an example of the kind of change this would allow. The RFC includes the existing TDC from function types to specific delegate types. However this doesn't get us to the base type System.Delegate

These could be done separately to this RFC, though perhaps it's worth folding them in now, unless they are approached some other way.

[charlesroddie]

Should there be a TDC from function types _ -> _ to System.Delegate. This comes up in "minimal APIs" which take System.Delegate as an argument.

If TDCs are transitive by design then it might be needed for consistency, but by itself it would be a flaw rather than a feature as having a marker of transitions from safe to unsafe code and a verbosity-disincentive to use unsafe APIs which use System.Delegate is important.

[dsyme]

I've been playing with this RFC and have mixed feelings, especially with regard to op_Implicit, and would like to discuss how to "tune" it.

The problem is really that op_Implicit becomes tempting for the F# coder, who often have the "make things as succinct as possible" ethos - which is fine in many ways but dangerous if it leads people into a kind of hellscape of implicit conversions.

For example, consider JsonValue from FSharp.Data. It becomes tempting to try to implicit-conversion the heck out of it in the name of getting F# code close and closer to JSON:

/// Represents a JSON value. Large numbers that do not fit in the
/// Decimal type are represented using the Float case, while
/// smaller numbers are represented as decimals to avoid precision loss.
[<RequireQualifiedAccess>]
[<StructuredFormatDisplay("{_Print}")>]
type JsonValue =
  | String of string
  | Number of decimal
  | Float of float
  | Record of properties:(string * JsonValue)[]
  | Array of elements:JsonValue[]
  | Boolean of bool
  | Null  
  static member op_Implicit(x: string) = String x
  static member op_Implicit(x: int) = Number (decimal x)
  static member op_Implicit(x: decimal) = Number x
  static member op_Implicit(x: float) = Float x
  static member op_Implicit(x: (string * JsonValue)[]) = Record x
  static member op_Implicit(x: (string * JsonValue) list) = Record (Array.ofList x)
  static member op_Implicit(x: JsonValue[]) = Array x
  static member op_Implicit(x: JsonValue list) = Array (Array.ofList x)
  static member op_Implicit(x: bool) = Boolean x

let x1 : JsonValue = "aaa"
let x2 : JsonValue = 3.0M
let x3 : JsonValue = 3.0
let x3b : JsonValue = true

However this is just yuck, because it really doesn't get you where you want to go, for example this fails:

let x4 : JsonValue = [| "a", 1 |] // no implicit conversion because type is not fully known at point of type failure and so this RFC doesn't apply

But this works!

let x5 : JsonValue = ([| "a", 1 |] : (string * JsonValue) array)

Ugh. I mean what have we gained here? Traded one kind of annotation for a worse kind.

The problem is that F# programmers will "fiddle" with this too much. They may also scatter op_Implicit around like confetti just in case it "helps" their users be more succinct (when in fact it will just confuse the heck out of their users). And it will always be unsatisfactory.

It's not so much about mechanism as policy. My belief is that we should do this change to make more warnings on by default:

1. Uses of last-chance conversions by subtyping (e.g. string --> obj) should give a warning that is OFF by default.

2. A use of a built-in implicit conversion (int --> int64 etc.) should give a warning that is OFF by default.

3. A use use of an F# code-defined op_Implicit conversion should give a warning that is ON by default, with a link to the same aka.ms page.

4. A use of a C#/IL code-defined op_Implicit conversion should give a warning that is OFF by default.

The logic here is that op_Implicit is part of .NET library design, but is not really part of F# methodology. Thus it is reasonable to assume .NET libraries have well-designed op_Implicit, and F# programmers aren't being led into the trap above. If they start scattering around op_Implicit they will hit warnings linking to documentation explaining that this isn't a great idea.

[charlesroddie]

Warning on definition does make sense. Maybe "Defining op_Implicit obscures the function which is being used to convert between types, and therefore should be used with great care. See ...".

On the off-by-default warnings in the original list: yes that makes sense. Actually I did not know that off-by-default warnings exist and was going to ask for this feature :). Is there a list somewhere?

[dsyme]

Here's a list off off-by-default warnings: https://github.com/dsyme/fsharp-presentations/blob/master/design-notes/off-by-default-warnings.md

[dsyme]

@charlesroddie I'm actually tempted by this suggestion: #525 (reply in thread)

That is warn on op_Implicit wherever it is used in non-method-argument position.

[charlesroddie]

That is warn on op_Implicit wherever it is used in non-method-argument position.

Ah I see. If I understand that right (default on warning for usage, except default off in methods), then I would personally find those good settings and wouldn't change the defaults.

[dsyme]

To clarify: here "default off in methods" means "default off when an implicit conversion happens for an argument when calling a method"

From the testing, this doesn't give a warning by default:

module CheckNoWarningOnUsingImplicitForAPI =
    type C() =
        static member op_Implicit(x: int) = C()
        static member op_Implicit(x: float) = C()


    type API() =
        static member M(c: C) = ()

    API.M(3)
    API.M(3.1)

[dsyme]

BTW here are some of the op_Implicit in .NET libraries:

System.DateTimeOffset op_Implicit(System.DateTime dateTime) 

various Decimal 

System.Nullable`1<!T> op_Implicit(!T 'value') 

System.Diagnostics.Tracing.EventSourceActivity op_Implicit(System.Diagnostics.Tracing.EventSource eventSource) 

System.DateTime op_Implicit(System.Xml.Schema.XsdDateTime xdt) 
System.DateTimeOffset op_Implicit(System.Xml.Schema.XsdDateTime xdt) 

System.Xml.Linq.XName  op_Implicit(string expandedName) 
System.Xml.Linq.XNamespace op_Implicit(string namespaceName) 

System.Data.SqlTypes.SqlBinary op_Implicit(uint8[] x) 
System.Data.SqlTypes.SqlBoolean op_Implicit(bool x) 
System.Data.SqlTypes.SqlByte op_Implicit(uint8 x) 
System.Data.SqlTypes.SqlDateTime op_Implicit(System.DateTime 'value') 
System.Data.SqlTypes.SqlDecimal op_Implicit(System.Decimal x) 

System.Drawing.PointF op_Implicit(System.Drawing.Point p) 
System.Drawing.SizeF op_Implicit(System.Drawing.Size p) 
System.Drawing.RectangleF op_Implicit(System.Drawing.Rectangle r) 

[dsyme]

@charlesroddie Another option would be to give an opt-in (off by default) warning when an op_Implicit is applied for arguments to members. That would correspond to the general existing rule in F# where some adhoc conversions and method overloading are applied at method overload resolution. What do you think?

[dsyme]

[ As an aside it's actually an interesting problem to think how we might one day enable a kind of "one annotation" effect for code producing Json values:

let x : JsonValue =
    [ { x = 1
        y = 3.0
        zs = [ { a = 1; b = 4.0 cs = [] } ] } 

where this might also include computed lists etc:

let x : JsonValue =
    [ { x = 1
        y = 3.0
        zs = 
          [ { a = 1; b = 4.0 } 
            if today() then 
                { a = 2; b = 5.0 cs = [2;3] } ] } 

In this setting I could also see the use for computed bindings, e.g.

let x : JsonValue =
    [ { x = 1
        if today() then
            y = 3.0
        zs = 
          [ { a = 1; b = 4.0 } 
            if today() then 
                { a = 2; b = 5.0 cs = [2;3] } ] } 

The idea here would be that JsonValue would be enough information (whether as type annotation or method/function call) to guide a very different interpretation of F# syntax.

It's probably not possible to do all this in the context of F# as it stands, but it's an interesting thing quite a lot like computed-list expressions. ]

[cartermp]

Putting this here for now

@dsyme @KathleenDollard I wonder what to do with the existing docs on flexible types: https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/flexible-types

Every code sample in there no longer requires the use of the constraint, rendering the doc as just noise.