Thoughts about Class proposal

[samdphillips]

I cannot make it tonight to the Zoom discussion, but here are some questions/comments I had.

• Are interfaces closer to field-less objects than just collections of method signatures?
• An annotation on an interface method only seems to apply in the default case and not if the implementer overrides the method. Should we look at or consider how annotations on different levels are handled?

Here is the toy example I've been experimenting with, and its expanded code:
https://gist.github.com/samdphillips/2ceb10fbe05e18c0c7cb00d920b48a5d

[mflatt]

Are interfaces closer to field-less objects than just collections of method signatures?

Field-less classes, right? Yes. They can have method implementations as well as unimplemented methods, though, and they support multiple inheritance (because a class can implement any number of interfaces, and an interface can any number of superinterfaces).

An annotation on an interface method only seems to apply in the default case and not if the implementer overrides the method. Should we look at or consider how annotations on different levels are handled?

Yes, this is one of the main issues. More in the meeting summary...

[samdphillips]

Field-less classes, right?

Yes that's what I meant.

They can have method implementations as well as unimplemented methods

Maybe I missed it in the documentation, but I didn't see how to construct an interface method without a default implementation.

[mflatt]

Currently, the keyword is unimplemented, but we discussed changing it to abstract.

Also, unimplemented just expects a method name right now, but if result annotations are pushed to overriding methods, then it makes sense to at least have a result annotation there (not sure about arguments).

[mflatt]

During the meeting, we wrote some examples to go through most elements of the draft design.

Some ideas from the meeting:

• Make a class final by default, even if its a subclass. Consider making methods final by default, too.
• Use abstract instead of unimplemented, and require a specific declaration on class if it has abstract methods (analogous to Java, but maybe even per-method where an obligation is propagated to subclasses), and make methods declared in an interface abstract by default.
• Probably, we can let a subclass use a default constructor even when a superclass has a custom constructor, as long as the constructor obeys some constraints. Otherwise, adding a custom constructor to a class (e.g., to perform some validation or initialize a new field) would generally break subclasses unnecessarily.
• Try using super to refer to the internal implementation in constructor, binding, and annotation clauses, instead of having the programer write one. With that change,constructor also can have a fun-like form, instead of requiring an explicit entry-point fun.
• A method accessed from a class namespace could be curried, instead of having the target object be an extra initial argument.
• Propagating result annotations from a method to overriding methods may be a good idea. With that change, it makes sense for a static method call to have static information about its result. It's not yet clear what to do about consistency for method arguments, though; one possibility is to do nothing for arguments and only propagate result annotations.

[mflatt]

I've pushed most changes suggested by the first four bullets.

Methods are not final by default, and interface methods are not yet abstract by default. I'll pick that question up again when working on the last bullet.

To go along with bullet 3 and the idea that you want to evolve a superclass with a new immediate field, there's now a private modifier for a field in the initial set of field_specs. If you don't provide a default value for a private field, then a custom constructor is required. But providing a default value doesn't force a subclass to have default values for all by-position arguments.

The changes for super are somewhat different than suggested, partly to make everything consistent with private in a field_spec. A custom constructor doesn't change all that much, except super is the keyword for calling an underlying constructor, as suggested, and it accepts private fields from the field_specs as well as public ones. Meanwhile, the name bound by internal acts like super when used as a constructor, including currying when the class has a superclass. There is no super for annotation and binding specializations, and you have to use internal; but instead of the weird syntactic currying of the previous design, the internal identifier as a binding or annotation covers only the fields (including private ones) of the immemdiate class. There are && binding and annotation operators for combinding a use of the internal name with a superclass name (normally the public superclass name, but direct use of an internal name for a superclass might make sense sometimes).

Meanwhile, names bound by internal now consistently provide access to private fields.

[mflatt]

The last bullet is now addressed with propagation of result annotations.

For now, I'm not planning to do anything with argument annotations or change method access to be curried (the next-to-last bullet). My plan is to write a little more overview documentation and then merge to the main prototype branch.

[mflatt]

A method accessed from a class namespace could be curried, instead of having the target object be an extra initial argument.

It did not occur to me during the meeting, but one motivation for the current approach was to support the way l.length() and List.length(l) work. Writing List.length(l)() doesn't seem right, for example.

[AlexKnauth]

Will they be expressive enough to allow me to define a lazy_method entry point with behavior such that

method add: lazy_method:
              if today_is_tuesday()
              | fun(o): Posn(y + o.y, x + o.x)
              | fun(o): Posn(x + o.x, y + o.y)

Would evaluate today_is_tuesday() only the first time p.add is evaluated?

That's a case where just turning it into a property method would fall short. I believe I would need special entry-point cooperation to make this happen.

[jackfirth]

@AlexKnauth Why not write this instead?

method add(o):
  if today_is_tuesday()
  | Posn(y + o.y, x + o.x)
  | Posn(x + o.x, y + o.y)

I realize the suggestion is a bit trite, but my point is: if you want something evaluated every time a method is called, why are you trying to put it outside the method body in the first place? What use case are you thinking of?

[mflatt]

When user-defined entry_points are supported [...]

Those both seem possible, especially assuming those fun positions are entry-point positions.

As it exists currently internally, an entry-point macro gets an extra argument with three parts: a list of identifiers for arguments that the context wants added to the start of the function (e.g., an argument that will be connected to this), a compile-time function that takes a body expression and returns a replacement body expression (e.g., to make this work, to add a check on the result value), and a boolean indicating whether the function should be tagged as a method for arity-error purposes. I'm not sure about that last part, and I've gone back and forth on whether an entry point should expose argument and result annotations in its expansion (currently not expected). An entry-point macro can pass along the requests that it receives or create a different entry-point request for nested entry points.

[AlexKnauth]

Why would those fun positions are entry-point positions matter? Unless you're intending if to be an entry-point form too and propagate that to its branches, it should be an expression. The way I intended dynamic_method and lazy_method would be entry-point forms that take expressions, not nested entry-point forms.

[mflatt]

Still ok. You're back in the position of managing the trade-offs, and managing them differently without an entry-point distinction, but I see no reason that you couldn't implement anything that makes sense.

[mflatt]

#249 is now merged. For someone just starting, the documentation built by the pkg-build is probably the easiest way to get a sense of the current implementation:

https://docs.racket-lang.org/rhombus/Classes_and_Interfaces_Overview.html

https://docs.racket-lang.org/rhombus/Classes_and_Interfaces.html

[jackfirth]

I've started working on #201 using the new class system. I've written up some issues I encountered below.

Views are hard to make

In implementing an ArrayList class, I'd like to have a sublist view of the array list. Some useful functionality for manipulating an ArrayList is only possible through the view. For instance, to delete a range of a list, you'd write list.sublist(start, end).clear(). This kind of indirect exposure of functionality through views makes it possible to expose a lot of features without vastly increasing API surfaces. Implementing this requires that views access internals of their backing classes. I can't figure out how to implement that behavior in Rhombus: classes only seem to have private and public for protecting accesses to their members. There's a few possible approaches to this problem:

• Add Java-style inner classes. An inner class is a class declared within another class, whose instances are inherently associated with an instance of the outer class. Collection views and collection iterators are natural use cases for inner classes, as they need to have pointers to the outer class and access private members of the outer class to implement their behavior.

• Expand the kinds of access controls class members can have. Perhaps there could be a modifier to declare that a field or method should be visible to everything within the same module, but not outside it.

Fields are hard to use

Consider this ArrayListIterator class, which implements the iteration behavior for an ArrayList:

class ArrayListIterator(backingList :: ArrayList, version :: Integer):
  implements Iterator

  private field index :: Integer: 0

  constructor(backingList :: ArrayList):
    super(backingList, backingList.version)

  override method hasNext():
    when version < backingList.version | error("concurrent modification")
    index < backingList.size

  override method next():
    when version < backingList.version | error("concurrent modification")
    unless index < backingList.size | error("no such element")
    let element: backingList.get(index)
    index := index + 1
    element

Each ArrayList has a version counter that increments when the list undergoes a size-changing operation. The ArrayListIterator notes the version of the class when iteration started, and it checks that the list hasn't changed each time it accesses an element. This lets it detect concurrent modifications that would invalidate its iteration behavior. However, implementing this behavior is a little awkward. It requires a custom constructor because the version constructor argument is intended to be derived from the backingList argument. This can be circumvented a bit by declaring the class with class ArrayListIterator(backingList :: ArrayList, version :: Integer = backingList.version), but clients of the class aren't supposed to be able to pass the version argument directly. It would be a lot more convenient for my purposes if fields could be initialized to values dependent on constructor arguments or other already-initialized fields. Mutable fields would then need a marker like mutable to distinguish them from immutable fields that just hold on to an instance-wide variable for convenience purposes. That would let me declare ArrayListIterator like this:

class ArrayListIterator(backingList :: ArrayList):
  implements Iterator

  private field version :: Integer: backingList
  private mutable field index :: Integer: 0

[jackfirth]

Update: private constructor arguments totally solve that problem. This begs the question though: why declare something a private field instead of a private constructor argument, or vice-versa?

[mflatt]

An advantage of a private field clause it that it can be generated by a macro, possibly in combination with other clauses, such as an accompanying method clause. An advantage of a private field declared at the top is that it can be ready only and initialized from other initial values. It would be nice if there was just one way to declare fields, but having both of these is one way I found to balance some competing goals.

[jackfirth]

The macro generation case is compelling, and it brings to mind a use case that has issues with the current behavior: consider a memoized macro that wraps a method to memorize its results, by caching results in a per-instance map. So something like this:

class Foo():
  memoized method foo(a, b, c):
    ...

// expands to

class Foo():
  private field fooCache: MutableMap()

  method foo(a, b, c):
    let key: List(a, b, c)
    unless fooCache.has_key(key)
    | fooCache[key] := fooUncached(a, b, c)
    fooCache[key]

  method fooUncached(a, b, c):
    ...

As I understand it, this doesn't work under the current semantics because field initializers are evaluated once per class, not once per instance. If they were once per instance, this would work. And if they were once per instance and their initialization expressions were only allowed to refer to already initialized fields, I think there wouldn't be any uninitialized object problems.

[mflatt]

I've made the simple adjustment of having the right-hand side of field get evaluated on each instantiation, so the above example can work.

Allowing access to fields is not so straightforward. Technically, the object and its fields don't yet exist at the point where the expressions are evaluated. Initial values for some fields are available, and we could introduce new bindings around each right-hand side to access the initial field values that are ready (using the same name as the field). I don't have a strong argument again, but it would be an unusual scope pattern, and it feels like the kind of thing that turns out bad.

[jackfirth]

With that change to fields, I find myself wanting to declare fields as immutable. Or even better, make fields immutable by default and require mutable field for fields I want to be mutable.

[mflatt]

Implementing this requires that views access internals of their backing classes. I can't figure out how to implement that behavior in Rhombus: classes only seem to have private and public for protecting accesses to their members.

When you give a class an internal name, then you can use that name to access private fields. I think that's probably what you're looking for in this case.

#lang rhombus

export:
  Posn // but not _Posn

class Posn(x, y):
  internal _Posn
  private field internal_storage: 5

  method get_insider():
    PosnInsider(this)

class PosnInsider(p):
  method use_original():
    (p -: _Posn).internal_storage

Posn(1, 2).get_insider().use_original()

[mflatt]

A natural follow-up here would be: what if you want to have different sets of private methods that are accessible to different parties? The intent is that you'd group them into interfaces and use private implements.

[jackfirth]

A natural follow-up here would be: what if you want to have different sets of private methods that are accessible to different parties? The intent is that you'd group them into interfaces and use private implements.

Could you give an example of this?

[mflatt]

A represents one set of parties, and B represents another. The definition of C grants a access to anyone who can refer to _A, and b access to anyone who can refer to _B.

#lang rhombus

use_static

interface A:
  internal _A
  method a()

interface B:
  internal _B
  method b()

class C():
  private implements:
    A
    B
  private override a():
    "C a"
  private override b():
    "C b"

val c: C()

(c :: _A).a()
(c :: _B).b()

[jackfirth]

I've got some thoughts about properties. For starters, there's currently some inconsistencies in how properties are used. I expected this to work:

class Foo():
  property foo(): 5
  method test(): foo

But it raises the compile error foo: method must be called in: foo. Using foo() works, but that's inconsistent with how outside clients of the class use the property. Using this.foo works, but that's inconsistent with how a class uses its own methods and fields.

Additionally, for read-only properties, it's a bit weird I need to include the parentheses at all. Can we make this work?

class Foo():
  property foo: 5

For read-write properties, the current syntax for defining them is a little confusing. It wasn't obvious to me that it required using the case-arity function definition syntax, and it's a bit confusing to read because it's overloading the concepts of property mutability and function arity. I think I'd find something like this more readable:

property foo:
  get: 5
  set(x): doStuff(x)

Also, it's weird that setters can return values. I think all property setters should be required to return #void. Otherwise, the behavior of o.x := e becomes much less predictable to readers.

[mflatt]

These kinds of improvements for properties make sense to me. The current implementation was a kind of minimal layer over methods for something that I wasn't sure we'd want. But assuming that we keep properties, then the language support can be much better in these ways.

[mflatt]

Done, but that the new syntax for property with assignment looks like this:

property | foo: 5
         | foo := x: doStuff(x)

[jackfirth]

So I'm defining a List interface, and I want List(1, 2, 3) to construct an immutable list. Is there a way to overload an interface name as a function?

[mflatt]

Not currently, although I think that should be added to interface. More generally, there needs to be a way to change the binding of an identifier at some spaces and not others, but direct support in interface also makes sense.

I also think there should be a way to bind a construct as a syntactic form instead of a function, so that MyKindOfList[1, 2, 3] can be supported. Probably that takes the form of a constructor.macro clauses, where either constructor or construct.macro is allowed, but not both.

[mflatt]

I've pushed the change to add expression in both class and interface — instead of calling it constructor.macro, since there's not really a constraint that the identifier acts as a constructor. Also, annotation now works in interface.