OEP8: Objects (dictionaries?), Geometry as data, and Module References

This proposal includes three basic features:

• Objects (dictionaries? structures? associative arrays?)
• Handling geometric objects as data
• Module references and module literal syntax

Credits

I believe that Doug Moen did the original implementation of core of the render(g) function. Andy Little, kwikius, did the original implementation of module literals and references.

Objects (dictionaries? structures? associative arrays?)

The idea is to add a data type equivalent to what JavaScript calls objects[*], Python calls dictionaries, and some other languages call structures or associative arrays.

Background

Current development-snapshot OpenSCAD has some support for an object data type, in support of the textmetrics, fontmetrics, and import(json) functions:

• An object has any number of member elements, each of which is of any data type, and each of which has a unique name.
• A member element is identified by its name, either in a constant identifier form obj.name or in an array-style form obj[expr]. The identifier form allows only identifier-legal characters; the array-style form allows any characters. The lookup is case-sensitive.
• An object can be traversed using for; for (name=obj) yields the names of the object's member elements.
• is_object(expr) returns true if the expression evaluates to an object.
• echo() and str() yield a printable representation of an object, but it does not correspond to legal syntax. The current snapshot does not include any way for an OpenSCAD program to create an object; objects are created only as return values from the textmetrics, fontmetrics, and import functions.

Literal syntax

This proposal adds a syntax for creating an object value:

• { name1: expr, name2: expr, ... }

In that form, names must be legal identifiers. Names can be enclosed in quotes to allow the use of non-identifier characters:

• { "name with spaces": expr, "name-with-dashes": expr }

Names there must still be constants. A name can be derived from an expression by enclosing it in parentheses:

• { (expr): expr, ... }

Object comprehension

Object comprehension resembles list comprehension:

• for (...) (expr): expr
  • Specifying the name as a constant is legal but mostly useless.
• if (expr) name: expr
• if (expr) name: expr else name: expr
  • Where name can be any of the three forms above.
• each obj_expr
  • Adds each of the members from the object value.
  • Roughly equivalent to for (k=obj_expr) (k): obj_expr(k), except that obj_expr is evaluated only once.
• let(name=expr, ...) ...
  • As for list comprehension, assigns the specified values and then evaluates the entry that follows.

These are currently in the object comprehension syntax but I'm not sure why:

• each <object-comprehension>
• ( <object-comprehension-elements> )

String form

echo() and str() are modified to return a string form of the object in the syntax above, using identifier form if possible and constant-string form if not.

object() function

An early proposal included an object() function, sort of the object equivalent of concat(). It is not clear whether object comprehensions make it unnecessary.

• object(name=expr, ...)
  • Constructs an object with the specified elements.
• object([[name_expr, val_expr], ...], ...)
  • Constructs an object with the specified elements. Note that the names can be expressions, and that the name-value list can be the result of arbitrary list construction.
• object(o1, o2, ...)
  • Constructs an object with the elements from the objects passed.

These can be combined arbitrarily. If a particular name appears more than once, the last definition is used, so

• object(o, a="new") makes a copy of o with a replaced.

An array-style entry with no value deletes the name from the accumulating object:

• object(..., [[name_expr], ...], ...)

thus

• object(o, [["a"]]) yields a copy of o without a.

Geometry as data

In current OpenSCAD, the only way to "return" a geometric shape is as the result of a module evaluation, and the only way to pass a geometric shape is as a child to a module evaluation. There is no simple way, for instance, to have an array of shapes and pick one as desired, or to have a subroutine return more than one shape, or to pass a variable number of shapes to a subroutine.

This proposal adds a new "geometry" data type.

Note that although geometry values superficially resemble modules, they are not modules. They are fully-evaluated CSG trees, with nothing executable remaining. The difference is analogous to the difference between "Design / Display AST..." and "Design / Display CSG Tree...".

Literal syntax

The syntax for creating a geometry value is:

• {{ ... arbitrary OpenSCAD ... }} Thus, e.g.
• g = {{ difference() { cube(10, center=true); sphere(7); } }}; Such a value can be passed around like any other value, put into lists and objects, passed to and returned from functions, et cetera.

Adding to the model

In the experimental build, there are currently two mechanisms for adding a geometry value to the model.

geometry_value;

Uttering a geometry value as a top-level statement adds the value to the model. This is restricted, not an arbitrary expression. The forms allowed are:

• g;
• object.g;
• object[expr];
• list[expr];
• (expr); with objects and lists nesting arbitrarily. Notably, functions (whether returning geometry values, objects, or lists) are not allowed at the top level. Allowing them introduces syntax conflicts.

import(expr)

where expr evaluates to a geometry value.

Rendering

A new render(g) function is introduced:

g = {{ ... }};
r = render(g);

For 3D objects, render() returns a polyhedron()-like list of vertexes and faces. For 2D objects, it returns a polygon()-like list of vertexes and paths. In either case, it returns additional metadata.

• echo(render({{ cube(10); }}));

yields (reformatted for readability):

ECHO: {
    min : [0, 0, 0],
    max : [10, 10, 10],
    center : [5, 5, 5],
    size : [10, 10, 10],
    objects : [
        {
            points : [
                [0, 10, 10],
                [10, 10, 10],
                [10, 0, 10],
                [0, 0, 10],
                [0, 0, 0],
                [10, 0, 0],
                [10, 10, 0],
                [0, 10, 0]
            ],
            faces : [
                [0, 1, 2, 3],
                [4, 5, 6, 7],
                [3, 2, 5, 4],
                [2, 1, 6, 5],
                [1, 0, 7, 6],
                [0, 3, 4, 7]
            ]
        }
    ]
}

• echo(render({{ square(10); }}));

yields

ECHO: {
    min : [0, 0, 0],
    max : [10, 10, 0],
    center : [5, 5, 0],
    size : [10, 10, 0],
    objects : [
        {
            points : [
                [0, 0],
                [10, 0],
                [10, 10],
                [0, 10]
            ],
            paths : [
                [0, 1, 2, 3]
            ]
        }
    ]
}

Note that although render() currently always returns a single data-object in the objects[] array, the intent is that in the future, e.g. for multi-material or volumetric color, it might return more than one, and return additional per-object metadata.

String presentation

echo() and str() yield OpenSCAD-syntax representations of the shapes, e.g. (reformatted for readability):

ECHO: {{
    difference() {
        cube(size = [10, 10, 10], center = true);
        sphere($fn = 0, $fa = 12, $fs = 2, r = 7);
    }
}}

Module references

Along the same lines as existing function references, this proposal adds a module-reference data type and a module-literal syntax.

• m = module (args) { ... arbitrary OpenSCAD ... }; which is invoked like an ordinary module:
• m(3);

As for functions, there is a semantic ambiguity: is m a variable containing a module reference, or a module name itself? The resolution is the same as for functions: what is used is the closest-scoped module definition or variable containing a module reference. Variables containing values other than module references are ignored.

Like geometry values above, various forms are allowed:

• m(...);
• object.m(...);
• object[expr](...);
• list[expr](...);
• (expr)(...);

Like geometry values above, functions returning module references are not allowed at the top level:

• f(...)(...);

is not allowed.

It works out that most or all of the need here is met through functions returning geometry values, so it isn't clear whether feature this should be pursued. However, it is a straightforward cross-product of modules and function references and so symmetry supports it.

A note on syntactic ambiguity

The syntactic ambiguities discussed above for module-reference invocations and for adding geometry values to the model are variations of this ambiguity:

• a(b)(c)(d);

Is this:

• module a with argument b, with child module-reference c with argument d, or
• function a with argument b, returning a function, with argument c, returning a module or function reference, with argument d?

Open questions

Terminology

What do we call the "data objects" described above, in an application where "object" is often used to describe a geometric shape?

• "object" for both, disambiguated by context, explicitly "data object" and "geometric object" when required
• "object" for data-objects, "shape" for geometric object
• "dictionary" for data-objects, "object" for geometric object
• ... something else?

Syntactic ambiguity

Ideally, both add-geometry-value-to-model and invoke-module-reference would allow arbitrary expressions that evaluate to the right data type. Unfortunately, existing syntax and the various interactions do not permit that generality. So how do we resolve it?

• As shown here, with restricted expressions, and the note that you can parenthesize an arbitrary expression in either context.
• Require parentheses in one or both of the cases. (I think that resolving the ambiguity requires restricting both.)

object() needed?

Is the object() function desirable, given object comprehension? Is it technically needed? Even if it is not technically needed, is it a useful alternative way to look at the problem?

Should we wish to reconsider the "high level model" closed question below, object() plus list comprehensions might be a plausible replacement for object comprehensions.

Module references needed?

Functions returning geometry values can be used for every(?) case where one might want to use a module reference. Should module references and module literal syntax be included?

How to add a geometric value to the model

The g; syntax is terse and sort of obvious, but brings in syntactic ambiguity and looks a bit weird. The import(g); style is fully general but doesn't look like a first-class citizen. Which to use?

Future-proofing render() value

The object returned by render() is intended to be future-proof; in particular, it is intended to be able to support multi-material and volumetric-color features. Is it future-proof enough?

Closed(?) questions

High-level model

This proposal introduces distinct data-objects and geometric-objects. A previous proposal (see historical notes below) had a single syntax and a single data type, containing both a geometric object and named data elements. Which to use? Although I started on the "hybrid" side, I came down on the "distinct" side for two reasons:

• Straightforwardly allows an object to contain two peer geometric objects.
• Object comprehension syntax didn't have to co-exist with top-level OpenSCAD syntax.

Python-style dictionaries vs JavaScript-style objects

In Python, a dictionary literal is { name_expr: value_expr, ... }. Constant names must (as in any expression) be in quotes. In JavaScript object literal syntax, the name is always a constant, perhaps in quotes; elements with dynamically created names are created using array-style assignments after the object is initially created. Because OpenSCAD data values are immutable, that latter mechanism is not available. The advantage to the Python form is that constructed names are straightforward; the advantage to the JavaScript style is that constant identifier-legal names are visually simple. I think that constant identifier-legal names are a much more common case than dynamically-created or non-identifier-legal names.

One commenter opined that dictionaries and objects were in some way different from one another, but I was never able to understand the distinction.

Futures

• Methods. Invoking a function or module reference that is contained in an object (or list?) might, similar to JavaScript this, set a special variable $this that points at the container.
• Varargs. Have functions or special variables that yield a module or function's argument list as an array, an object, or some combination. (But note that echo(x=1, x=1) is legal but cannot be represented as an object.)
• Spread syntax. Have a syntax that expands an array or object into a list of arguments to a function or module, e.g. f(*positional_arg_list) or m(*named_arg_object) or m(*positional_arg_list, *named_arg_object). Or perhaps JavaScript style ...value.

Historical Notes Follow. Nothing below this is part of the current proposal.

Object construction and top-level structure:

• x = { a = 2; b = 3; };
  • x.a == 2
  • x.b == 3
• x = { cube(10); };
  • Note that this does not add a cube to the model. It constructs an object that represents a cube. That object might or might not later be added to the model (see below).
  • Geometry will be accessible in only two ways:
    • A declarative statement of the object adds the geometry in the object to the model:
      • x; is equivalent to having placed cube(10); at that location.
        • Also a[i];, o.member;, and (expr);, and combinations.
        • But not f(args);; function calls must be embedded in an expression because otherwise f(a)(b) could be either
          • a function call f that takes arguments a, that returns a function reference that takes arguments b, that returns geometry, or
          • a module call f that takes arguments a, with child geometry b.
      • translate([5, 0, 0]) x; operates as expected, being equivalent in this case to translate([5, 0, 0]) cube(10);
    • Data regarding the object's geometry will be accessible with:
      • data = render(x);
      • data = render({ sphere(5); }); would be an inline declaration of an object literal with its data directly accessed.
      • Note that this proposal modifies the proposal of [https://github.com/openscad/openscad/pull/3956 PR#3956], turning the data = render() cube(10); syntax into one in which there is now a function named render which receives objects as values, and returns the data describing their geometry. The returned data will be an object containing bounding box information and mesh information such as points and faces (3D) or paths (2D), with the format of this returned object deferred to the linked PR. The intent of this returned information is that it could be processed during further calculation steps for use of the values, or passed to polyhedron/polygon in identical or modified form.
• x = { a = 2; cube(10); }
  • You can access both:
    • x.a == 2
    • x; places a cube(10); at that location in the model.
    • data = render(x); obtains the rendered bounding box and mesh data.
• x = { function f() ... }
  • To Be Decided (TBD), this either:
    • Is a syntax error. (This is the simplest default for implementation, and could be optionally modified later if a use-case arises.)
    • Declares a private function f accessible from only within the object literal, but which cannot be accessed as x.f because it is not a reference value in the context.
• x = { module m() { ... } }
  • To Be Decided (TBD), this either:
    • Is a syntax error. (This is the simplest default for implementation, and could be optionally modified later if a use-case arises.)
    • Declares a private module m accessible from only within the object literal, but which cannot be accessed as x.m because it is not a reference value in the context.
• x = { a = { b=2; cube(10); }; f = function (x) x+5; }
  • Note that object construction can embed object construction or function references, because they are just expressions.
  • x.a; produces a cube(10);
  • x; produces NO geometry, because it does not contain any. It only contains an object reference which itself could produce geometry.
  • x.a.b == 2
  • echo(x.f(3)); will echo 8.
• function f(x) = { size=x+2; cube(size); }
  • Note that a function can return a constructed object, because it is just an expression.
  • obj = f(10); obj; produces a cube(12); from the obj; statement after the first semicolon, and has obj.size == 12.
  • Note that the internally declared size was accessible within the object in the call to cube, following existing patterns of local value access within a scope.
  • f(10); is a syntax error, or calls another existing module unwisely called f, because this was not a function call context. Note that this is not a permitted method of producing the cube(12); because standard OpenSCAD syntax does not permit calling a function in this manner. This object proposal does not alter this existing behavior.

Dynamic Functionality

• o1 = { a=2; cube(10); }
• o2 = { b=3; sphere(5); }
• ocomb = { each o1; each o2; }
  • ocomb.a == 2
  • ocomb.b == 3
  • ocomb; produces cube(10); sphere(5); as if by implicit union.
  • Note that if o1 and o2 were to have identical variable names, this would warn as if by redefinition of a variable. The solution of ogeometryonly = { o1; o2; } remains available to combine object geometries without their variables, or each can be used on only one to select the variables of only a single component object.
• function CubeLine1(N=10) N <= 0 ? {} : { translate([N*8,0,0]) cube(4); each CubeLine(N-1); }
  • An illustration of a function recursively production an object containing by default 10 cubes spaced along the x direction.
  • Note here that each places CubeLine into a value context, and thus it is a valid function call.
• function CubeLine2(N=10) N <= 0 ? {} : let(rec = CubeLine(N-1)) { cube(4); translate([8,0,0]) rec; }
  • An alternate illustration of a function constructing an object containing the same cubes.
• Because the internals of objects generally follow standard top-level expression syntax, the presence of each to expand an object within an object should apply outside of objects as well. Therefore, as an example:
  • module Foo(obj) { each obj; cube(size); } produces a module Foo which will produce a cube of obj.size if obj has a member size, or warning about no size variable otherwise. Any geometries defined within the passed in obj will also become part of the geometry produced by module Foo.
• Other dynamic construction or dynamic access options involving string-defined keys, construction from lists, or additional support for another comprehension type are deferred to the formation of a proper dictionary type, where this would be more suitable.

Comments

• Note that the only syntax added is the ability to use { … } as an expression, and the expansion of objects with each. The contents of the { … } expression follow standard top-level assignment/module-invocation syntax, and once constructed the value is an object value and can be handled like any other object value.

Related features

These are mostly orthogonal or additional to the features above, but are sometimes tied together in people’s ideas.

Module literals (module references)

• m = module(parameters) {...};
  • m is a reference to that module, can be invoked like so:
  • m(arguments);
• Note that module literals critically differ from object literals in their handling of dynamically scoped special variables, as the object literal is evaluated at the moment of creation and becomes an immutable, while a module literal optionally receives parameters and thus must be newly evaluated each time it is invoked, operating in the dynamic scope at which it is evaluated.
• x = { f = function () …; m = module () …; };
  • x.f is a function.
  • x.m is a module.
  • x.f(...) or x.m(...) would pass $this=x to f or m, so that f or m can refer to the object that it was called on.

Even Older Historical Notes Follow. Nothing below this is part of the current proposal.

CSG tree

IMPORTANT: Following is retained for history only; current thinking is that the CSG tree is opaque and the program cannot inspect it.

• Ignoring how you get to the CSG tree from the created object.

• Note that this is a CSG tree, equivalent to what you see with Design/Display CSG Tree. It is the result of executing statements. It is not executable in any sense. You would not see ‘op=xxx‘ referring to a non-built-in module any more than that could appear in Design/Display CSG Tree.

• { cube(10); }

  • Maybe just a single operation

    {
        op="cube";
        args={ size=[10,10,10]; center=false; };
    }
    

  • or maybe an array that implies union

    [
        {
            op="cube";
            args={ size=[10,10,10]; center=false; };
        }
    ]
    

  • or maybe explicit union

    {
        op="union";
        children=[
            {
                op="cube";
                args={ size=[10,10,10]; center=false; };
            }
        ];
    }
    

  • or maybe op=”group”.

• { cube(10); cube(20); }

  • Maybe an array that implies union

    [
        { op="cube"; args={ size=[10,10,10]; center=false; }; },
        { op="cube"; args={ size=[20,20,20]; center=false; }; }
    ]
    

  • or maybe explicit union (or group):

    {
        op="union";
        children=[
            { op="cube"; args={ size=[10,10,10]; center=false; } ;}
            { op="cube"; args={ size=[20,20,20]; center=false; } ;}
        ];
    }
    

• { translate([1,2,3]) cube(10); }

  • One of the schemes above for representing a single node, and that node is:

    {
        op="translate";
        args={ v=[1,2,3]; };
        children=[
            { op="cube"; args={size=[10,10,10];center=false;};}
        ];
    }