Monthly Archives: December 2007

Constructors #2

In my last post I wrote about the problems with constructors. In this post I’ll describe the approach I’m currently considering using for my hobby language. But first I have to set things up a bit and describe some related language constructs.

Protocols

First of all, there are no classes but only protocols. You can think of protocols as classes without fields or, equivalently, as interfaces where methods can have implementations or, most accurately, as something very similar to traits. Here is an example:

protocol Point2D {
def x();
def y();
def to_string() {
return "a Point (x: ${this.x()}, y: ${this.y()})";
}
}

In a class-based language Point2D would have had two instance variables, x and y. Since protocols don’t have fields this one has two virtual methods instead where the fields should be, x() and y(). It also has a to_string method which uses string interpolation, which I’ve stolen directly from the neptune language.

One way to create an instance of Point2D is to create a subprotocol that defines the methods x and y:

protocol Point2DAt3Comma4 : Point2D {
def x() { return 3; }
def y() { return 4; }
}

def p := new Point2DAt3Comma4()1();
print(p.to_string()); // -> a Point(x: 3, y: 4)

Even though protocols are like interfaces there is nothing to stop you from creating instances of them. And because they don’t have any fields, initialization is trivial.

Records

However, it obviously doesn’t work that you have to write a named subprotocol whenever you want a concrete point. Instead, there is a dual to protocols: records. Where a protocol is pure behavior and no state, a record is pure state and no behavior. In the case of Point2D the associated record could be

def p := new { x: 3, y: 4 };
print(p.x()); // -> 3
print(p.y()); // -> 4

All a record can do is return the field value when the associated method is called. As with protocols, initializing a record cannot fail because it is only created after all the field values have been evaluated.

So now that we have behavior without state and state without behavior we just have to combine them. This is done using the “extended” method call syntax:

def p := new Point2D() { x: 3, y: 4 }
print(p.to_string()); // -> a Point(x: 3, y: 4)

This syntax creates a record like the previous example, but make the record an instance of Point2D. Initialization is trivial, as with plain records, because all field values have been evaluated before the instance is created. This works in the simple case but unfortunately doesn’t solve the general problem. First of all, we’ve exposed the “fields” of Point2D. If we later want to change to a polar representation, every occurrence of the above syntax that binds x and y directly will give us broken points. Also, this does not take inheritance into account.

For this reason, the syntax above is not intended to be used in client code, only in Point2D‘s methods. Instead, Point2D should define factory methods for client code to call:

protocol Point2D {
...
static factory def new(x: i, y: j) {
return new this() { x: i, y: j };
}
}

The static modifier means that the method is the equivalent of a smalltalk class method. x: and y: are python-style keywords with a more smalltalk-like syntax. The factory keyword does not have any effect until we get to subprotocols. In any case, with this definition you can now create a new point by writing

def p := new Point(x: 3, y: 4);

If, at some later point, you decide to switch to a polar representation you simply have to change the definition of new(x:, y:):

static factory def new(x: i, y: j) {
def t := arc_cos(i);
return new this() { rho: j / sin(t), theta: t };
}

As long as people use the factory methods you are free to change the implementation of the protocol. Also, the protocol can do arbitrary pre- and postprocessing and the actual instantiation and initialization is atomic. All that’s left now is to account for subprotocols. Enter Point3D

Inheritance

Here is Point3D:

protocol Point3D : Point2D {
def z();
override def to_string() {
return "a Point (x: ${this.x()}, y: ${this.y()}, z: ${this.z()})";
}
}

This protocol has three “fields”, x, y and z that have to be initialized. One way to do it would be for Point3D to initialize all of them:

protocol Point3D : Point2D {
...
static factory def new(x: i, y: j, z: k) {
return new this() { x: i, y: j, z: k };
}
}

This sort of works but it is error-prone, breaks encapsulation, and breaks down if we were to change Point2D to use polar coordinates. What we really want is for Point3D to call the new(x:, y:) method on Point2D. That’s where the factory keyword from before comes in. A factory method is one that allows you to specify a set of fields. Because new(x:, y:) is a factory method the instance created by this call

new this() { x: i, y: j }

will not only have an x and y field but also any fields specified by the caller of new(x:, y:). This allows us to rewrite new(x:, y:, z:) as

static factory def new(x: i, y: j, z: k) {
return new super(x: i, y: j) { z: k };
}

If Point2D were to change its representation to polar coordinates then Point3D would still work, provided that Point2D had x and y methods. The result would just have a rho, theta and z field.

Let’s look at another example: the one from the previous post where points have a serial number.

protocol Point2D {
def x();
def y();
def id();
static factory def new(x: i, y: j) {
def s := current_id++;
return new this() { x: i, y: j, id: s };
}
}

protocol Point3D : Point2D {
def z();
static factory def new(x: i, y: j, z: k) {
return new super(x: i, y: j) { z: k };
}
}

Even though this requires nontrivial computation in a superprotocol there is no risk of creating improperly initialized objects. In terms of the construction steps described in the last post this allows initialization to proceed like this:

  • Arbitrary preprocessing in Point3D
  • Arbitrary preprocessing in Point2D
  • Object instantiation and full initialization
  • Arbitrary postprocessing in Point2D
  • Arbitrary postprocessing in Point3D

The fact that protocols don’t declare fields but that fields are provided upon construction also allows new kinds of specialization. For instance, we don’t actually have to change the implementation of Point2D to create an instance that uses polar coordinates:

protocol Polar2D : Point2D {
def rho();
def theta();
def x() { return this.rho() * cos(this.theta()); }
def x() { return this.rho() * sin(this.theta()); }
static factory def new(rho: r, theta: t) {
return new super() { rho: r, theta: t };
}
}

In this case we choose to bypass the new(x:, y:) method and instead provide a method implementation of those fields instead of instance variables. This way we can share all methods in Point2D. This is also possible in other languages if all access to x and y takes place through accessors, but there you would get two unused fields in all instances.

Finally, let me just mention that any method, not just new, can be a factory; the syntax is just slightly different when you invoke it; for instance

def q := Point2D.at(x: 3, y: 4) { z: 5 };

It looks slightly different but the mechanism is exactly the same.


1 The meaning of new syntax, as in neptune, is not a special construct but an ordinary method call. The syntax new X() is exactly equivalent to the ordinary method call X.new().

Newspeak Constructors

There is one language design problem I’ve never been able to solve to my own satisfaction, and which no language I know solves completely: constructing objects. Gilad brought the subject up in two blog posts a while back but while the newspeak solution he describes has some nice properties it just doesn’t feel quite… right. Now, I’m currently (as always) working on a hobby language and had to find some, any, solution to this problem. While working on that I came up with a slightly weird solution that has some nice properties. But before going into that this warm-up post takes a look at the newspeak solution. Note: because I know little of newspeak I may be making some wrong assumptions about how the language works; apologies in advance if I get things wrong.

Update: Peter has written an update that demonstrates that I did indeed get things wrong, newspeak constructors are not as restricted as I assume in this post. So you may want to just skip over the newspeak part.

I’ll warm up by rehashing a well-known problem with the most widely used constructor mechanisms: they are just not object oriented. In Java, if you write

new C(...)

you’ve tied your code directly to the class C because the constructor is forced to return a new instance of C. Not a subclass, not a cached instance, not anything else. There is no abstraction there, no wiggle room.

One way to make things slightly better is to use a static factory method instead, that frees you from some of the restrictions of constructors so you’re no longer tied to a particular implementation, but it still ties you to a particular factory. You can then take a step further and implement factory objects that can be passed around as arguments and changed dynamically but that gives you a lot of boilerplate code. A much more lightweight solution in languages like SmallTalk that support it, is to use class methods on first-order class object — it’s really just factory objects but the language happens to give them to you for free. This is a pretty decent solution but has the disadvantage that it requires first-order class objects. A class object is a reflective, meta-level, thing which people shouldn’t have to use in programs that are otherwise non-reflective. But that is a small detail and probably not a problem in practice.

However, the issue of how best to invoke constructors is only part of the problem with object creation and not the hard part. The hard part is: how do you initialize instances once the constructor has been called; how do you set the values of an object’s instance variables.

What makes this especially hard is the fact that instance variables are essentially non-object-oriented. There are ways of making them less bad, like hiding them and providing accessor methods, as in Self and (apparently) newspeak. That gives you some wiggle room, for instance by allowing you to change something that was an instance variable into a computed value or overriding the accessor, without breaking client code. However, even with Self-style accessors, we’re still not off the hook because instance variables have to be initialized. This is where, in my opinion, the newspeak model breaks down.

Consider this newspeak example:

class Point2D x: i y: j = ( |
public x ::= i.
public y ::= j.
| ) (
public printString = (
ˆ ’ x = ’, x printString, ’ y = ’, y printString
)
)

There is a 1:1 correspondence between arguments to the primary constructor and instance variables in the object. And it must be so, except for trivial variations like reorderings, since any nontrivial initialization of instance variables would introduce the possibility of exceptions or non-local returns and hence void the guarantee of correct initialization[1].

Imagine that we want to give each point a unique integer id. In Java you could easily implement it something like this (ignoring synchronization issues):

class Point2D {
static int currentId = 0;
int x, y, id;
Point2D(int x, int y) {
this.x = x;
this.y = y;
this.id = currentId++;
}
}

In newspeak you can use a secondary constructor to implement this but as soon as you try to subclass Point2D you’re in trouble: the subclass has to call the primary constructor, which is only allowed to do trivial computation before setting the instance variables. So either the id generation code has to be duplicated in a secondary constructor for the subclass, or the id slot has to stay uninitialized and then be initialized in a second pass. The first option is obviously bad and the second pass erodes the guarantee of correct initialization. At least, that’s how I understand the mechanism.

The root of the problem, and this also applies to Java constructors, is that steps of object construction that logically belong together are executed in the wrong order. A simplified model of how the construction process takes place is that there are three steps:

  1. Preprocess: calculate the values to be stored in the object, for instance the point’s id
  2. Initialize: store the calculated values in instance variables. Only trivial field stores take place here, no general computation
  3. Postprocess: possibly additional work after the object is fully initialized.

If we consider the simple Point3D inheritance hierarchy where

Object <: Point2D <: Point3D

the newspeak model allows the following sequence of steps:

  • Arbitrary preprocessing in secondary constructor in Point3D
  • Instance creation by primary constructor of Point3D, recursively through primary constructor of Point2D
  • Initialization of Point2D slots
  • Initialization of Point3D slots
  • Arbitrary preprocessing in secondary Point3D constructor

Since no arbitrary computation is allowed from the object has been created to the whole chain has had a chance to initialize their variables the object is guaranteed to have been initialized. However, only the bottom-most object on the inheritance chain is allowed to perform pre- and postprocessing.

In Java the model is different: there, each constructor through the chain is allowed to do arbitrary computations, as long as it just starts off calling its superconstructor. Expressed in the simple steps from before, Java allows this sequence:

  • Immediate instance creation
  • Arbitrary preprocessing in Point2D
  • Initialization of Point2D
  • Arbitrary postprocessing in Point2D
  • Arbitrary preprocessing in Point3D
  • Initialization of Point3D
  • Arbitrary postprocessing in Point3D

Object initialization and arbitrary computations are interspersed which gives plenty of opportunity to leave the object half-initialized if an exception occurs and for superclasses to accidentally call methods overridden by subclasses before the subclass has had a chance to properly initialize itself.

The ideal model would be one that looked something like this sequence of steps:

  • Arbitrary preprocessing in Point3D
  • Arbitrary preprocessing in Point2D
  • Object instantiation
  • Initialization of Point2D
  • Initialization of Point3D
  • Arbitrary postprocessing in Point2D
  • Arbitrary postprocessing in Point3D

Here, all classes are allowed to do arbitrary pre- and postprocessing but the actual instantiation and initialization is atomic so the object is guaranteed to be well-formed. Finally, once the object has been initialized, each inheritance level can do postprocessing with a fully initialized object.

My next post will explain how I think that can be made to work.


[1]I assume that newspeak constructors guarantee that objects are always correctly initialized because of this paragraph in Gilad's post: "One detail that’s new here is the superclass clause: Point3D inherits from Point2D, and calls Point2D’s primary constructor. This is a requirement, enforced dynamically at instance creation time. It helps ensure that an object is always completely initialized."