Blog from October, 2007

I replaced my blocking queue sitting between pipeline actors (threaded consumer/producers) and speed of my word freq program dropped from 5.2s to 3.5s on 5M file simply by replacing the blocking queue with my new ping-pong buffered version. That is close to the 3.0s achieved by the nonthreaded version.

// threaded, pipeline version; 3.5s (from 5.2s) on 5M file
f => Words() => { string w | ... }

// nonthreaded, nested map operation on big list; 3.0s
f.lines() => a;
a:{ string line | line.split():{ string w | ... }};

Buffer size is 1000. When I drop to 100, slows down by .2s. When I increase to 4000, no change. 40000, slows down by .5s. Size 400 seems to be slightly faster. Default queue size set to 400 now.

Thanks to Sriram Srinivasan for the idea of ping-ponging the buffers.

So I am narrowing down my understanding of how to use mixins. The Comparable example is awesome, but does not need fields. In my effort to reduce the size of the average mantra object, I needed to remove the in and out fields from every object. The should be moved to an element that all of the actors can include or inherit from. I did not want to force all actors to inherit from the same base class, so I decided that a mixin was perfect but mixins don't currently allow fields. I spent three hours yesterday adding fields only to discover that, while I can get it to work, they are extremely inefficient because I need to use general message sends to call accessor methods. Blech.

Consequently, I am going to literally make the includes keyword include/copy all of the elements from the mixin into the requesting class. The key is to have a single place where the programmer can specify the Actor. Whether I do an actual include or a shared delegate is merely an implementation issue. For speed, I am going to use an include mechanism. I will back out all of my changes from yesterday.

I also noticed that mixins can also act as interfaces. Any class that includes the Actor mixin is also known to answer all those methods and have fields potentially; the fields I will have to access with getters because Java does not allow fields in interfaces. This also gives me a mechanism to describe what an input or output stream looks like. Right now toInputStream(), for example, returns a plain object when I really want to return InputStream. I can make InputStream a mixin with my new mechanism and then I can use the generated interface to make Java to direct method calls instead of mine general message send.

My development plan for the morning is then:

1. make a copy of work yesterday and then back out the changes
2. make the semantic analysis phase decorate the symbol table with tree pointers so I know what code implements that symbol.
3. add fields to mixins in Mantra.g, SemanticAnalysis.g, CodeGen.g
4. In CodeGen.g have it generate code for the included methods and fields; tried to make the templates ignorant of the addition by manually invoking the appropriate tree grammar rules to generate code. The pointers to the appropriate method and field ASTs are obtained from the symbol table modifications in (2).
5. Alter code generation so that mixins results in interfaces rather than static classes.
6. Allow the name of a mixin as a type name; this should already work, but test it

Here is how I want to define an Actor:

package mantra::io;
mixin Actor {
        object in = stdin;   // getters/setters created automatically
        object out = stdout;

        main() { ; }
        object read() { return in.read(); }
        write(object o) { out.write(o); }
}                               

Then we can do:

class Lines includes Actor {...}

Finally got something ready for people to look at. Main components are in.

http://www.linguamantra.org/

Awesome. Mantra does a dynamic mixin. Just map a string to a closure with "self" as first arg:

int.mixin("toHex",
   {int self | return java {new mstring(Integer.toHexString(((mint)self).v))};}
);
println(32.toHex()); // emits 20, the hex verison of 32. :)

also note you could alter the MetaClass per object (set .class field) to alter behavior per instance! You can ask about meta object now:

println(32.class); // emits <int>

To avoid strange bugs, you can only add not override/alter behavior.

Check this out:

ErrorMgr errors = ErrorMgr(); // answers error()
Vehicle.delegate(errors);
Car c = Car(); // car does not answer error()
c.error("we crashed");

here I am calling the delegate method on the Vehicle class, of which Car is a subclass. I make an instance of a car and then send it the error message, which is forwarded to the error manager. Here are the other definitions:

class Vehicle {
    string color;
    string brand;
}
class Car extends Vehicle {
    float mpg;
    int numDoors;
}
class ErrorMgr {
    error(string msg) { println("error: "+msg); }
}

This is a delegation scheme per class not per instance. All instances of Vehicle share the same delegate, but subclasses can override. If there is no delegate for an object's class, the message is forwarded up the class inheritance hierarchy. So, for example, you could add a delegate to object that all instances in your program could access my method.

I'm now working on a dynamic mixin facility, which differs from a dynamic delegation in that the mixin knows which object forwarded the message to it. The 'this' pointer of the delegate must be the usual whereas a mixin is assumed to have a 'this' pointer of the delegator.

By the way, I'm going back to calling mantra's mixins "mixins". Use "include" to bring in methods:

class int includes Comparable {...} // statically mixin Comparable's  methods
mixin Comparable {...}

Now I just have to update the doc to include all of the changes I've just made. I even added a meta object protocol, the natural place to store delegates per class. Getting verrrry close now.

See http://www.linguamantra.org for more information.

Type annotations

Mantra is not a statically typed language like Java, though you'll see types specified in the code. These types are annotations kind of like "executable documentation". For example how many times have you done this in other dynamically typed languages like Ruby and Python:

f(a): ... # a is an int and f returns a list with blah blah

Mantra integrates such informal type information into the code like a statically typed language, but that information does not affect the validity of the statements in Mantra methods in any way. Mantra uses dynamic typing. "string s = 34;" compiles fine in mantra. The type information can be used to generate pre-and post-conditions that validate incoming and outgoing objects, relieving the programmer of that burden. The type information can also be used for optimization.

Delegates

This feature is essentially the same as a Ruby mixin ('cept no fields)

Delegates behave like abstract superclasses except that a class may delegate to more than one delegate (very much like multiple inheritance). Recall that abstract superclasses add functionality to subclasses and may refer to methods that don't have implementations in the abstract superclass. A subclass "includes" all of the methods and fields from the superclass except for those methods overridden in the subclass. A delegate differs only in that you cannot define fields and you do not have to provide abstract method declarations for method you call. References to methods become self.methodName() rather than the usual implied this.methodName().

Philosophically, delegates differ from abstract base classes in an important way: delegates are just code they do not talk about identity whereas the superclass/subclass relationship is normally one of identity ("is a" relationship). For example, a manager "is an" employee and a house cat "is a" feline (well, some people think they are human). In contrast, an input stream is not a kind of StreamSupport object, but it does want to reuse that code. Delegates in mantra provides the ability to share code without polluting the organization of the class hierarchy.

My desire for something like delegates came up when I realized that the exact same code for stream comparison should be included in all input streams. How do you do that without sharing the base class or duplicating code? One way is to can create a separate class containing the comparison code (call it a delegate). This code can be shared by multiple classes (the delegators) as long as they have pointers to an instance of the support (delegate) class. The support code reaches methods in the delegator through the use of a "back pointer". The following code illustrates the necessary infrastructure:

class Delegator {
    Delegate d = new Delegate(this);
    void foo() { bar(); }
    void bar() { d.bar(); } // delegator to other class
}

class Delegate {
    Delegator self;
    public Delegate(Delegator self) { this.self = self; }
    void bar() { System.out.println("bar()"); }
    void duh() { self.bar(); } // invokes bar from Delegator
}

All of that infrastructure is a hassle to do manually, but it works efficiently and clearly. Mantra delegates does all of this automatically.