SIGPL~N Notices 50 1969 Auga~t

ALTERNATI~ TO EXTENSIBLE ~ N G U A G E $
Ms Do Mcl!rey~ Bell Telephone Labsratories~ Murray Hill, New Jersey° 1969 ~ y 13o
One of the most important things~ it seems to me~ in extending a languages is some-
how to have the extension smooth with respect to the base° Smoothness is an aesthetic
requiroment~ and I can't be too precise about it~ but it is also a very strong require°°
ment~ and one that is very hard to live up to when you actually come down to try to
write an extonsiono For example~ in extending a language by introducing a new data
type~ one should allow that now data type to appear in all contexts of the language or
as many as possibleo Not only in expressions~ which we seem to know how to do, but
also in places like input/outputo If the language has parametric data types (for ex~
ample~ declarable precisions) and has corresponding poly~orphic operators defined over
all of them~ then extension should likewise have comparable capabilities for para~et~
erization and polymorphismo This is not possible in Algol 68~ for instance° (I will
make frequent reference to Algol 68 in this talk for the reason that Algol 68 is de~
fined and available~ has been thought about very carefully~ and therefore, is much
easier to shoot at since it is a reasonably stable target°)
Smooth extensibility can depend on amazingly innocuous features of the base lan~
guageo Again consider Algol 68 as an example~ where one has ~ ~ ~ ,
l ~ ~ ~ , ~ ~ ~ and so one Suppose I decided that what I
wanted long integer to be was arbitrary precision integers which I would implement in
the customary way of having some dynamic storage allocation package that allocates
pieces adequate to each integer computedo And then I would try to put into the lane
guage an onviro~ent inquiry (because Algol 68 believes in environment inquiries)
~at~s the biggest ~ ~ ? ~ In answering that inquiry it would use up all the
storagoo And I can't do much more computing°
Until quite recently~ most forays into the exciting language8 have had fuzzy and
over-ambitious goalso It is easy to set down a collection of representative things
one would like to do in language extension and a very short list can set up an un~
reachable goalo We all know more challenges than we can implo~ento
I have personal familiarity with only a few actually implemented idea~ beyond
straight ~aero processors~ and most of these have succeeded by virtue of limited goals°
For instanco~ Dick Bennett's Sot processors He observed that ~aero~ work~ and that
the syntax of function calls was handy~ by restricting himself to functional syntax
and nothing ~ore~ he was able to ~trip away ~o~e of the p r o b l e ~ a n d actually got ~o~e~
where with the problem of extengion of se~antic~ if not syntax~ There was the simple
idea of type otho_r in various Fortran compilers by CDC~ where a handful of data types
could be provided by user~definod routines with know~ na~es~ Handy and woll~usedo
Michigan's Mad translator had a neat way (for the initiate) of introducing new oper~
ators~ and very powerful things have been done with this~ but they depended in detail
on the exact internal representation of tables within one particular compil~ro Mere
recently~ there has been ML/1 by Peter Browne I ~ not sure it is correctly referred
to in an extensible languages symposium because it has no base language whatsoever°
But precisely because it has no base language and is a string~to~string translator~
it doos~ in fact~ give remarkable power to the virtuoso user willing to start at ground
zeroo And finally~ there are the compile ti~e facilities of PL/I~ and these are very
significant because indeed this is the first time any sort of macro facility has
really made it into the public domain as part of a higher level language°
Based on observation of those few solid successes, a~id a torrent of hopeful prom
posals~ l~m ready to say that we can't expect to extend any languages in a clean way
all the way across the board except perhaps for some languages as trivial as Set w a ~
which had no syntax worth mentioning and no characteristic programming style°
SIGPLAN Notices 51 1969 August

As most of the speakers have emphasized~ extremely smooth underlying structure

is necessary for a language to be smoothly extensibleo For example~ it wasn't too
many years ago that the innovation came into currency (and I don't know to whom it is
due) that statement and expression in customary programming languages should be den
fined by mutual recursiono As this is a ~ u a non for syntsxtic extension, no pro~
posed base language today lacks !to How many more of these fundamental insights are
we missing? I couldn't agree more with Tom Cheatham that the effort in extensible
languages i~ going to lay bare the fundamentals of programming ands therefore, from
an academic standpoints the effort is highly justified°
As has been mentioned~ particularly by Don MacLaren, the pure pre~proeessing ap-
proach to extension gives rise to several lexical problems: name conflicts~ departures
from scope rules~ and what have you° And these problems are magnified when one is try~
ing to extend in s global, environment° Modern computing is done in a computers and by
~computer~ ~ I mean the computer plus all its files plus all its users~ That wonderful
production, number one in Algol syntax, or ~ ~ is not properly part of modern
eomputing~ An Algol program~ and in fact a program in almost any other programming
language~ is a universe unto itself~ out of touch with the genuine universe of comput=
ing~ naaely the computer in the extended sense° Working in such a wide environment
with the desire to have access to all parts of it~ but at the same time not knowing
what is there~ we have some severe problems, among which are lexical problems of scope
and name conflict°
From languages defined from deep pyramids of extensions one doesn't expect to ob~
tain decent code° I think that extended languages will~ for some time, be useful only
for experimental, one-shot~ or otherwise small applications° Attempts to describe
good compiling algorithms in machine-independent terms have been overwhelmingly op-
aqueo Witness the monumental paper of Galler and Perliso I do knew a few macro packs
ages which produce very high-class object code for quite sophisticated languages~ but
I don~t recommend a single one of these programs as literature to be studied. The
elevernesses for getting good object code are unfortunately still clevernesses and
promise to remain so for some time°
Another difficulty is that, to dater most of the implementation-lndependent
schemes for extension that have been reduced to practice yield appallingiy slow compil-
ers° I think that experts using modern but standard compiler writing techniques are
now able to modify compilers with considerable dispatch for small perturbations on the
source language comparable to the kinds of perturbations which are conveniently put in
by extension° The modified compiler will be a much better running thing if you intend
to use it often°
I can't agree with Jim Bell that transforms improve one's ability to test new id-
eas with turnaround more rapid than other methods afford. A compiler you can rewrite
and rebuild in a minute or two, and these are not uncommon, is an equally good test
bed.
I have seen, as I have said, many respectable languages defined as macro packages
on top of assemblers. Unfortunately, they all turn out to be highly personal things°
For one things diagnostics turn out usually at some very low level and none at the ex-
tended language level. Intelligible to the author, yes~ but not intelligible to any
other user of the extended language. This isn°t an argument against extensible lan-
guages~, but it does show that sxtenslble langua-es aren't apt to be a great
boon to folks with applications, who don~t happen to have a secret yen also to be
language builders.
With Cheatham~ I think that hidden meaning is unfortunate° I also think, with
MacLaren, that the extended language is much more likely to be full of the hidden
meanings than would be one general, widely known, deeply studied~ PL/I-ish~ if you
will~ sort of thing.
SIGPLAN Notice8 52 1969 August

Extensible languages ought to be useful to experimenters~ but unfortunately they

have one severe drawback that other people have mentioned before° All possible exten-
sions~ at least in our present disciplines~ were in a very real way conceived in the
mind of the designer of the base language and extension system° From the language
point of view~ only trivial experiments can be performed within an extensible language°
We in no way understand how to make radical extensions available without recourse to
work in an on machine language or in and on operating systems° I challenge you~ for
examples to show one proposed language facility which applied to PL/I would permit
the introduction of a new storage class~ or of co~routines~ or of some slightly per~
turbed operating environments I think here I have to call Tom Cheatham guilty of
over~selling~ a la the college grantsman~ when he advertises such things to be the
needs~ absent in PL/I~ that might be supplied by extensions°
That finishes my small list of things that are going to be hard to doo We are
not going to kill off PL/I~ Algol 68 and other things like them by a great product
from the academic drawing boards in the next few years° I also think~ thought we
ought to keep working on~ because~ indeed~ it is a lot more fun and it will help lay
bare foundations of programmingo
DISCUSSION
Irons: I don~t want to put myself in a defensive 'posture too much~ but I think
it is appropriate to make some remarks about efficiency because I have had some exper~
lense and have done a little measuring~ The compiler system we wrote I think was feb
stricted in a certain sense to be able to achieve these obJectives~ but I think they
are worth mentioning anyway° Compile time in our systems which includes a parse of
our context-free languag.~ is about ten times Control Data~s fast Fortran compiler on
our CDC 6600° I don~t consider this prohibitively inefficient~ at the same time the
generated code is at least good enough to produce operating systems and compilers°
On top of that~ the parsing process~ which is the part which really makes the language
extensible~ accounts for only about twenty percent of the time of compilation anyway°
So extensiblilty does note by any means~ automatically mean impossible efficiency° I
think~ actually~ things are likely to improve considerably as time goes by~
~ ~ To what depth of definition is your compiler running when it is running
at this efficiency?
Iron~: The whole compiler uses the extension mechanism down to the most rudiment~
ary representation of the machine language~ so complete depth is the answero