TRIBUTE TO ROGER HUI

on receiving the 1996 Kenneth E. Iverson Award

The following text was written by Donald B. McIntyre at the invitation of the SIGAPL Board. The text was published by ACM SIGAPL in APL Quote Quad, Vol.27, Number 1, September 1996, p.24-29. It is re-published with permission. For an explanation of the Award see editorial notes on p.3 & 24.

I am delighted that Roger Hui is the recipient of the 1996 Kenneth E. Iverson Award. No award would be more appropriate or give greater pleasure. Roger’s story is the account of a crucial period in the evolution of Ken’s language, APL.

As banquet speaker at APL83, I sat next Adin Falkoff when he was the first to receive the prestigious Award now named for his long-time collaborator. At APL94 I was sitting beside Roger. I thought he, more than anyone else, deserved the Award that night, and nearly fell off the chair in surprise and embarrassment when I received it myself. Roger was the first to shake my hand, and it is a very special privilege to pay him tribute now.

We first met at APL89. Ken knew I was struggling with a permutation problem for my talk, and took me to Roger at the Sharp booth. Roger amazed me with a new function that solved my problem elegantly. I know now I had been introduced to an early version of what became J.

Roger was born in Hong Kong in 1953, and immigrated to Canada with his entire family in 1966. They travelled by APL (American President Lines) and settled in Edmonton, the city closest to the farm where Ken spent his youth.

In 1973 Roger entered the University of Alberta. Those who knew him then still speak of his extraordinary talents. In his second year he took a course on Probability and Statistics in which students were expected to learn APL with little or no formal instruction. He used all the time he could muster on a heavily-burdened computer, and benefited from the APL\360 User’s Manual (APL Language was not published until March 1975). Because the manual was written by Falkoff and Iverson, Roger thinks it reasonable to say he learned APL from Adin and Ken.

As a summer student in 1975 and 1976, Roger worked at I.P. Sharp Associates (IPSA) in Calgary, on workspaces for statistical and probability calculations. The major attraction of the job was the unlimited computer time with access to APL. Arthur Whitney (a year ahead, but on one occasion a classmate) had preceded him at IPSA in Calgary in 1974. Not surprisingly they have continued to keep in touch, both contributing to the development of J, as my story will tell.

After receiving a B.Sc. degree with First Class Honors in Computing Science in 1977, Roger worked for two years as a full-time programmer/analyst in the new Edmonton office of IPSA, where his principal duty was to support clients in their use of APL time-sharing. He attended APL79 at Rochester, where Ken gave two papers: The Role of Operators in APL and The Derivative Operator. On the way, Roger stopped at IPSA in Toronto and obtained a copy of Operators and Functions [IBM Research Report No. 7091, 1978]. He has been studying that paper and its successors ever since.

In September 1979 Roger entered the Department of Computer Science at the University of Toronto, and received his M.Sc. in May 1981 with a thesis on The complexity of some decompositions in matrix algebra. Although the thesis was not directly APL related, through this experience Roger developed a feel for efficient computations, which helps explain the speed of J’s primitive functions.

After completing his Master’s degree, Roger worked from 1981 to 1985 as an APL systems analyst and programmer for the Alberta Energy Company in Edmonton. In February 1982 Roger purchased A Source Book in APL (1981), in which the most memorable papers were The Design of APL (1973), The Evolution of APL (1978), and Notation as a Tool of Thought (1980). By this time Roger was reading every paper of Ken’s that he could get his hands on.

Roger’s work was described at APL85 in a paper, DESIGN: A Financial Modelling System, written jointly with his Supervisor, Fred Appleyard. The basic objects in the system were in Direct Definition (Iverson, 1976, 1980), and Falkoff and Iverson’s The Design of APL was cited. Roger left Alberta Energy shortly after being promoted to a non-APL and non-programming position, and was out of work, and had no access to computers, from September 1985 to April 1986. This gave him plenty of time for intense study of Ken's Rationalized APL (1983) and A Dictionary of the APL Language (as it was called at the time).

Ken, meantime, had moved to IPSA in Toronto (January 1980), where he was actively reviewing APL syntax and exploring possible extensions to the language. The titles of Ken’s publications show the thrust of his thinking: between 1976 and 1983 he published on Combinatoric Operators, The Dot Operator, Operators and Functions, Operators and Enclosed Arrays, Rationalized APL, and APL Syntax and Semantics. Roger was well prepared to appreciate the significance of Ken's ideas, and eagerly devoured these seminal papers.

In the fall of 1976 Eugene McDonnell began a series of articles on Recreational APL in Quote Quad. His puzzles drew marvelous solutions from some of the giants of the APL community. Roger’s first contribution received special mention in Quote Quad, 1978, Vol.8, Number 4 (June 1978), p.40; and his solution to The N-Queens Problem, deserved an article on its own in APL Quote-Quad, Volume 11, Number 3 (March 1981), p.26-28. One of the most popular puzzles was the Pyramigram (suggested by Linda Alvord). Commenting on the responses received, Eugene wrote in APL Quote-Quad, Volume 11, Number 4 (June 1981), p.26-28: "Of the non-looping solutions, most were fairly straightforward attacks on the problem, sometimes combined with a reasonable amount of ingenuity, and most were a pleasure to read. There was one, however, to which I must award the prize, for the precise, spare, and direct means used to achieve its ends." Even should your memory back to 1981 fail, you no doubt guess from Eugene’s words that it was Roger Hui who won the prize! Eugene’s commentary continues: "I’m going to try to break [Roger’s solution] down for you piece by piece, going backwards from his solution to the way he may have arrived at it". The analysis takes a page and a half, in the middle of which Eugene exclaimed "Here is where Hui’s ingenuity rose to the level of the master stroke" – Roger’s key insight was using minus-scan to produce the left argument of replicate.

Roger met Ken for the first time, in Toronto, in late 1980 or early 1981; they met at Ken's residence, and a group of IPSA people went out to dinner afterwards. This crucial introduction was made by Eugene McDonnell, whose own contribution to APL and J are, as is well known, many and important. Because Eugene was an editor of Quote Quad (then a quarterly), it is now uncertain which of Roger’s delightful contributions to Recreational APL Eugene had seen before he introduced Roger to Ken.

In 1981-1982 Arthur Whitney was at IPSA in Toronto, working closely with Ken. He built an interpreter in APL, implementing and clarifying many of the ideas in Operators and Functions (1978); together with Ken he introduced several new ideas. Ken and Arthur reported at APL82 in their paper Practical Uses of a Model of APL. [1996 8 7]

In May 1986 Roger joined IPSA in Toronto, where he worked on various projects and products, a system monitor, a stock borrowing/lending system, a bond trader’s workstation (which he demonstrated in the booth at APL89), etc. More importantly, Roger was re-introduced to Ken. Although working together was not among Roger’s duties, they did so extensively. Ken's ideas up to that time culminated in A Dictionary of APL, published by IPSA in July 1986 and by APL Quote Quad in September 1987. The Dictionary went through many drafts and precursors, all closely studied by Roger, whose name appears in the acknowledgments. Roger tells me he still has five drafts, the earliest dated December 1984. Although the dictionary was an internal publication, IPSA was open about distributing it; at one of the Minnowbrook conferences each participant was given a copy. [1996 8 7]

At APL87, in Dallas, Ken spoke on the line of development he had begun in 1978. He had given a preview of that paper (APL87) at the internal IBM conference commemorating the 20th anniversary of the initiation of IBM’s APL timesharing service. Ken retired from IPSA in 1987, soon after the APL87 Conference.

At the Dallas meeting, Roger presented a companion paper (Some Uses of { and } ) in which he exploited the facilities of Ken's Dictionary and modelled extensive parts of the language now known as J. Roger’s principal acknowledgment was to Ken.

Roger had benefited from experience in using SAX, Sharp APL Unix, which became available in 1987. It has only APL characters and executes existing Sharp APL programs. Although based on A Dictionary of APL, not many new dictionary APL ideas were implemented. SAX has compose and atop and under, as indeed does Sharp APL/MVS, but unlike the MVS product SAX permitted non-primitive functions as arguments to operators. As a result, the possibilities for tacit definition were first glimpsed here; one of the missing steps was function assignment. [1996 8 6]

Roger tells the next part of the story in An Implementation of J (1992): "One summer weekend in 1989, Arthur Whitney visited Ken Iverson at Kiln farm and produced – on one page and in one afternoon – an interpreter fragment on the AT&T 3B1 computer. I studied this interpreter for about a week", Roger writes, "for its organization and programming style; and on Sunday, August 27, 1989, at about four o'clock in the afternoon, wrote the first line of code that became the implementation described in this book."

In an e-mail Roger wrote: "By this time Arthur was working for Morgan Stanley and had already produced his A interpreter. The fateful weekend when Arthur visited Ken, it was Arthur's suggestion that I was invited out to the farm on the first day; I was not present the following day when Arthur wrote his one-page interpreter." [1996 8 6]

In an earlier e-mail Roger wrote: "Although Arthur's one-page interpreter was the final impetus that led to J, the language owes much more to A Dictionary of APL, both in the language design and in the system architecture. I studied the one-page for about a week for its organization and programming style, but I'd studied the dictionary through its various drafts and precursors for about ten years, and had modelled extensive parts of the language. (see Some Uses of { and } ). ... Ken's paper A Personal View of APL in the IBM Systems Journal, 1991 12, also recounts events from that period. In the paper, Ken said he “showed this fragment [Arthur's one page] to others” and “soon recruited Roger Hui”. Roger’s comment was “Funny, I always thought I recruited him. With Ken, the surprise was total: he showed me the one-page, and I came back 2-3 weeks later with an interpreter which had the parser implemented, and was definitely recognizably APL.” [1995 2 4]

"In particular, the J interpreter was designed and built around the parse table in Table 2 of the dictionary ([1987] p. 38). I'd previously modelled the parser at least two different ways, and when I sat down to write J , I set an objective for myself that I can show the parser in the C source to Ken and have him verify its correctness. (Ken is not a C programmer.) I don't recall that Ken ever did that (he seems to just trust me to do it correctly ...), but the effects of trying to meet the objective are entirely beneficial." [1995 2 4]

"Looking at the parser, more precisely, the parse table, it is clear that any syntactic form can be given meaning, at least any form that involves looking at no more than 4 tokens. If the rules pertaining to fork and hook are removed, one'd get rationalized APL\360 (i.e. assuming removal of the anomalous [ ; ]). Fork and hook are meanings assigned to constructs that would be errors in APL, and do not affect existing interpreted expressions; therefore, fork and hook are compatible extensions and are (I believe) permitted by the APL standard." [1995 2 4] Roger has never doubted that J is a dialect of APL: "It is the latest in a line of development known as dictionary APL. The spelling scheme uses the ASCII alphabet. The underlying concepts, such as arrays, verbs, adverbs, and rank, are extensions and generalizations of ideas in APL\360. Anomalies have been removed. The result is at once simpler and more powerful than previous dialects." [1995 2 4]

"When Ken and I started work on J", Roger wrote later, "we had in mind to implement the 1987 Dictionary of APL plus the hook and fork of the APL89 paper Phrasal Forms by Ken and Eugene [K.E. Iverson and E.E. McDonnell: APL Quote Quad 19, No.4, 1989]. There was no question in our minds that we were working on APL, if the Dictionary of APL is APL. The development can be traced from Operators and Functions [1978], through all the APL conference papers written or co-written by Ken to 1987, culminating in A Dictionary of APL in September 1987. (Rationalized APL, an IPSA research report [1983], should also be included as part of this development.)" [1996 8 6]

I asked Roger if he remembered when he and Ken first discussed the changes in APL that led to J. My question was not well phrased, but Roger’s answer was "No, and I doubt that Ken can remember either, because we never thought in terms of replacing APL or leading to J. The name J was an accident." [1996 8 7]

"For a few months, the interpreter had APL characters as well as the . and : characters; I finally removed the APL characters at Ken's suggestion (or was it with Ken's agreement?). It was Ken's idea to use only characters from the 7-bit ASCII alphabet and to use the . and : spelling. Explicit definition (the : operator) was not available for quite some time due to implementation reasons, so we were forced to explore the possibilities of tacit definition (then known as operator expressions); since we had both fork and function assignment, we had everything we needed, although we didn't realize it at the time. (Fork and function assignment turned out to be very easy to implement, much easier than explicit definition.)" [1996 8 6]

The new dialect of APL was called "J" by Roger (Ken Iverson, A Personal View of APL, IBM Systems Journal vol. 30, No. 4, 1991, p.583). Roger tells us that "J" was christened the first time it became necessary to save the source file [1995 2 4]. We also have Roger’s explanation for the choice of name: "Why “J’? It is easy to type." (Implementation, 1992, Preface). The roots of J clearly go back to Iverson notation (1960, 1962), but it is no more than a happy coincidence that J follows I in our alphabet.

"It was a very exciting time," Roger records, "the ideas and implementation were coming fast and furious, and we did not stop to record who had what idea or who did what when. ... I did the work after hours and on weekends, and was spending many evenings at Ken's residence at Erskine Avenue. Before I went on vacation for two weeks in the Far East in October 1989, Ken told me that he too was going on vacation at the same time. I remarked on what a coincidence it was, and Ken corrected me by saying that he'd adjusted his vacation schedule to mine. Whence Jean Iverson took a long sideways glance at me, and that look indicated to me for the first time that Ken took the project very seriously indeed." [1996 8 6]

"By late October [1989], with an intervening 2-week vacation in the Far East, all the elements of the interpreter were in place (the architecture has not changed to this day)." [1995 2 4]. J was presented in public for the first time in a regular meeting of the Toronto APL SIG on February 26, 1990. [1996 8 6]. Roger left IPSA (by that time called Reuter:file) on Friday 13 April 1990, and the following Monday he was officially with Iverson Software Inc. [1996 8 6]

"On August 13, 1990, at the APL90 conference in Copenhagen, J was presented as the paper APL\?, authored by Hui, Iverson, McDonnell, and Whitney. The publication deadlines were such that the title of the paper was chosen several months before that date. By the time of the conference it was definitely called “J’. At the same time, J was released as freeware. I believe you can still get this version from the APL Shareware Exchange." [1996 8 6].

On January 27, 1992, Roger’s An Implementation of J was published. Roger writes: "I proposed to Ken that he should be listed as co-author, but he refused. Failing that, I put Ex ungue leonem as the acknowledgment of the book, because anyone with a passing familiarity with Ken's work could easily see his major and essential influence (for example, the chapters match exactly the organization of the dictionary). As well, I have the paw marks all over my body to prove it." [1996 8 6]

"Ex ungue leonem is a Latin phrase", Roger writes, "that I first heard from Eugene many years ago, which he said means we know the lion from his paw. Webster's New Collegiate Dictionary, in Foreign Words and Phrases section, translates it as from the claw (we may judge of) the lion and from a part we may judge of the whole. The phrase is also alluded to in William Dunham's Journey Through Genius [Wiley, 1990; Penguin, 1991, page 202]: Legend has it that Johann Bernoulli, after reading an unsigned solution from Isaac Newton for the brachistochrone challenge problem, uttered in awe, I recognize the lion from his paw". [1996 8 6]

The Oxford Dictionary of English Proverbs (2nd Edition, 1948, p.370) cites Leonem ex unguibus aestimare, Erasmus I.ix34, 1579-1580. The lion is known by his claws. Bartlett’s Familiar Quotations (14th Edition, 1968, p.86b) compares this with Ex pede, Herculem. Herodotus c.485-c.425 BC. From Aulus Gellius (Noctes Atticae I, i), who tells how Pythagoras deduced the stature of Hercules from the length of his foot.

Ken acknowledged his indebtedness to Roger, not only "for his rapid development of a flexible, reliable, and highly portable implementation of J", but "for his close collaboration in the design of the language, for critical reading of successive drafts of the dictionary, and for significant contributions to the treatment of certain topics, in particular, the identity functions." [J: Introduction and Dictionary (1996)]

In developing J, Ken and Roger have formed a classic partnership. Thinking of other great examples of collaboration in science, I recall Cayley and Sylvester, the "invariant twins" in 19th century mathematics; and James Hutton and his friend, John Clerk of Eldin, in 18th century geology. In evaluating Clerk’s contribution to Hutton’s Theory of the Earth (the foundation of modern geology), the mathematician, Professor John Playfair (who knew both men well), wrote "It cannot be doubted, that, in many parts, the system of the latter has had great obligations to the ingenuity of the former, though the unreserved intercourse of friendship, and the adjustments produced by mutual suggestion, might render those parts undistinguishable even by the authors themselves" (1805).

Ken has commented that in conversation, in working together, ideas fly back and forth, and often the ideas are valuable not in themselves but in the reactions elicited. The acknowledgment in Roger’s APL87 paper said: "Ideas herein evolved over many conversations with K.E. Iverson ...".

Both Ken and Roger are remarkably generous in giving time to helping others. On countless occasions, I have turned to Ken for help with questions about some new feature in J; and increasingly he has told me that Roger was the one to give the answer. Roger well deserves to be honored for his implementation of J written in C : "Although the implementation language is C, the programming style is unmistakably APL" [Implementation, 1992, p.1]. But we rightly pay tribute to Roger also for his collaboration with Ken on the design of the language itself.

Someone hazarded the comment that " Basically, Iverson would write up models of J expressions in J, and Roger would translate them to C." But Roger replied: "It does not happen that way. Most primitives are described in a telephone conversation. Sometimes the entry in the dictionary comes first, and then the implementation proceeds from that. Quite often, there is not a clear idea of what the primitive should be, and successively more detailed implementations are used as a tool to refine the idea." [1995 10 20]

An issue often discussed ever since 1991 is whether J is a dialect of APL. Roger’s statement is authoritative and deserves to be widely known and respected [1996 4 3]:

"The J parsing rules are specified as a 10 by 5 table in section II E of the dictionary. The table is taken directly from the interpreter; that is, these really are the rules, and not the rules as you wish they were. The table, pasted into here from the C source, reads as follows. ... The J interpreter is built around this table. ...

"As indicated in article APL & J one more time, posted on March 12 [1996], the J parsing rules are the APL\360 parsing rules with the following modifications and extensions:

Roger’s work on J is a labor of love. Even when employed by Morgan Stanley he took time to answer questions on newsgroup comp.lang.apl. I have a file of over 200 contributions. To the question "How can I convert a string containing cr lf frets into a matrix?", Roger replied }: ;. _2 This striking answer was accompanied by a lucid explanation of the cut conjunction, examples, and variants. In his typical way, Roger sent us back to the source in Ken’s writings: "Finally, the original description of cut, in K.E. Iverson, Rationalized APL, IPSA Research Report #1, April 1983, pp. 19-20, is the best specification I have ever seen. I photocopied the two half pages onto a single 8.5 by 11 sheet, studied it over many days and read it at least a dozen times, and was deeply impressed." [1994 3 20] If genius is an infinite capacity for taking pains, Roger is well qualified.

Roger frequently advised that Ken’s writings (and his own) demand close reading; here are examples of his admonitions:

Some of Roger’s postings are pedagogic gems, giving valuable guidance succinctly; such as: [1994 10 16]

Roger’s no-nonsense style appears when he responds to unfavorable judgements:

Sometimes users try to fight back, but to no avail. Roger holds his ground:

Though Roger is a severe critic, he has a rich sense of humor, and likes to share a joke. Here is one he sent on: [1996 3 29]

"One day, utterly frustrated by my English teacher’s stubborn habit of nitpicking my essays, I cried out, "If God handed in an essay, you probably only give Him a 95%!"

He chuckled, and then shot right back: "Yeah, I'd deduct marks for improper capitalization."

Mike Kent shared a characteristic anecdote: "It was at APL 90. I saw Roger standing in the balcony area, just after he'd given his paper presentation. The Q&A afterwards had struck me as a sequence of (longish question, short response). So I asked Roger, “Do you EVER use two words when one will do?’ and he thought about the question for a while, maybe half a minute or so, smiled, and after another few seconds replied “No’. It was quite striking."

Roger is a perfectionist with the highest standards, but he is also a man of great kindness and generosity. A friend who has known him since his undergraduate days says: "I cannot find enough good to say about him. The future of the world may look bleak to someone of the older generation. But when we see someone like Roger, we know it will work out well. He is an absolutely splendid person."

I haven’t known Roger that long, but he sent me a letter that must rank as one nicest I ever received. It contains two sentences, and says a lot about Roger.

"Today [1992 2 14] I read your paper in the IBM Systems Journal, Language as an intellectual tool: From hieroglyphics to APL. It makes my heart sing."

Coming from Roger, that makes my heart sing!

Roger recounts: "I'd recently told Arthur that I wish he'd written two pages instead of one that August afternoon in 1989 -- the influence on the J programming style would have been that much greater. (Someone once asked Peggy Lee who she thought was the best jazz singer. Her answer was, You mean besides Ella? Likewise, the answer to the question Who's the best programmer? is Besides Arthur?)" [1995 2 4]

"Besides Arthur?" -- Roger is my candidate for that role. In 1993 Roger was named an ACM Lecturer and received the Outstanding Achievement Award from the British APL Association for his work on J. In the unlikely event that there are any in SIGAPL who do not already know Roger’s work, my tale here should persuade them that his qualifications as recipient of the 1996 Kenneth E. Iverson Award are unlikely to be surpassed.


Roger’s e-mail messages are identified by [year, month, day]

To return click on Browser Back button