Project Euler, MIT Mystery Hunt Edition

Sun, 10 Jan 2010 23:52:00 +0000

The MIT Mystery Hunt starts this Friday at noon, and I’ll be participating seriously for about my 10th year. In the hunt, teams solve a collection of puzzles to discover the location of a gold coin hidden somewhere on campus. The puzzles may be numerous (sometimes over 100), are generally provided without instructions (except when they are provided with painfully explicit instructions), and the overall structure of the hunt is also unknown at the start. Hunt is nearly always finished before Monday, but during the 60 hours starting Friday at noon, many highly capable teams, some with several dozen members, will try their hardest to solve some very challenging puzzles.

kdb+ now available for free download

Sat, 12 Apr 2008 10:05:00 +0000

If you’ve talked to me about programming languages and Wall Street in the last 4 years, I’ve probably mentioned kx. This is a company which makes a combination programming environment and database based on a language called q which is derived from APL. (Yes, APL, the language invented in 1957 before there was a computer to run it on.) And this environment is in turn used by many of the top quants on Wall Street (and other parts of the financial world) for both research and production systems. Becoming a kx programmer is a good way to double your salary and quadruple your job security.

OOPSLA 2007: New Ideas, Good Quotes, and Cynical Self-Satisfaction

Sun, 28 Oct 2007 08:26:00 +0000

I’m still in Montreal, and still thinking about a lot of what I saw and learned at OOSPLA. But I wanted to jot down some links while the cool things were still fresh in my mind. So here is a short set of links worth following that I picked up while at OOPSLA.

There was of course the workshop on associations as a first class language construct, which I already blogged about he r e.
As part of that, Jiri Soukup talked about his project IN_CODE and in particular his UML Layout tool, which attempts to provide sufficiently good graphical layout of UML that the machine-generated layout can be the only layout. The theme of graphical languages with a single canonical display was recurring. I picked it up again from Jonathan Edwards (author of Subtext), who has been working on a canonical visual representation for conditional constructs, and presented it as No Ifs, Ands, or Buts. While I missed the talk, I like the ideas.
I saw a demo of AmbientTalk/2, which is a cute language that makes event-driven programming the norm. The big idea is that all objects belong to a specific thread of control (an actor), and their methods can only be executed in that thread. At method call sites the caller explicitly calls either synchronously (with a ‘.’ operator) or asynchronously (with a ‘<-’ operator). These ideas may have originated in earlier languages with which I was not familiar (E was mentioned). This definitely buys a chunk of attractive simplicity. However, the demo failed to present any complicated synchronization patterns. I’m forced to wonder if we need better benchmarks for the expressivity of parallel languages. Building an IM system just isn’t very hard.
Erik Meijer’s Confessions of a Used Programming Languages Salesman was pretty clever, mostly talking about his experience trying to bring fire down off the mount to Visual Basic programmers. Computer Scientists need regular injections of reality when it comes to things like industrial adoption (usually non-adoption) of their ideas. It was also one of several talks at OOPSLA about LINQ and how awesome C# 3.0 is. I’m definitely envious of some of the features of C#, and I hope Java catches up soon (1.7 is looking pretty good in some ways, but certainly not all). Erik talked a lot about the extension points added to C#, but I’m at the moment unable to find anything on the web that tells me how to add LINQ-style keywords to the language for arbitrary languages, like say an extended version of SQL.
The Domain Specific Languages panel was entertaining, but neither conclusive nor particularly inspired. It was the first place the phrase “Silver Bullet” came up, but far from the last. Unfortunately (and ironically) everyone seems to have a different idea of what a Domain Specific Language (DSL) is, leading to wildly different concerns. More on this in a future post.
Sean McDirmid’s talk and live coding demo of SuperGlue was the most impressive new thing I saw at the show. SuperGlue is an event driven language and a live programming environment. As you edit programs (in Eclipse, of course) the program is continuously executed and re-executed. Sean built up a game of Pac-man, Julia-Child-style. As each new block of code was added to the program, the game gained new behavior. Even, thanks in part to eclipse, responding to typing successive characters of a numeric constant, like “330″. I really hope that he makes his code, or at least videos, available. Unfortunately, I’m afraid that Sean’s demonstration and discussion of the benefits of live programming overshadowed his more theoretical contribution of object-oriented signalling. His paper from ECOOP 2006, SuperGlue: Component Programming with Object-oriented Signals may discuss that in more detail. Sean is unlikely to keep working on SuperGlue, which is implemented in Scala on Eclipse, as he is going to work for Microsoft.
Sharing the stage with Sean, linked in the minds of the organizers because both talks had “live” in their title, was Living in the Comfort Zone, which proposes restricting program inputs using a “rectifier” in order to avoid exposing programs to inputs that, while legal, might be unexpected and trigger bugs. Interesting idea.
The Tuesday night performance of 50 in 50 by Guy Steele and Dick Gabriel was indescribably awesome. I expect a video, and will post a link when it becomes available. Various people compared it to Steele’s 1998 performance-essay Growing a Language, which I had not previously seen and which I can now recommend (you may not understand what he is getting at until page 4).
Tony Kay presented a way to punctuate transactional-memory transactions to make them nest, which just further convinced me that transactional memory is far from the solution to the difficulty of threaded programming. Their model might be better, but it is at best a patch which solves only some of the problems caused by an insufficient underlying abstraction.
StreamFlex was definitely interesting. Inspired by StreamIT and other stream processing work, it provides an events-and-operators model not unlike Aurora (the prototype for StreamBase), where the operators are kernels of Java code. The main contribution, to me, was the approach to real-time semantics. They instrumented their virtual machine to provide some simple transactional memory features, and then use them to assure real-time performance of the real-time thread. All method calls into objects shared with the real-time thread are logged using transactional memory. In any conflict between the real-time thread and another thread, the transaction is aborted for the non-real-time thread. However, to make this work they introduce a somewhat complicated object-ownership system, enforced by compile-time typechecking. Combining the real-time approach with the object ownership of AmbientTalk I mentioned above might be interesting.
Thomas Kuehne tried to answer the question Can Programming be Liberated from the Two Level Style?, but I remained unconvinced. The two level style means classes and objects. In many programs, the “item-description pattern” is used to allow a richer set of “is-a” relationships than the class hierarchy generally affords. While I support the idea of turning design patterns into language features, the mechanics of DeepJava failed to inspire me with a desire to teach them to the average 6.170 student. Inheritance is tricky enough without meta-inheritance.
No Silver Bullet Reloaded was the Panel Of The Stars at OOPSLA, with Fred Brooks, David Parnas, and other panelists who would have seemed famous in other company. Martin Fowler, in particularly, put on a fabulous performance as The Werewolf (who spent much of his time gloating). The forgone conclusion was that no, there had been no successful silver bullet in the 10 years since Mythical Man Month was updated. As for discussion of what we should do about it, the general idea was that we should do all those good things software engineering pundits have been telling us for years, and that we should make sure to employ smart people. Interestingly, panelists from industry were the ones pushing for employing only smart people, which seems like an ineffective strategy when you are trying to develop software in the large at a company like Cisco. The attitude of cynical self-satisfaction (“industry just needs to do what we already told them to do, which they will never do”) seemed distressingly pervasive at OOPSLA this year.
I caught the end of a talk by John McCarthy about Elephant 2000, a new language which he has been developing for over a decade. I didn’t really follow his description of Elephant, and after talking to others about it I think it is ill conceived. The second question was, roughly, “Could you talk about your experience inventing LISP instead?”. McCarthy seemed happy to do that, and what followed was 30 minutes of entertaining anecdotes about early computing.
There were a number of interesting projects presented only on posters at the conference. Chris Pickett of McGill presented Software Speculative Multithreading for Java, which not only contained several interesting ideas, but was the most interactive poster I’ve seen (photo here), being made of movable felt pieces that could be used to talk through demonstrations. The general idea was that at each method call site, come up with a result predictor, and spawn a speculation thread that will continue the computation as if that result had been returned. Speculation can be running several levels deep at a time. This required significant VM modification, which he is now turning into a library so that it can be added to the IBM JVM. There is also a lot of work to be done on heuristics for choosing call sites for speculation, and choosing predictors.
Talking to people about startups versus large companies, I got an interesting anecdote from someone involved in VoIP at Nortel: In their estimation, it takes them 2 years from project start to get a product into the marketplace, while it takes their startup competitors only 1 year. However, after 2 years they have a product which meets the needs of their customer base, while it takes the startup 3.5 years. Makes me wonder what these constants look like in other product areas.
I attended the Seaside BoF. I find a lot of the Seaside technology very interesting, but as befits their roots in the Smalltalk community, they have a lot of difficulty marketing themselves. I ended up serving as a resident heretic, which I hope was helpful, as their main topic of discussion was how to make Seaside adoption as easy as Ruby on Rails adoption. My heresies included “What do you mean I have to use a special smalltalk editor instead of emacs?”, “Are continuations really scalable? Why can’t I get a straight answer?” and “How am I supposed to choose a runtime/database/editor/etc when I don’t know anything about Smalltalk yet?” This turned into some extensive discussions about the adoption curve of destabilizing development tools, which I will try to write up later.
I had a slow morning on Thursday and missed Context, Perspective and Programs, which for many people was something of a religious (or drug-induced) experience. That’s what I get for not reading speaker bios in addition to abstracts. Hopefully there will be a video.
I sat in on a talk titled Neovictorian Computing, which had a lot of good quotes, but didn’t say a whole lot. The best quote, which reflected a lot of what went on at OOPSLA, was “The pedagogy of software engineering is almost entirely about guilt.” The speaker is the creator of Tinderbox, a tool for keeping personal notes. The thrust of the talk was that we need to empower our users to be craftspeople, and to produce small software systems that meet their own unique needs. Tinderbox, as described in the talk, seemed potentially the right system for my contact management needs (I’ve been unsatisfied with all off the shelf contact managers). Unfortunately, I found it difficult to use and not nearly as interesting as the talk had made it seem. Maybe it is the best we can do, when targeting end user programmers (the tool is used mostly by authors to manage book notes), but I would hope we could do better.
I had a couple of talks with Canadians about how their country could benefit from more startups. Unfortunately, most encouragement of startups seems to be government funding, which I think is fundamentally flawed. More on this later.
The final talk of OOPSLA was Brian Foote, revisiting his paper Big Ball of Mud, about the commonness of evolved software systems. He is a very good speaker, and it was an entertaining talk. His focus was on finding the right metaphors to communicate how something is working. My favorite was “sustainable development”, suggesting that many deadline-driven software projects take unsustainable shortcuts, much like some farmers in the Amazon, prematurely depleting a programs maintainability. But he was also quick to point out that some techniques that software engineering pundits disdain, like cut and paste, are a fundamental part of nearly all software development, and that when you let reuse become the overriding concern, you can end up with something highly inefficient, like the Space Shuttle (the reuse of the machine, not the software, in that case). He also described at length about how we are losing the “War on Bugs”, how attempts to crack down on domestic production have merely driven bug production overseas. Programmers start with “gateway bugs” like unchecked error codes and undocumented exceptions, and move on to harder bugs like cross cutting dependencies and threading. Foote went on like that for a while. I may just pull the quotes out of my notes and post them separately.

That’s it for my train of consciousness notes from OOPSLA 2007. It inspired a lot of thoughts, so you may be hearing more from me on some of these topics.

OOPSLA 2007: “The Popularity Cycle of Graphical Tools, UML, and Libraries of Associations” – not the workshop I expected

Tue, 23 Oct 2007 09:59:00 +0000

Today was my first day at OOPSLA 2007 in Montreal. After a brief exposure to the amazing “Underground City” (really a shopping mall that is infecting downtown like a cancer), I crashed a workshop for which I had not signed up, and had a completely different experience than I had planned at The Popularity Cycle of Graphical Tools, UML, and Libraries of Associations.

The title implied it was the workshop about graphical programming languages and tools. As an author of a graphical programming language, StreamBase StreamSQL EventFlow, it seemed like the place to be. My plan had been to pick up some contacts in the graphical languages space, and learn about what has come before. However, the participants (with one vocal exception) had already accepted as a given that UML would eventually decline in popularity, to be replaced by a and they were focused on what would come next.

Transactional Memory Not Solution To All Problems

Fri, 01 Dec 2006 20:10:00 +0000

I was at MIT today and so I ended up going to an invited talk on computer architecture, Subtle Semantics and Unrestricted Implementation of Transactional Memory. Transactional Memory is a very hot topic in systems and architecture. It is perceived to be a better model for programmers, so language designers like it. And there are a variety of options for pure-software and hardware-assisted implementations. And because it enables optimistic concurrency control, transactional memory can help make programs faster and more scalable on new multi-core architectures. There is every reason to believe that processor vendors will begin including some form of hardware support for transactional memory.

Programming Languages on Bulrush Labs — Richard Tibbetts

Project Euler, MIT Mystery Hunt Edition

kdb+ now available for free download

OOPSLA 2007: New Ideas, Good Quotes, and Cynical Self-Satisfaction

OOPSLA 2007: “The Popularity Cycle of Graphical Tools, UML, and Libraries of Associations” – not the workshop I expected

Transactional Memory Not Solution To All Problems