Introducing Chops

Recently I’ve been doing a lot of my exploratory programming in JavaScript, thanks to the fact that I know it’ll run on crazy platforms like the 3DS. -_^ I’ve programmed in JavaScript off and on since high school, and boi has my JavaScript style changed…. @_@

JavaScript’s lack of macros is a bit troubling for me, especially given the recent tendency for JavaScript programmers to deal in lots of nested asynchronous callbacks: “Unindented functions to clean callback mess.” “Asynchronous callbacks in JavaScript”

I haven’t felt the pain of the asynchronous callback nesting problem firsthand, but I know a horror close to that one: Monadic programming in Groovy. Observe the indentation.

In fact, that indentation-heavy part of the Blade code is what originally got me fed up with using Groovy and interested in making Penknife as an intermediate project on the way to Blade. Well, my work on Penknife has been so focused on modularity and hygiene that it’s slack on what really matters: Syntax. Oh, syntax, how you really really matter so! :-p

A Language Should Support the Future

There’s one design goal of Arc I find completely self-evident. Arc’s designed for good programmers. Duh. Programmers in the future will be better than us, because they’ll be capable of observing our mistakes. If a language doesn’t target good programmers, it doesn’t have a very bright future.

There’s way more to this concept than than Arc tackles, though.

Programmers in the future will learn from the mistakes we make designing languages today, and they’ll design better ones. For there to be a single language that persists despite future innovations, snowballing its own library support along the way, it ironically needs to be all the various languages people want it to be. It needs to be a customizable language. (And it needs to target platforms other than its runtime.)

Foreign Affairs

Any two of those languages will easily target each other’s runtimes, so if things happen the way I expect, we’re likely to find a landscape full of competing languages that all abstract over each other with no clear winner.

To limit that effect, it’s important for those languages to try to have no obvious defects from the outset, so that they don’t build up audiences that feel they need to spin off into various camps to pursue slightly better languages.

For this reason, I think it’s a mistake for any language that takes itself half-seriously to impose an arbitrary limitation on its users. A limitation that makes other things easier is fine, but the point of a tool is always to make people more productive, not less. Good programmers will police themselves as necessary.

But for the same reason, I think the strongest position in a winnerless landscape of customizable languages is a minimalistic one. If a customizable language’s runtime has only the features it needs for convenient syntax customization, the designer has fewer opportunities to mess up and introduce flaws. In that sense, the language should be severely limited, but for a non-arbitrary cause.

Civil Unrest

Meanwhile, each of these languages will itself host a lot of competing syntaxes and abstraction layers. Not all of those will follow the language’s own philosophy, ’cause they’ll be their own things. But many will be general-purpose tools for people to use when customizing the language’s syntax, and their presence will effectively recolor the language for people who want to take advantage of their added features. I believe the language should have a strong philosophy encouraging consistency between these libraries and the main language runtime API, so that the experience doesn’t fray into separate and competing sub-experiences.

But a language’s library landscape is anything but minimalistic, so it can’t share that aspect with the core. Libraries will inevitably introduce extra concepts with flawed designs and buggy implementations. Instead of trying to avoid this, library programmers should assume the programmers who use their code know better than they do, by nature of those programmers being in the future. Thus, when in doubt, they should leave their implementation details exposed rather than hiding them. They should maximize their users’ freedom, just like the language itself does.

A naive take on this would be challenging for libraries in active development. They tend to change their implementation details from release to release in incompatible ways, so if they feel pressured not to break others’ code, that limits their freedom! To solve this, I believe library users should be capable of using unstable features, but that the language culture should encourage library writers to make it clear what’s stable and what isn’t, perhaps in such a way that even if it’s easy for library users to use the unstable parts, it’s also easy for them to refrain from doing that.

Conclusion

So altogether, I believe the best kind of language is a minimalistic, customizable-syntax, cross-targeting build language which has a philosophy encouraging consistently unstable library APIs, makes nothing intentionally hard, and most of all prepares for future innovation.

Not only is this my kind of language, I think it’s imperative to make these languages extremely well from the beginning so that it we don’t let suboptimal versions gain the advantage of entrenchment.

A Language Should Target Moving Platforms

Platforms are not languages to me. Languages don’t deal with security, file operations, threads, and so on. Languages are syntaxes people use to develop tools on platforms.

Platforms stack, vaguely. The LAMP stack is a classic example: The server is Linux running Apache, content-specific sub-platforms are developed on top of PHP or Perl for Apache to use, data sub-platforms are developed on top of MySQL for the PHP/Perl code to use, and the machine code programs (the OS included) rely on a hardware platform of some sort or another.

Platforms also target other platforms and hide them away. For instance, a PHP coder hardly cares about the instruction set in use on the machine, ’cause the interpreter takes care of that. It’s easy to write a simple PHP script that doesn’t really care what the hardware, the OS, the server, etc. look like. In this case, PHP itself may be all we’re thinking about. In that case I’d say that PHP is our platform.

The reality today is that platforms and languages are developed symbiotically. Innovative general-purpose platform concepts spawn new languages suited for them, like Erlang. Languages which intend to support multiple platforms just end up defining their own. Most platforms can’t be perfectly translated into each other, so ports of platform-defining languages can only be sparse or imperfect, and when it comes down to it, one’s choice of platform determines one’s language options.

Many programmers, including myself, prefer to choose languages not based on the platforms they target but based on their smoothly adapting syntax as project-specific needs come up. Languages which support extensible syntax, like most lisps, are ideal for this purpose, since they hold an implicit infinity promise that they’ll eventually be every bit as beautiful and convenient as every other language, if people only bother to develop the necessary syntax extensions.

But customizable languages fall short today. People are used to developing programs that target just one platform—the runtime of the language they’re writing in—and that makes customizable languages only as flexible as the one runtime they’re limited to.

For a language to have an inflexible runtime is important for code reuse, but if we’re going to have a convenience-and-aesthetics language that’s capable of continuing to build momentum over the next, say, 100 years (Arc’s shtick), we need a language that will target 100 years of platforms we haven’t invented yet. Therefore, the platform the language code runs in and the platform it targets (usually) won’t be the same.

Instead, the language needs to encourage a development process that’s like writing a compiler or build script, where the code produces project artifacts like binaries and other-language scripts, and those are all that’s deployed.

AST Processing

If we start with a “compiler” that just outputs what it’s told to output, then writing code generators in the language will already be at least as convenient as writing the code manually, so the infinity promise will already fulfilled, albeit unimpressively. Templating languages are a step beyond this even, but flat string concatenation is an ineffective way to go forward. Instead, I believe the customizable language’s core focuses should be on AST manipulation, parsing, dependency resolution, and other compiler duties.

This is the direction I’ve been taking Blade and Penknife, my own programming languages. Platform innovation is well and good, but I believe truly general-purpose languages are nothing if not homes for syntax innovation.

Lathe’s Arc-to-Implementation-Language FFI

$ cd /path/to/jarc

$ java -server -cp \
>   "jarc.jar;$GROOVY_HOME/embeddable/groovy-all-1.7.2.jar" jarc.Jarc

Jarc> (= lathe-dir* "path/to/lathe/arc/")
"path/to/lathe/arc/"
Jarc> (load:+ lathe-dir* "loadfirst.arc")
nil
Jarc> (use-rels-as jv (+ lathe-dir* "imp/jvm.arc"))
#3(tagged mac #<procedure>)
Jarc> (jv.jvm!groovy-util-Eval-me "2 + 2")
4

It’s on now.

After about a month of me submitting bugs to the Arc forum and Jarc’s creator, JD Brennan, fixing them, Jarc is now compatible enough with Arc 3.1 that my Lathe (the library I introduced last time) now wholeheartedly supports the Jarc implementation of Arc. This means it works on four Arc setups: Jarc, official Arc 3.1 (which itself needs a Windows fix), Anarki (which needs the same fix), and Rainbow. Most of you reading this are probably from the Arc forum, in which case you knew all that already. :-p

Those are all the setups I want Lathe to support for now, but I might consider arc3f, arc2c, or another Arc implementation if I realize it’s actually active. Another complication is that quite a lot of Arc users modify one of the PLT Scheme versions of Arc (Arc or Anarki) to suit their needs. In those cases, I figure the burden of Lathe compatibility is on them if they need it.

Still, there is a bit of a need to add new features to Arc. To this end, Anarki, Jarc, and Rainbow have provided ways to call from Arc into the host platform (PLT Scheme, the JVM, and the JVM respectively).

Modules, Multimethods (Kinda), and More for Arc

I’ve been doing things here or there in Arc without really having a specific plan in mind for sharing them. Most notably, I had the fundamentals of an untested Arc multimethod system laying around, which were made in an effort to collect my thoughts about Blade, a potential idea for an everything-is-a-multimethod language. And that’s what I expect a large part of my projects to be: Approximations of a language I’d rather program in. So finally, last week I started up a GitHub repo, Lathe, where I could submit miscellaneous utilities that smoothed out non-Blade languages according to my own aesthetic.

MVTron gets Xuggle-er with MediaTools

It’s been a while since I did much with MVTron, but I took a brief look hack at it the other day, and it’s actually coming along, I think. That said, it isn’t something I’m actively working on, so there probably aren’t going to be any spectacular results or anything.

Earlier, my approach to MVTron was to wrap AviSynth in the hopes that it could take advantage of all the fast AviSynth plugins people have written in C. I managed to get a scene detector working that processed my test video in about twice the video’s duration. That struck me as a pretty poor performance, but I was still hopeful, and I was planning to get my plan in motion from there.

Well, it’s kind of embarrassing to say this, but I completely forgot about that AviSynth-Groovy success story until just now when I read my own blog. That said, this time around I duplicated that success in less than a day without interfacing to AviSynth at all, using Xuggle’s (relatively) new MediaTools API. And now I’m usually looking at processing times only 170% the test video’s duration. Thanks, Xuggle!

Programming Language Mix-Up

Last time, I talked about TADS 3 and Groovy. I noted that TADS 3 had features especially suited for IF—undo support, a concise, declarative syntax for game content, a special syntax for defining IF-style command-line grammar—and some other features I really liked, but that for some reason I still felt like I preferred Groovy.

Why? Not-invented-here syndrome, in a sense. The IF system I would like to work with is not nearly the system that the TADS 3 language and libraries help out with. Just to name a couple of cases, I dream of experimenting with undo history and parser grammar beyond what the TADS 3 system and grammar syntax support. This wouldn’t make TADS 3 a bad language for the job, but I expect this would make its domain-specific features into dead weight when they didn’t suit my needs, and when that happens I’m afraid I’ll find myself programming in a domain-limited equivalent of Groovy after all.

Since Then

Between the time I made that post and now, a few months have gone by, and I’ve done a lot of different things:

TADS vs. Groovy

After thinking a long, meandering thought process starting from Groovy Science, wandering through programming language design, and somehow following a segue into interactive stories again, a friend re-sparked my interest in IF (interactive fiction). I’ve gotten absorbed reading about IF theory and practice before, but that was about five years ago, and now I know much more about programming and a little bit more about writing. So I downloaded the TADS 3 and Inform 7 SDKs and some IF interpreters, and I got to work reading the developer reference material and actually playing some games, including Photopia, Violet, Glass, and Alabaster.

One of the first things I noticed about the IF languages was that superficially, TADS 3 the language is incredibly close to Groovy. It uses C-style imperative syntax, it has closures, it uses dynamic typing, it minimizes the difference between properties and methods, and it has what look like interpolated strings (which print automatically, meaning it takes a bit of output-rerouting to use them for anything else). Now, I’m not saying these are all great features, but I knew right away that if I was going to be comfortable enough in any IF language to implement the kinds of crazy things I find myself wanting to implement all the time, it was going to be TADS. I could port my general-purpose Groovy code to TADS as I needed to.

“Dun, Dun, DUNNNNN”

A few weeks ago, I was watching Lost, and I thought I heard the TARDIS showing up in the show. (At least, I thought it would’ve been awesome if it did. XD ) At the end of a particular scene, someone said something particularly dramatic, and then there was a “whoooooOOOSH” and the scene changed. That sound happened several times more during the episode, and despite all the “It’s the TARDIS! The TARDIS is everywhere!” entertainment I was having, I admitted in my heart of hearts that this was just Lost’s way of making the dramatic twists sink in.

Pretty soon, I noticed the same kind of thing happening in all kinds of shows, including Chuck and (I think) CSI. There seems to be a tendency toward a kind of rising hum or twang. Yet, that’s not the only option; Boston Legal’s score does the same thing, but it’s composed of jazzy vocals and stuff.

So, you’ve got rising action, a twist, an obligatory sound effect, and a quick scene change. I guess it’s a formula.

So why do I bring it up? I’ll get to that in a second. (Er, make that several seconds. -Future Me)

And So MVTron Discovers Marsyas

I’ve been looking over audio analysis techniques, wrapping my mind around how exactly I might go about implementing (or finding someone else’s implementation of) an FFT or DWT, getting to the point where I can understand this abstract, and I notice something there. The writers of that used something called MARSYAS. What’s that?

Well, Marsyas is an open-source C++ project that apparently is exactly as ambitious as MVTron would like to be in exactly the same ways. Although its main focus is music and other audio, apparently there’s MarsyasX, a branch or something, which is a reimagining of Marsyas to be more video-inclusive. Considering Marsyas’s seeming focus on feature extraction, similarity detection, and… well, all kinds of stuff, it seems like MVTron would be just another application in the sea over there. That takes a lot of (self-inflicted) pressure off of me as a lone programmer. :-p

It almost looks at this point like MVTron will end up being a project entirely submerged in Marsyas, maybe even to the point that it’s written in C++… but I guess I shouldn’t be so hasty. I’ve only just heard of Marsyas, and besides, there’s an entry on their ideas page calling for “Porting the Marsyas dataflow architecture to Java.” ^_-