Discovering new settings for your Emacs

If you’re like me, you enjoy configuring Emacs so that it is a reflection of what would be the perfect editor for you. That process usually involves finding about new features and functionalities and adding them to your init.el file. Thanks to GitHub, you can now view thousands of init.el files. Just do a search for path:init.el and you’ll find the configuration file of every Emacs user that deemed useful to keep his/her configuration in a VCS.

Direct link

Clojure and Starcraft

In my last post, I mentioned that I was working on a port of my pyreplib library to Clojure. Although I still have work to do with it, it’s in a state that I don’t feel uncomfortable sharing it. You can get clj-starcraft from GitHub. If you have comments on code style, reliability, API friendliness, optimization possibilities, etc. don’t hesitate to let me know.

Although it’s not a large project by any measure (it’s less than 700 lines of code), it is my largest Clojure project to date, and I’d like to give some of my impressions of my experience as I recall them. There’s no particular order or structure, it’s just random thoughts written out as they came to me.

Access to Java: Clojure’s direct, wrapper-free access to Java libraries is extremely beneficial. Starcraft replay files are packed, and the code to unpack them is very ugly (C code, Java code) and I was extremely glad that I could easily re-use the work of somebody else and spare me the pain of all that shifty bit shifting.

In the future, I want to add a small GUI to clj-starcraft to visualize some statistics (unit and action distribution, the number of actions per minutes for every minute during the game, etc.) and knowing that there are already tools written to do these tasks helps me stay motivated.

There are not too many one year-old languages that give you access to such a vast ecosystem.

The JVM has also some nice profiling tools (that I found out about in this article) that were very helpful in getting a good idea of where the bottlenecks were. -Xrunhprof was especially useful in realizing how costly reflection is when you don’t use type hints.

Speed: To process 1050 replay files that I downloaded for testing purposes, pyreplib takes 10 minutes. clj-starcraft takes 4 2:45 minutes (edit: a small modification in a function gave me a nice performance boost). I have not looked deeply into why pyreplib is slower (and have no real intention of doing so), but it’s clear that Clojure is faster than Python for most tasks.

One tool that was very useful to perform quick performance tests is the time macro. Just wrap an expression into it and it’ll tell you how long it took to execute.

Expressivity: When I wrote pyreplib, I really wanted to have a declarative way to define the different types of actions. I was inspired by the model and form declarations from Django and tried to emulate them. The result worked, but it was pretty complex and involved too many lines of codes, meta classes and fragile creation indexes. In Clojure, the task was much easier; I used a very simple 4-line macro. Generally, my Clojure code is shorter and cleaner (no mutations) than the equivalent Python code.

Community: The Clojure community is a friendly, knowledgeable and growing one. When I started poking around with Clojure, the IRC channel had about 60 people on it. Today, it always has over 100 people connected at once. Of course, most of them are lurkers, just like in any other channel, but I think it’s a good reflection of the growth of Clojure. People on IRC and in the Google group are very eager to help; you don’t usually stay stuck for long. Rich Hickey, Clojure’s author, is also extremely accessible: one afternoon, I discussed on IRC about an issue regarding type casting that appeared in r1265, wrote a small example program, posted it to the mailing list and the next morning, there was a patch that fixed my problem along with a message from Rich advising me on how to better write the functions that were affected by the bug.

Pitfalls: It wasn’t all roses all the time however, I learned a lot by being burned.

• Java has no unsigned data types: I didn’t know this when I started the library, but Java does not have unsigned integer types like C. This caused weird bugs in my program until I finally figured it out and converted bytes and shorts to integers (I did not convert integers to longs because I don’t think a game of Starcraft could possibly last 62,138 hours).
• Reflection is slow: at one point, pyreplib was significantly faster than clj-starcraft. Where it would take 0.5 seconds for pyreplib to process a single replay, clj-starcraft would take 8-9 seconds. With the help of the profiler and people on IRC, I learned that adding type hints to avoid reflection is essential to have good performance.
• You *really* should avoid macros if you can. I initially started writing my parse-buffer function as a macro, and I found it hard and error prone. Pretty much everyone on IRC commented that there wasn’t really any reason why this shouldn’t be a function, and solving the problem became much easier.
• Too much flexibility can be constraining: Clojure offers so much flexibility that sometimes you’ll have absolutely no idea how you want to write your code. When faced with this, apply the following method: try the simplest possible thing, and if it doesn’t satisfy you, try the next simplest thing.