Putty featured in Prison Break

In yesterday’s episode of Prison Break, you can see the putty icon on the laptop at the beginning of the show. My blog automatically resizes the image, so click on it for a better shot.

Reformatting a CSS file with Vim

Derek Slager wrote a post (and made a screencast) of how you would clean up an ugly CSS file. I decided to do it in Vim. Here are the steps. No screencast from me, unfortunately.

" Replace all sequences of white spaces with one space
:%s/[ \\t\\n]\\+/ /g

" Go to the end of the command, then forward one character and insert
" a newline
]/lr^M

" Add a newline after every semi-colon
:%s/;/;^M/g

" Add a newline after every opening brace and make put one space
" between it and the preceeding text
:%s/\\([^ ]*\\) *{/\\1 {^M/g

" Add two newlines after every closing brace
:%s/}/}^M^M/g

" Remove 'trailing' spaces in front of the semi-colons
:%s/ *;/;/g

" Make sure there is only one space after a colon
:%s/: */: /g

" Make the text before the colon lowercase
:%s/\\(.\\{-}\\):/\\L\\1:/g

" Remove all trailing spaces at the beginning of lines
:%s/^ \\+//g

" Go to the beginning of the file
gg

" Record a macro to sort the attributes
qa2jV}2k:sort^M}q

" Repeat the macro a few times
20@a

" Indent the whole file
gg=G

Edit: Fixed the backslashes. WordPress sucks.

What Reddit can do for your blog

Saturday, I wrote a post where I compared Vim and Emacs. In the afternoon, it was posted on reddit. This increased the traffic on my blog significantly. I have a small Perl script that reports the number of unique IPs that visited my site each day. I took that data and used the cool Gruff library for Ruby to generate a graphic of the traffic I got in the past days. A quick side note, I wasn’t able to find a package for Perl that didn’t require downloading half of CPAN and couldn’t find something simple (not matplotlib) for Python. Any suggestions?

Click here to view the graphic

Blatant security problem

The provincial election campaign is on in Quebec. I did a quick search for the ADQ, l’Action Démocratique du Québec. Check out the 3rd link returned by Google.

Emacs and Vim

I spent this week using Vim. Yes, me, the Emacs fan-boy using Vim. Why you ask? Because it was better than using Zend Studio. See, we are working on a PHP project at work, and the general slowness of Zend and its primitive key bindings left me wanting for more. I tried Emacs, but I ran into a few indentation problems. Also, I knew that I wouldn’t be writing as much new code as correcting mistakes in old code, so I thought that Vim would be the best for that job. And it was; I quickly got back into Vim mode in spite of not using it for much more than configuration files editing. This made me think about Vim and Emacs as editors and of course, which one is the better one? The short answer: neither is better than the other. They both got great features in common that other editors lack such as file recovery. There are some points however where I have a preference for one or the other. This is what this post is going to be about, a bunch of features where I think one of the two has a advantage.

Editing (Vim)

Vim is king when it comes to editing code. There is no contest whatsoever. Take any general- or specific-purpose text editor, pit it against Vim and Vim will come out as the winner. I like modal editing, it gives you a whole lot more commands to work around with your text. As you may have heard if you are no a vi/vim user, the commands such as deleting, changing (deleting and going back in insert mode) and yanking (copying) can be mixed with the very powerful movement commands. You want to delete text until the second closing parenthese? d2t). That’s it. The 2t) command brings the cursor over the character preceding the second parenthese after the current position of the cursor. To delete the text between these two positions, you just prepend ‘d’. Emacs doesn’t have this kind of power. You need to go in mark-mode (where you select text), and use either the movement commands or do a search on ‘)’, hit C-s to go to the next match and then C-w to kill the mark. This works, of course, but it’s a bit more work. Vim also does an operation on an entire line if you press on the operator key twice: dd deletes the current line. The way to do that in Emacs is C-a C-k C-k (or just one C-k if you set kill-whole-line to T). No matter what other modeless editors try to do, I don’t think they can match the editing power of Vim, period.

Edit: A helpful commenter mentioned the zap-to-char command, which is mapped to M-z, so my example that Emacs can’t delete up until a particular character doesn’t work anymore. However, if we change the operation from deleting to upcasing or downcasing up until a particular character, Vim works the exact same way (with the gU and gu commands.) Emacs doesn’t have such a built-in command, so you will need to select the text and then use M-x upcase-region.

Key bindings (Vim)

I love the Vim keybindings. The non-reliance on the modifier keys such as Ctrl and Alt is appreciated by my wrists. I don’t have carpal tunnel, however, after long Emacs editing sessions I sometimes feel a lot of pain in my wrists. Vim is also better when I use a french canadian keyboard. As an example, in Emacs to indent a region, the keyboard shortcut in C-M-\. On a US keyboard, this isn’t so bad, however on a french keyboard, writing a backslash is done by pressing AltCar (the Alt key to the right of your space bar) and the ` key as located on a US keyboard. This key sequence is so awkward that most of the time, I just use M-x indent-region instead. With Vim, I select the region with V (visual line) and press = to reindent. No pain there. Vim also just has a whole lot more keybindings than Emacs. In a column called My save-excursion, Steve Yegge, a die-hard Emacs user, writes a Lisp function to count the number of characters in the current buffer. He also counts word by simply dividing the number of characters by 5. Commenters are quick to point out that he could’ve selected the entire buffer and ran the wc Unix program on that selection (the complete sequence is C-x h M-| wc <RET>). Surprisingly, I don’t think I saw any Vim user say that he could’ve used g CTRL-G had he been using Vim. If you want to see some unusual Vim commands, look at all the g and z commands.

Selection mode (Vim)

In Vim, selecting text is called visual mode, in Emacs it’s setting the mark. I prefer Vim here too for the same reasons as editing. One of Vim’s visual mode is the visual block mode. It allows you to define a rectangle and edit on that section. Emacs has something similar, however you need to visualize the rectangle: the entire lines are selection (except the first and last) and you need to remember that when you are going to use a rectangle editing command (which are all different from the normal mark commands by the way, another source of annoyance) the section that’s going to be worked on is the wide as much as from the beginning of the mark up until the end of the mark. It’s kind of hard to explain, but Vim makes the whole thing easier by showing you just the rectangle you are about to edit. And in Vim, the keys do not change; d will delete the rectangle as it deletes a normal selection. In Emacs, deleting a normal selection is C-w, but deleting in rectangle mode is C-x r d. I often forget the rectangle editing commands in Emacs.

Syntax highlighting (Vim)

There’s only one word to win this for Vim: inkpot. Inkpot is the sexiest color scheme ever, it puts to shame all other vim and Emacs color schemes. Vim also seems to color more elements than Emacs, so that’s another point in its favor.

Hey, funny story, while I was looking for a screenshot of inkpot, Firefox froze up and for some reason froze openbox, my window manager, too. I hadn’t saved this post (which I am writing in Vim), but Vim’s recovery system worked flawlessly. I wasn’t even worried.

Auto indenting (Emacs)

Maybe it’s because Emacs has a more powerful and expressive extension language, but Emacs has less problems with auto indenting than Vim. Here are two examples:

# Emacs indenting Python code
foo(10,
    "hello")

# Vim indenting the same Python code
foo(10,
        "hello")

# Emacs indenting Ruby code
a = {
  'one' => a.sort { |a, b|
    a.length  b.length
  },
}

# Vim indenting the same Ruby code
a = {
  'one' => a.sort { |a, b|
    a.length  b.length
},
}

In pretty much any language I used, the Emacs indenting was always better than Vim’s. Emacs has also a command that I find myself using all the time called M-x align. Basically, you select a bunch of assignment lines, you call align and all the equal signs will be aligned. It also works with dictionaries/hashes. Aligning code makes it easier to read, and it’s nice that I don’t need to do that by myself.

Command line (Emacs)

When I talk about the command line, I refer to the minibuffer in Emacs (accessible with M-x) and when you do a :, / or ? in Vim. Emacs has a few advantages here. Although both do tab completion, Emacs’ minibuffer works with dabbrev-mode, so you can type the beginning of a word then M-/ and Emacs will complete the word. If there are multiple matches, pressing on M-/ again will cycle to the next match. Vim cannot do that, using CTRL-P and CTRL-N will go up and down the history of commands. Another thing I prefer about Emacs is that when you use C-x C-f to open a file, the current working directory appears, so you know exactly where you are; Vim doesn’t do that. The iswitchb-mode in Emacs shows all the buffer names when you press C-x b, and when you start typing, iswitchb will match that string anywhere in all buffer names and remove those that don’t match. This is a very powerful and very useful way of changing buffers. This makes it easier, in my opinion, to work with multiple files in Emacs.

Extension language (Emacs)

Emacs is configured in Lisp, Vim is configured in VimScript. Lisp is just more powerful than VimScript and it is also more present in the system: I can do C-h f to view the documentation on a Lisp function, I can do C-h v to view the documentation on a Lisp variable. I can use any buffer to input Lisp code and C-x C-e will evaluate it. When I was looking for a way to wrap text in rcirc, an IRC client for Emacs, at the width of the narrowest window, I needed to write a function to find the smallest width. Here’s that function:

(defun narrowest-window-width ()
    (apply #'min (mapcar #'window-width (windows-list))))

Easy, isn’t it? And I was able to find about window-list and window-width by using the afore-mentionned built-in help system for Lisp. When I look at the available modes and applications for Emacs, such as Gnus, ERC, rcirc, tetris, I can’t help but think that the Emacs folks chose the best possible language to extend Emacs. Vim supports many other languages besides VimScript, such as Python, Perl, Ruby, TCL, mzscheme, but I don’t think we’re likely to see popular extensions in those, because since they rely on a scripting extension that might not be available, portability is lost. Maybe a future version of Vim will officially support a second scripting language that will be included in the default distribution.

Documentation (tie)

Both editors have a tremendous amount of documentation that is browseable directly from the editor. I know I said earlier that I would list only the features where one editor had an advantage, but good documentation is often ignored in the open source world, and it’s important to mention that both editor are top-notch in this area.

Integration with other tools (Emacs)

Emacs has been integrated with virutally every tool in a programmer’s toolbox: debuggers, make, compilers, interpreters, manual pages, grep, version control systems, shells, you name it, there’s an Emacs mode to interact with that. The reason Vim gives to explain this lacking can be found in the *design-not* section of the documentation (:help design-not):

- Vim is not a shell or an Operating System. You will not be able to run a
shell inside Vim or use it to control a debugger. This should work the
other way around: Use Vim as a component from a shell or in an IDE.
A satirical way to say this: “Unlike Emacs, Vim does not attempt to include
everything but the kitchen sink, but some people say that you can clean one
with it. ;-)”

In spite of this goal, it appears that Vim users want more and more. The Clewn project integrates gdb with vim, Vim-Shell makes it possible to run shells and other programs in Vim’s split windows. Most languages have syntax and indent files, but people are now uploading plugins to http://vim.sf.net to make the manipulation of certain type of files (C files, Python files) easier. To an Emacs user, this sounds like they are trying to replicate the features found in Emacs’s programming modes. In version 7, new features have been added to VimScript (:h new-vim-script) such as lists, dictionaries and function references to make the extension language more powerful. A more powerful extension language means more advanced plugins. I expect that Vim will soon have programming support that rivals Emacs’ most powerful modes.

Folding (Vim)

Vim does folding out of the box, Emacs has something really basic called narrow-mode. Vim’s folding is much more like what we’ve come to expect from an editor that what Emacs offers. Vim also has the advantage of having multiple folding methods (manual, indent-based, marker-based, etc.) and has a bunch of z commands to open and close folds.

Self-contained (Vim)

Starting with version 7, Vim has its own built-in spell checker. This is a good thing, because you don’t have to rely on an external tool to do the spell checking. The *develop-spell* documentation section describes more technical reasons why a built-in spell checker was a better idea for Vim. Emacs integrates with existing spelling programs. On Unix systems, it’s not so bad to install ispell or aspell, but I’m not sure how easy it is to get them working on Windows systems. With Vim, I know I can have spell checking anywhere without having to install a separate, non-related package. Bonus points for the cute squiggly lines under the misspelled words. And as with anything in Vim, there is a flurry of commands to go to the next/previous misspelled word, add it to the dictionary, correct it, etc.

Version 7 also has a command called :vimgrep which works like :grep, but which doesn’t depend on an external grep command. This means that on Windows, where you are unlikely to have grep installed, you can still use that functionality. Again, Emacs’ M-x grep command relies on an externally installed grep program.

Having these commands built into the editor means that your workflow remains the same regardless of the platform you are working on. If you rely on external tools on one platform, then you move to another, your workflow is affected because the same commands cannot be used because the external programs they rely on are unavailable. A side effect of having more features built-in as opposed to being third-party additions is that pretty much everyone will use them, and bugs are much more likely to be found and fixed. It also reduces the number of files you need to carry around to have a comfortable environment. My elisp/ directory is 8 MB and I need it all to feel comfortable. With Vim, I just need my .vimrc file, the inkpot color scheme and the french spelling files. Beyond that, everything I need is already in Vim.

Completion (Vim)

In version 7, Vim introduced omni-completion, which is a bit like IntelliSense. Emacs has an external mode called CEDET that supposedly does something like that, but I was never able to install it, so I don’t know how well it works. OmniComplete is great however. If you type CTRL-P or CTRL-N and there is more than one match, a colorful menu is displayed in which you can select the appropriate completion. See this page for some screenshots of the C++ completion. Completion scripts for other languages are expected to be released, keep an eye out at www.vim.org for them.

K&R: Finished chapter 1

I finished doing all the exercises in chapter 1 of the K&R book. Onward to the second chapter!

K&R: Finished chapter 1

I finished doing all the exercises in chapter 1 of the K&R book. Onward to the second chapter!

Speech recognition

I never believed in speech recognition, it’s never ever going to work for multiple reasons: people’s accents, people’s need for privacy, stupid computers, you name it.

This video shows the pain that awaits people going with speech recognition.

Use a keyboard and be done with it.

Lost passwords in Django

Tom Moertel writes about how to allow users who forget their password to have access to their account again, without having to store passwords in plain text in your database.

A couple weeks ago, I implemented exactly what he suggested for an application at work in Django. Like in my captcha post, I will show to implement that yourself. I will not do the form to actually change the password, that would make this post too long I think. It’s really about how to implement a lost password feature.

First, in your model you will need a small table where you will keep rows containing usernames, unique keys and the creation date.

from django.db import models
from django.contrib.auth.models import User

import datetime

class LostPassword(models.Model):
    user    = models.ForeignKey(User)
    key     = models.CharField(maxlength=70, unique=True)
    created = models.DateTimeField(auto_now_add=True)

    def is_expired(self):
        return (datetime.datetime.today() - self.created).days > 0

Here’s how I mapped URLs:

from django.conf.urls.defaults import *

urlpatterns = patterns('',
    (r'^$', 'demo.lostpassword.views.index'),
    (r'^change/(?P<key>.{70})/$', 'demo.lostpassword.views.change_password'),

)

In views.py, I wrote a simple new_key() function that will return a new key. The index() function will read a username, and if it exists in the User model, a new LostPassword object will be created and the URL to change the password is displayed in the template. In your application, you will want to email that link to the user’s email, but I was concerned about keeping things simple here. The second view, change_password(), will take a key as a parameter and display a 404 if there’s no such key in our database. If that key does exist, we check to see that the page isn’t expired. If it is, we delete the lost password and inform the user that the page has expired. Other wise, you would just display a normal 2 field password form.

from django.shortcuts import render_to_response, get_object_or_404
from django.contrib.auth.models import User
from demo.lostpassword.models import LostPassword

def new_key():
    while True:
        key = User.objects.make_random_password(70)
        try:
            LostPassword.objects.get(key=key)
        except LostPassword.DoesNotExist:
            return key

def index(request):
    if request.method == 'POST':
        try:
            user = User.objects.get(username=request.POST['username'])
            lostpassword = LostPassword.objects.create(user=user,
                                                       key=new_key())
            message = 'http://%s:%s/lostpassword/change/%s/' % (
                request.META['SERVER_NAME'],
                request.META['SERVER_PORT'],
                lostpassword.key)
        except User.DoesNotExist:
            message = 'Unknown user'
    else:
        message = ''

    return render_to_response('lostpassword/index.html',
                              {'message': message})

def change_password(request, key):
    lostpassword = get_object_or_404(LostPassword, key=key)

    if lostpassword.is_expired():
        lostpassword.delete()
        message = 'Page expired'
    else:
        message = 'Time to change that password!'

    return render_to_response('lostpassword/change.html',
                              {'message': message})

index.html

<form method="post" action=".">
    Username: <input type="text" name="username" /><br />
    <input type="Submit" />
</form>

<p>
    {{ message }}
</p>

Finding the Next Big Language

Introduction

Steve Yegge has an interesting article on The Next Big Language. In his article, he describes a language that will become the new dominant language and explains the features this language will need to (a) not suck, and (b) be popular. Some features are both non-sucky and popular, while others are one or the other, mostly (b), because being popular is more important than not sucking (case in point: having a C-like syntax.)

In the entire article, Steve never tells the name of the NBL. He says it’s to avoid flak if that language should eventually fail; he doesn’t want to be the guy who said “640k or RAM ought to be enough for everybody.” My gut feeling is that he has no idea what language fits all the rules he gives. Nevertheless, I would like to go over every feature he mentions and give a list of languages that currently exist or are in open development that I think might fit the description and at the end, I might have an idea of what the NBL is. The languages I will list are not languages that are currently extremely popular (like Java or C++), because it would not be the Next Big Language, would it?

Please not that I am not familiar with all these languages, so it is perfectly possible for me not to be aware of the established future directions of a language or actual features of a language. If I’m wrong or if I missed a language, there’s a comment section to correct me.

Rule #1: C-like syntax

In Steve’s mind, it makes no doubt that to be popular, a language needs to have a C-like syntax. I agree with him. C-like syntax is definitly not the best around (it is debatable which syntax is objectively the best), however since it is a syntax used by a lot of existing languages — C, C++, Java, C#, JavaScript, D and to a lesser extent, Perl — it would be wise to use it.

• D: D is a language written by Walter Bright. The syntax is pure C-like.
• Perl 6: Although Perl 5 had a syntax that is in many places quite different from C and that Perl 6 will push that even further, it still uses curlies to delimit blocks, the operators are the same (except Perl has a gajillion more), the precedence of operators is the same, etc.
• Python: Python doesn’t use a syntax similar to C with respect to blocks, but the keywords and the operators are pretty much the same and I think the operator precedence is quite similar.
• Scala: I don’t know anything about Scala, but the examples show that it’s definitly a C-like syntax language.

Rule #2: Dynamic typing with optional static types

According to Steve, it’s easier to add optional static typing to dynamic language than to bring dynamic typing into a static language. I’m no language designer so I have no idea, although since static-into-dynamic has been done with Common Lisp, Smalltalk (Strongtalk) and others, I guessing he’s right.

• Lisp: Common Lisp already does it.
• Perl 6: Perl 6 will have optional static typing.
• Python: Guido blogged about static typing and Python. Will it happen? I don’t think so, he said Python 3000 would be the last time there were any big changes made to the language, so it’s doubtful.
• Factor: Factor is a very interesting concatenative programming language developped by Slava Pestov. Currently, it’s currently only dynamically typed, however one of the plans for Factor 2.0 (Factor 1.0 isn’t out yet) is optional static typing. That may seem far away, especially with the 18-24 months timeline Steve mentionned, but development is happening so fast in Factor, we never know.

Rule #3: Performance

The NBL will have performance at least equal to Java’s.

• D: D is compiled to native code and currently ranks first on the Computer Language Shootout, even ahead of C and C++.
• Perl 6: Perl 6 will run a virtual machine called Parrot, which has a goal of being faster than Perl 5 currently is. I guess that would place it in the same ballpark as Java’s
• Python: Python currently has Psyco to improve its performance on x86 machines. The Pypy project also has a goal of creating a Python in Python implementation that will run faster than the current C implementation, although it’s not done yet. IronPython has been touted as being faster than CPython too.
• Ruby: Ruby 2.0 will have a new VM called YARV which should improve the performance of Ruby.
• Factor: One of the goals of Factor is performance, and although some parts are still slow (e.g.: IO), every Factor release brings a faster implementation.
• Lisp: There are implementations of Common Lisp and Scheme which have performance almost on par with C and C++.
• ML: O’Caml and and SML are both have extremely efficient implementations which rival C and C++.

Rule #4: Tools

No matter how good a language is, if there’s no IDE, it’s very hard for that language to become popular. This is a category where pretty much all languages have an equal shot: you just need someone to write an Eclipse plugin, an Emacs mode and/or a vim syntax and indentation file, preferably all three. Bonus points for supporting other editors such as UltraEdit or jEdit. Since the language should have a C-like syntax, maybe an IDE like Code::Blocks could be extended to support the NBL.

Aside from editor support, a language should have a good profiler and a CPAN-like repository of third-party modules. Most languages I know provide a profiler (with varying degrees of ease of use), and more and more languages have a repository for third-party modules: Ruby as RubyGems, Python has easy_install, Common Lisp has asdf, etc.

Rule #5: Kitchen Sink

Here Steve lists a bunch of features that people have come to expect:

1. Object-literal syntax for arrays and hashes
2. Array slicing and other intelligent collection operators
3. Perl 5 compatible regular expression literals
4. Destructuring bind (e.g. x, y = returnTwoValues())
5. Function literals and first-class, non-broken closures
6. Standard OOP with classes, instances, interfaces, polymorphism, etc.
7. Visibility quantifiers (public/private/protected)
8. Iterators and generators
9. List comprehensions
10. Namespaces and packages
11. Cross-platform GUI
12. Operator overloading
13. Keyword and rest parameters
14. First-class parser and AST support
15. Static typing and duck typing
16. Type expressions and statically checkable semantics
17. Solid string and collection libraries
18. Strings and streams act like collections

I will list languages and the feature numbers they support. This is probably the section where I will need the most correcting from my (three) readers. I am not sure what a first-class parser and AST support is, so I will not likst this feature.

• Perl 6: 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 15, 16?, 17, 18
• Python: 1, 2, 3, 4, 6, 8, 9, 10, 11, 12, 13, 17, 18
• Ruby: 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 17, 18
• Factor: 1, 2, 5, 8, 10, 11, 12, 17, 18
• D: 1, 2, 3, 5, 6, 7, 8, 10, 11, 12, 13, 16, 17, 18

Conclusion

The language that seems to have the most going for it as far as features are concerned is Perl 6, which is odd because Steve has a lot of bad things to say about Perl 5. Does it mean that in spite of Steve’s disdain, Perl 6 could be the next big language? The thing I see going against Perl 6 is complexity. It’s going to have pretty much every single programming feature you can think of. This means a very large language, something one can’t keep in his head. One person might not even be able to know all Perl 6.

D could be a definite contender too, the only thing lacking is the dynamic typing with optional static typing: D is statically typed. I don’t think this would be such a problem for its acceptance, in fact it might help it: it is more in line with C-like languages that the industry and most programmers are used to (C, C++, Java, C#). Like Steve said, programmers are a stubborn bunch and are very averse to big changes.

Python could have a definite shot, I intoduced it to a co-worker of mine and he really likes what he sees. The simplicity and consitancy of the language are appealing and the number of libraries available grows rapidly. I don’t think that it will ever get optional static typing in the core language. Maybe with function decorators?

Although it has a lot to offer, I don’t think Factor is the answer: I think Average Joe Programmer would rip his head off trying to do things without loops and variables. The syntax is definitely not what people are used to. It remains a nice project to keep an eye on.