Mechanical Robot Fish

The Mixed-Up Thoughts of Michael Francis Booth

Actuarial

As usual, today’s XKCD makes you think.

XKCD cartoon: Actuarial

  • Life-Line, the story of an inventor who learns how to predict the time of a person’s death to within a single hour, was the first story that Robert Heinlein ever sold, in 1939. As in all Heinlein stories, the great part of this story is the portrait of a character: The inventor, who successfully (and a bit annoyingly) remains calm and rational about the fact of death, including his own death, even as everyone around him is freaking out.

Asimov treated the story of the soothsayer as a grand historical epic (Foundation); Clarke wrote stories full of spiritual awe (e.g. The Nine Billion Names of God, 2001), but Heinlein boiled it all down to this one guy in a shabby office. He was the Raymond Chandler of science fiction – until the 1960s, at any rate.

My friend Stewart Brand, who is now 69, has been arranging his life in blocks of 5 years. Five years is what he says any project worth doing will take. From moment of inception to the last good-riddance, a book, a campaign, a new job, a start-up will take 5 years to play through. So, he asks himself, how many 5 years do I have left? He can count them on one hand even if he is lucky. So this clarifies his choices. If he has less than 5 big things he can do, what will they be?

I appear to have eight big things I can do – forty years, 14691 days.

  • I think that a timeline which plotted the lifetimes of famous people – including the expected lifetimes of people who are alive right now – would be a great tool. I’ve loved timelines ever since I thumbed through The Timetables of History as a kid.

You need to occasionally think on a timescale that is longer than a single human life. You need to accept that people’s lives have arcs, like stories. You need to remember that the world will go on after you.

Drupal’s Data Structures and the Properties Pattern

I might be crazy, but as I read Steve Yegge’s essay on the Properties Pattern I felt as if I was being introduced to something I already knew. And that’s not just because the essay describes Javascript. I think that’s because Properties lists are the fundamental Drupal data structure.

This line stuck out:

I’ve talked about the main problems imposed by the Properties pattern: performance, data integrity, and navigability/queryability. They’re all trade-offs; you’re sacrificing in these areas in order to achieve big wins in flexibility and open-ended future extensibility for users you may never meet.

Yep, that sure sounds like Drupal’s $node, $user, and $form objects to me.

A Horse Designed by an Open-Source Committee

I have only three things to say about the heroic efforts of Mark Boulton to come up with a design for a Drupal wordmark that will satisfy the entire Drupal community.

First: Mark is a brave man.

Second: I hereby officially delegate my opinion on the Drupal wordmark to Mark, Dries Buytaert, Barry Jaspan, and Angie Byron. If they like it, I like it, by definition. My desire to further influence the decision is more than trumped by my belief that such artistic decisions can’t be made by a giant, public committee.

Finally: Voting. Draw up three strong contenders, then let the community respond to a questionnaire. Don’t request too much free-form feedback in open forums during your design process. You will go stark raving mad.

The firm that recently redesigned my alma mater’s logo did a great job with this: The first I heard of the project was when I received an email link to an online questionnaire. The questionnaire took ten or twenty minutes to complete, featured three variations of the logo, and asked all sorts of fuzzy questions – I can’t remember any of them, but they were something like “does this logo express the intellectual qualities you associate with the university.” Or whatever. In retrospect, it didn’t really matter what the questions were – the important thing was that they asked a sufficient number of important-sounding questions to convince me that I was participating in a thorough, serious decision-making process.

Then, a few months later, the university announced the winner, and, lo! I was happy. Because, hey, the new logo looks pretty good! And because I had been asked.

Javascript: Packing, Unpacking, and Beautifying

A lot of the Javascript on the web ships in compressed form. This is particularly irritating if you discover that, say, Drupal 5’s jstools module has bugs that you’d like to fix, but the module ships with nothing but compressed versions of the Javascript code. (What were they thinking?)

A common method of packing Javascript is Dean Edwards’ packer algorithm, which has a convenient online version. You can recognize code that’s been through the packer by looking for the cute function signature at the beginning:

eval(function(p,a,c,k,e,r){...

It turns out that you can unpack code like this using Dean Edwards’ online tool. This isn’t immediately apparent, because the tool doesn’t enable this feature by default (presumably because it’s trying to pretend that packing is an effective form of code obfuscation). But if you run the handy “reEnable” bookmarklet the packer page becomes an “unpacker” page.

Once your function is unpacked you are only halfway home, because it will look like this example (from the jQuery History/Remote plugin included with Drupal jstools):

(function($){$.ajaxHistory=new function(){var c='historyReset';var k=location.hash;var e=null;var g;this.update=function(){};var h=function(){$('.remote-output').empty()};$(document).bind(c,h);if($.browser.msie){var f,initialized=false;$(function(){f=$('<iframe style="display: none;"></iframe>').appendTo(document.body).get(0);var ...

No line breaks, not so readable. So feed it through the online beautifier for Javascript and you’ll get

(function($) {
$.ajaxHistory = new
function() {
    var c = 'historyReset';
    var k = location.hash;
    var e = null;
    var g;
    this.update = function() {};
    var h = function() {
        $('.remote-output').empty()
    };
    $(document).bind(c, h);
    if ($.browser.msie) {
        var f, initialized = false;
        $(function() {
           ...

Much better.

The Spammers Have Me Outnumbered

Whew, just cleaned out several hundred spam comments. Those bots are prolific. I guess it’s time to see what Dries Buytaert’s anti-spam project is all about…

In the meantime, lest you suspect that I’ve made a New Year’s resolution to stop writing, I should tell you that I’ve been spending too much of my writing time posting as mechanical_fish on Hacker News, a fairly awesome site full of people who know much more about software than I.

UPDATE: From my Mollom page: “Mollom blocked 7360 spam attempts the past 10 days”. So far, so good. Mollom is hereby recommended.

UPDATE 2: That seems to have been a typo on the Mollom site: I think they meant 736 spam attempts over 10 days. Much more believeable. And still scary.

Drupal and PHPass, Macs and GPG

As I was setting up a new Drupal site, I decided to try out the hilariously named phpass module. (I cannot read that as “PH Pass” to save my life. It always comes out as… something else.)

The good news about this module is that it builds upon a PHP project called, um, phpass to add better password hashing to Drupal. The traditional way to handle passwords is to ask the user for one, compute a hash function on it, and store the hashed version. Unix systems use the Unix crypt utility to make the hash. Some newer and more naive systems, like core Drupal, use MD5 hashing, presumably because it’s newer (and, therefore, niftier by definition) and also because it’s faster.

Unfortunately, it’s a bad idea to use a fast hash function to hash your passwords, because the speed makes the brute-force attacks that much more efficient. The Right Thing to do, if we trust the professionals at Matasano Chargen, is to use an adaptive hashing scheme like bcrypt, which is tricky and slow, and can be made slower and trickier as computers get faster and faster.

Unfortuately, neither my Mac nor my Ubuntu deployment box supports CRYPT_BLOWFISH, the encryption scheme that’s needed for all-out bcrypt support. So I am using the phpass fallback scheme for now. I could try to install CRYPT_BLOWFISH using the Suhosin PHP hardening extension, but would need to test this carefully to make sure I don’t break Drupal in the process.

In the meantime, I got halfway into the Suhosin downloading process before I decided to put it off until tomorrow. Part I of the process was to finally install Gnu Privacy Guard, which I have always resisted because it seemed to be a usability horrorshow with few actual uses. I only know two people who really seem to use GPG-signed mail, let alone GPG-encrypted mail. But it turns out that there’s now a handy set of instructions for installing GPG on a Mac using the MacGPG project. And I might even get GPG working with Mail once the GPGMail utility finishes being ported to Mac OS 10.5.

Installing a Git Server Using Gitosis

I’m switching all my personal projects to git from Subversion. After watching the Peepcode git screencast, Subversion feels oh-so-2002 and I can’t wait to bury it forever.

First we have to get git running on my Ubuntu server. I tried

sudo apt-get install git-core #do not do this!

but that led to trouble down the line. Better to just fetch the source and build from that:

sudo apt-get install libexpat1-dev zlibc curl gettext
cd ~/src
wget http://www.kernel.org/pub/software/scm/git/git-1.5.4.rc2.tar.gz
cd git-1.5.4.rc2
make prefix=/usr/local all
sudo make prefix=/usr/local install

Then, following the excellent instructions on scie.nti.st, we grab the gitosis code:

cd ~/src
git clone git://eagain.net/gitosis.git

Then:

cd gitosis
sudo apt-get install python-setuptools
python setup.py install

Next we have to create a git user to own the repositories:

sudo adduser \
    --system \
    --shell /bin/sh \
    --gecos 'git version control' \
    --group \
    --disabled-password \
    --home /home/git \
    git

We copy my public ssh key into /tmp/id_rsa.pub, then run

sudo -H -u git gitosis-init < /tmp/id_rsa.pub
sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update

And that’s the end of the server-side setup! On the local machine, we check out the files that are needed to control the server. First we have to account for the fact that I run SSH on a nonstandard port: edit ~/.ssh/config and put this inside:

Host www.example.com
    Port 32767

Then you can do:

git clone git@YOUR_SERVER_HOSTNAME:gitosis-admin.git
cd gitosis-admin

This is the directory where you administer gitosis.

Social Networks: Stop Designing Out the Fun

Today in the news we find that Google is adding lame, privacy-defeating “social networking” features to popular apps without user consent. This comes on the heels of Facebook’s Beacon trainwreck, which was preceded by a similar Facebook privacy trainwreck. There are many lessons to learn from these incidents: That users really do care about privacy as soon as you start abusing their data; that web users DO NOT WANT to have the same persistent, verifiable – and easily traced – ID attached to all their online actions; that my innate distrust of Gmail is not as paranoid as I once thought. But let’s focus on a fundamental problem: Social networking is one of the most significant developments on the web today, and our leading app designers apparently have no clue about how socializing works and why we humans work so hard at it.

Stop Designing the Neutron Bomb

It would be trivial to design a better interface than DOOM if the goal was to kill the bad guys as quickly as possible: give me a 2D map of the area with icons for enemy troops and let me drop bombs on them by clicking the icons. Presto: game over in a few seconds and the good guys win every time. That’s the design you want if you are the Pentagon, but it makes for a boring game. Jakob Nielsen

Facebook and Google – and, one fears, a bunch of current and future startups – operate on the assumption that sharing links with your friends is a tedious chore. Telling your friends about your most recent purchases is a chore. Figuring out which addresses in your book correspond to your best friends is a chore. So they try to automate these chores away: In the spirit of Jakob Nielsen’s high-efficiency first-person shooter, they build one big flat list of all your friends and offer you an elegant one-button interface: SOCIALIZE.

Hello? People use social networks because they are fun. Tweaking your Friend list is fun. Personally choosing the links to be sent to your friends is fun. Choosing when and how to poke people is fun. Flirting is fun! Deciding which of your friends should be introduced to each other is a source of social anxiety, but it is also fun.

Feel free to automate the cleaning of my laundry and the removal of my trash. I don’t really enjoy those things. But stop thinking of my social life and my public persona as tasks that I want to outsource, instead of as hobbies that define my personality.

Fun is Deadly Serious

Perhaps I should stop writing right now. Once you’ve told people to stop designing out the fun, is there any more to say?

Maybe. The problem with fun is that many people don’t respect it. They think it’s frivolous, and trivial, and arbitrary. Because people share links “just for fun”, designers feel free to quietly, unilaterally change the way that shared links work. “Why are you taking Google Reader so seriously?”, they say. “Lighten up and share your data!”

These designers are insane. Fun is serious business. Just look at how animals have fun: Cats think that stalking fast-moving objects is fun, dogs think that herding and chasing are fun, gerbils think that burrowing and chewing are fun, and parrots think that – believe it or not – social networking is fun. For animals, fun is associated with the practice of vital survival skills.

Humans work the same way. Reproduction? Fun. Looking at babies, especially your own? Fun. Hunting, watching birds, playing Half-Life? It’s fun, and it’s also a way to practice the stalking skills that you’d use for hunting game or fighting wars. Chess? Strategic training for warfare. (Just ask any well-educated kid from the Middle Ages.) Gardening? Practice for farming. Stamp collecting and Pokemon? You’re practicing taxonomy, exercising the parts of your brain that evolved to remember which things you can eat, which things can eat you, which are poisonous, which are likely to be found under trees in July.

Social networking is one of the most fun things of all, because it’s a survival skill that humans are heavily invested in. We’re designed for it. Human language itself is a social networking tool.

Some fun activities are no longer vital for survival: We’ve invested lots of time, energy, and money to ensure that I can eat without knowing how to tell one plant from another, defend myself without knowing how to shoot, and stay warm without flint and tinder. Social networking is not in this category. Social networking may be more important now than ever before in human history. Many historical humans had very little interaction with strangers. They didn’t marry anyone that their parents didn’t already know, and their friends lived within a ten-minute walk. (Of course, even these people had a social network that was too complicated for a computer to manage.)

Modern social networks are very complicated. We travel a lot. We move through many different social groups during our lives. We have multiple overlapping circles of friends. We have technical friends, gaming friends, friends from work, friends from school, friendly customers, friendly teachers, friends we aspire to marry, friends with benefits, platonic friends. We have friends who we met once at a conference in 2006, friends who we went to college with but don’t see more than once every three years, and friends who we used to see every day but who just moved to Idaho. We have relatives, who may or may not be our friends. We have the relatives of friends, and the friends of relatives. We have fundamentalist Christian friends and polyamorous anarchist friends.

Incidentally, we also have enemies.

Mismanaging your social interactions can have terrible consequences. It can cost you your reputation, your spouse, your children, your inheritance, your job, your career, your freedom, and your sanity. If you make the wrong friend, and that friend turns into a stalker, it can cost you your life.

So:

  • Personally micromanaging your social network is fun.

  • The reason it’s fun is that humans are designed for it. This, in turn, is because it’s a vitally important personal skill that determines your income and your happiness. This is especially true in the 21st century.

  • Nobody is going to entrust their social network to a computer, not ever. Computers have not spent millions of years evolving their social skills. Computers are stupid. You have not solved the strong AI problem. If, by chance, you do invent a computer that is smarter than a human, nobody will trust that, either.

  • Build tools that empower people to make better decisions, not tools that try to make decisions for them.

The Ruby vs. PHP BDD Beauty Contest: No Contest

For the last week I’ve been writing a Drupal utility in Ruby. This is a dubious decision, because most Drupal developers would prefer a tool written in Drupal’s native PHP. It’s less hassle to install, and less hassle to modify.

But I went with Ruby anyway, at least for version 0.1 – partly to keep myself in practice, and partly as an excuse to work out with RSpec, the Behavior Driven Development (BDD) framework for Ruby.

With BDD, before you write your code, you write a spec. Here’s a section of one of my specs – it describes an Environment object that can parse a Unix command line and pick out the arguments and the option flags:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
describe Environment, "created from the command line" do
  before(:each) do
    @env = Environment.new
  end

  it "should set the usage message attribute" do
    args = %w(foo bar baz quux)
    @env.parse_command_line(args)
    @env.usage_message.should_not be_empty
  end

  it "should recognize arguments that are not options" do
    args = %w(foo bar baz quux)
    @env.parse_command_line(args)
    @env.should have(4).args
  end

  it "should throw an exception if an illegal option is provided" do
    args = %w(foo -z bar)
    lambda { @env.parse_command_line(args) }.should raise_error
  end

  it "should allow the cvs path to be set" do
    test_path = '/bin/cvs'
    args = ['--cvspath', test_path]
    @env.parse_command_line(args)
    @env.get_option(:cvs_command_path).should == test_path
  end

  it "should be invalid if cvs path does not exist" do
    args = ['--cvspath', 'nonexistent']
    @env.parse_command_line(args)
    @env.should_not be_valid
    @env.errors.should have_at_least(1).error
    @env.errors_on(:cvs_path).should have(1).error
    @env.errors_on(:cvs_path).should match(/not exist/)
  end
end

The idea is that you write a sentence about what the code should do, along with an executable description that demonstrates the sentence. Then you write the code itself until the description runs without failing.

Advocates of Test-Driven Development (TDD) will recognize this as nearly the same thing. But it does have a few advantages:

  • BDD avoids using the word “test”, which carries uncomfortable connotations like “broken” and “buggy” and “ISO 9002” and “let’s do that last, after the documentation”. BDD uses words like “describe” and “should” to guide you into a proper, positive frame of mind.

  • BDD in Ruby uses nifty readable syntax. The syntax is astoundingly addictive. Why read this:

1
assert(wallet.dollars >= 2e6)

when you can read this:

1
wallet.should have_at_least(2e6).dollars

or this:

1
2
bank = mock('a bank with 2 million dollars')
bank.should_receive(:withdraw).at_least(:twice).with(1e6).and_return(:ok)

Oh, sweet RSpec mock syntax. How I wish you had an equivalent in other important languages, like PHP!

Well, the good news is that PHP has testing advocates of its own. Mock objects are apparently well covered by Simpletest, which just happens to be Drupal’s preferred PHP testing library. And the syntax looks familiar – it’s quite nice, despite some additional clutter:

1
2
3
<?php
$observer = $this->getStub('Observer', array('update'));
$observer->shouldReceive('update')->once()->with('something');

Meanwhile, at least one person, Pádraic Brady, is working on a BDD framework for PHP: the PHPSpec project. Unfortunately, the results are… less nice:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
<?php
class DescribeNewBowlingGame extends PHPSpec_Context {
  private $_bowling = null;

  public function before() {
    $this->_bowling = new Bowling;
  }

  public function itShouldScore0ForGutterGame() {
    for ($i=1; $i<=20; $i++) {
      $this->_bowling->hit(0);
    }
    $this->spec($this->_bowling->score)->should->equal(0);
  }
}

Compare this to the RSpec code which inspired it:

1
2
3
4
5
6
7
8
9
10
describe Bowling do
  before(:each) do
    @bowling = Bowling.new
  end

  it "should score 0 for gutter game" do
    20.times { @bowling.hit(0) }
    @bowling.score.should == 0
  end
end

It’s enough to make you cry.

The thing is, I’m completely convinced that Pádraic is doing the best he can. I am no expert on PHP syntax, having learned it by osmosis from Drupal code, and maybe after I finish Programming PHP I’ll know better, but PHPSpec seems to be fundamentally stymied by the ugly syntax of PHP. Specifically:

What’s :this?

Why, it’s PHP’s inability to understand object scoping! You would think that declaring _bowling to be private to a class would be enough of a clue, but it looks like you’re still obligated to refer to it as $this->_bowling.

Little things matter

Ruby uses object.method syntax. PHP uses $object->method. They look so comparable! That is, until you run into a chain like:

$this->spec(foo)->should->equal

which is less readable than:

this.spec(foo).should.equal

It’s all about the typography: the whitespace above the tiny dot separates the words better and makes their shape easier to recognize.

It’s good to be an Object

In Ruby, everything is an object, every object is a child of Object, and Objects can be monkey-patched. So you can magically attach new methods like should and should_not to the Object class and then write things like:

@bowling.score.should be_big

Here we’re calling the should method on @bowling.score, which is probably an Integer, although it might be a BowlingScore object, or a Real, or even Imaginary if we’re bowling on the planet Vulcan. Ruby does not care. Ruby can roll with those punches. Ruby rocks.

Alas, PHP prosaically insists that you call the should method on specific objects that can actually understand it. So we have to wrap the subject of should in a $this->spec(), like this:

$this->spec($this->_bowling->score)->should->equal(0);

This line makes my brain hurt because it’s got two instances of $this, both of which are 100% free of significant meaning. The thing which they refer to – the enclosing PHPSpec_Context object – is an irrelevant piece of scaffolding that I do not want to have to think about.

The PHP version of the Bowling spec reads like this:

This paragraph is a spec, and it-describes-Bowling. Consider the game of bowling that we will discuss in this paragraph. The score-should-be-zero-for-a-gutter-game. That is, if a ‘hit’ that scores zero pins happens twenty times, this sentence of this paragraph will be true: this paragraph’s game’s score should equal zero.

Here’s the RSpec version:

Let’s talk about Bowling. Consider a game of bowling. The score should be zero for a gutter game. That is, if a ‘hit’ that scores zero pins happens twenty times, the game’s score should equal zero.

(Note: Fun as it would be to write down the prose equivalent of a for loop – it would look like a number-theory textbook – I decided not to blame PHP for that. I’m pretty sure the language does have iterators…)

PHP’s syntactic cyanide is so bitter that I wonder: is PHPSpec worth it? If the language itself insists on mangling my syntax, why should I bother trying to apply the subtle shading that is BDD? It just gets lost in the noise. Here’s a comparison from the PHPSpec docs themselves: the TDD version (PHPUnit):

$logger = new Logger;
$this->assertTrue($logger->hasFile());

and the BDD version (PHPSpec):

$logger = new Logger;
$this->spec($logger)->should->haveFile();

Frankly, I think the first version is clearer: it’s shorter, the call to hasFile() is in its natural location, and since neither version is readable, why not pick the one that’s closest to idiomatic PHP? Others seem to agree with me; just look at this comment thread.

Pádraic has his work cut out for him, battling heroically against the fundamental structures of the PHP universe. I really feel for the guy, and would sincerely like to help. My personal inclination is to build a macro processor that allows you to write something sensible and have it JIT-compiled into PHP. But maybe I can get used to PHPSpec if I make some subtle vocabulary changes:

$this->thing($this->_bowling->score)->thing->should->equal(0)

This thing, this PHP syntax thing… we will learn to cope with it.


Update: Jan 6, 2012

Dan Bernardic writes:

“All PHPspec is missing is a DSL on top of what he has. Except that I expect it would be a better idea to implement a way for ruby to run PHP code and inspect PHP memory state, just cause rspec for ruby already exists…”

and includes a proof-of-concept Gist.

I’ve toyed with this idea myself - which is to say, I’ve sat in an armchair and imagined it, not actually hacked on it - but there are drawbacks. The biggest is that it’s not enough to test one’s PHP code in isolation; one should test it in an environment that’s as close as possible to the production environment. Unless you plan to run your production PHP code inside a Ruby wrapper - and, believe me, you don’t - a Ruby-with-embedded-PHP testing framework is going to trade reliability for syntax, and that’s not a good trade unless the syntax is truly brainf**ked.

Ruby, Rspec, and Autotest Bug on Mac OS 10.5 Leopard

Now this is an obscure bug: Autotest on Leopard will break when used with projects containing Rspec specs, because it can’t find the “spec” command in the right place. It doesn’t look in /usr/bin/spec, where Leopard places it.

I tried the tricky answer before I gave up and adopted the simple answer:

$ sudo ln -s /usr/bin/spec /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/spec