mechanical robot fish - The mixed-up words of Michael Francis Booth

Checking out an open-source git project: the right way

Mike — Fri, 07 Nov 2008 13:03:01 -0500

(Paraphrased from Long Nguyen’s guide to checking out projects from GitHub — which is excellent, except that Long uses the name “long” for too many different things, making the code hard for mere mortals to read.)

Suppose you would like to work on the open-source Insoshi project. You have the URL of its official git repository:

git://github.com/insoshi/insoshi.git

You could simply clone this URL to your own machine:

$ git clone git://github.com/insoshi/insoshi.git

after which you could mess around with your local copy as much as you like. But it’s hard to share a local repository with your friends, or among several machines, so perhaps you’d rather create a private fork on a server somewhere. It’s easy to do this with a GitHub-hosted project like Insoshi — you get a GitHub account, then press GitHub’s “fork” button, and you rewarded with the URL for your own personal GitHub-hosted fork:

git@github.com:mechfish/insoshi.git

Now you could make a local clone of that remote fork:

$ git clone git@github.com:mechfish/insoshi.git

but there are disadvantages to that. One is that the local clone will have a different parent: its origin will be remote fork, while the remote fork’s origin is the official project repository. Ideally, you would want the two repositories to be interchangeable, so that you won’t get confused when comparing them, and so you can upload the local version to replace the remote version if it gets lost or corrupted.

You’d also like to be able to pull changes from the official repository directly into your local copy, then push them up to the remote fork, rather than having to pull every change forward through the chain: official → remote fork → local copy. That’s painful.

So here’s what to do:

$ git clone git://github.com/insoshi/insoshi.git
$ cd insoshi
$ git branch devel master
$ git checkout devel
$ git remote add fork git@github.com:mechfish/insoshi.git
$ git fetch fork
$ git push fork devel:refs/heads/devel
$ git config branch.devel.remote fork
$ git config branch.devel.merge refs/heads/devel

Here’s what these commands accomplish:

Clone the official repository and cd into the local copy.
Create a devel branch to develop on. Note that if we want to base our development branch on the “edge” version of the project instead of the stable version we should do this instead: $ git branch --track edge origin/edge; git branch devel edge
Checkout our new devel branch.
Add a remote named “fork” that points to our private fork on the remote server.
Fetch the remote fork’s contents.
Push our new devel branch to the remote fork, where it should have the same name.
Set the config for the devel branch so that it automatically pushes changes to the devel branch on the remote fork when we do a plain git push, and automatically pulls from the devel branch on the remote fork when we do a plain git pull.

Note that with this setup you automatically get the feature that a git push will push up all changes to all branches, including the master branch. (That’s because the default refspec for remotes is refs/heads/*:refs/heads/*.) So you can pull changes from origin directly to your local copy, with the assurance that those changes will be automatically migrated to your remote fork repository when you push.

Installing PHP, Apache, MySQL, and Ruby on Mac OS 10.5 Leopard

Mike — Tue, 04 Nov 2008 15:24:01 -0500

If you want to get up and running quickly using Drupal on Mac OS 10.5 your best bet is to use MAMP, as John VanDyk suggests in his comments. There are MAMP setup instructions on Drupal.org.

But I decided to try moving away from MAMP. One reason is that MAMP doesn’t come with certain features — for example, I’m told that it’s hard to get SSL working. Another is that the upgrade cycle for PHP and MySQL is different from that of MAMP, and I sometimes want to be able to roll my own upgrade on my own timeframe. In addition, I want to play with Ruby technologies like Merb and Rack and Thin, and I didn’t relish trying to hack them into MAMP’s config files — though it would probably be possible to do so, the Ruby docs and blogs tend to be aimed at people who are working outside of MAMP.

So I’ve just gotten everything installed. Here’s a quick sketch of the process:

Follow Dan Benjamin’s instructions for compiling MySQL. This will involve creating /usr/local and adding it to your path, assuming you haven’t already done so.

I’m with VanDyk, who suggests running mysql_secure_installation once you have MySQL installed.
Follow Benjamin’s instructions for compiling your own Ruby and Rubygems. This is a more dubious move, since Apple has provided preinstalled versions of Ruby and Gems in Leopard, and they work pretty well. But I decided to install my own version so that I can keep control over the upgrades.
I recommend avoiding the stock Apache. I spent a while trying to compile a PHP that would work with it, and ran into trouble. Instead, install MacPorts and use that to install Apache, then download and compile PHP.

Note that I installed MacPorts’ Apache, then downloaded and built the PHP source myself, using the following ./configure options:

 ./configure --prefix=/usr/local/php \
--with-apxs2=/opt/local/apache2/bin/apxs \
--with-mysql=/usr/local/mysql \
--with-curl --with-pgsql=shared \
--with-pdo-pgsql=shared,/usr/local/pgsql \
--with-pdo-mysql=shared,/usr/local/mysql \
--with-mcrypt=shared,/opt/local \
--with-openssl=shared,/opt/local \
--with-pear \
--with-gd \
--with-jpeg-dir=/opt/local \
--with-png-dir=/opt/local \
--without-iconv

I’m not sure there’s a good reason to do this instead of just using MacPorts’ PHP. For that matter, I’m not sure that using your own MySQL is better than having MacPorts install MySQL, or using the binary distribution from MySQL AG. There are lots of options. But these options seemed to work for me.

Of course, you’ve also got to set up a php.ini and an httpd.conf and get PHP and Apache pointing at them. And I’ve still got eAccelerator and XDebug to install. It’s a bit of work. I’d advise anybody who wants to avoid trouble to stick with the MAMP option.

Actuarial

Mike — Fri, 24 Oct 2008 10:39:23 -0400

As usual, today’s XKCD makes you think.

Life-Line, the story of an inventor who learns how to predict the time of a person’s death to within a single hour, was the first story that Robert Heinlein ever sold, in 1939. As in all Heinlein stories, the great part of this story is the portrait of a character: The inventor, who successfully (and a bit annoyingly) remains calm and rational about the fact of death, including his own death, even as everyone around him is freaking out.

Asimov treated the story of the soothsayer as a grand historical epic (Foundation); Clarke wrote stories full of spiritual awe (e.g. The Nine Billion Names of God, 2001), but Heinlein boiled it all down to this one guy in a shabby office. He was the Raymond Chandler of science fiction — until the 1960s, at any rate.

Kevin Kelly has a countdown clock of the days he has left to live:

My friend Stewart Brand, who is now 69, has been arranging his life in blocks of 5 years. Five years is what he says any project worth doing will take. From moment of inception to the last good-riddance, a book, a campaign, a new job, a start-up will take 5 years to play through. So, he asks himself, how many 5 years do I have left? He can count them on one hand even if he is lucky. So this clarifies his choices. If he has less than 5 big things he can do, what will they be?

I appear to have eight big things I can do — forty years, 14691 days.

I think that a timeline which plotted the lifetimes of famous people — including the expected lifetimes of people who are alive right now — would be a great tool. I’ve loved timelines ever since I thumbed through The Timetables of History as a kid.

You need to occasionally think on a timescale that is longer than a single human life. You need to accept that people’s lives have arcs, like stories. You need to remember that the world will go on after you.

Drupal's Data Structures and the Properties Pattern

Mike — Wed, 22 Oct 2008 17:59:34 -0400

I might be crazy, but as I read Steve Yegge’s essay on the Properties Pattern I felt as if I was being introduced to something I already knew. And that’s not just because the essay describes Javascript. I think that’s because Properties lists are the fundamental Drupal data structure.

This line stuck out:

I’ve talked about the main problems imposed by the Properties pattern: performance, data integrity, and navigability/queryability. They’re all trade-offs; you’re sacrificing in these areas in order to achieve big wins in flexibility and open-ended future extensibility for users you may never meet.

Yep, that sure sounds like Drupal’s $node, $user, and $form objects to me.

A Horse Designed By an Open-Source Committee

Mike — Tue, 16 Sep 2008 20:18:02 -0400

I have only three things to say about the heroic efforts of Mark Boulton to come up with a design for a Drupal wordmark that will satisfy the entire Drupal community.

First: Mark is a brave man.

Second: I hereby officially delegate my opinion on the Drupal wordmark to Mark, Dries Buytaert, Barry Jaspan, and Angie Byron. If they like it, I like it, by definition. My desire to further influence the decision is more than trumped by my belief that such artistic decisions can’t be made by a giant, public committee.

Finally: Voting. Draw up three strong contenders, then let the community respond to a questionnaire. Don’t request too much free-form feedback in open forums during your design process. You will go stark raving mad.

The firm that recently redesigned my alma mater’s logo did a great job with this: The first I heard of the project was when I received an email link to an online questionnaire. The questionnaire took ten or twenty minutes to complete, featured three variations of the logo, and asked all sorts of fuzzy questions — I can’t remember any of them, but they were something like “does this logo express the intellectual qualities you associate with the university.” Or whatever. In retrospect, it didn’t really matter what the questions were — the important thing was that they asked a sufficient number of important-sounding questions to convince me that I was participating in a thorough, serious decision-making process.

Then, a few months later, the university announced the winner, and, lo! I was happy. Because, hey, the new logo looks pretty good! And because I had been asked.

Javascript: Packing, Unpacking, and Beautifying

Mike — Wed, 18 Jun 2008 16:00:04 -0400

A lot of the Javascript on the web ships in compressed form. This is particularly irritating if you discover that, say, Drupal 5’s jstools module has bugs that you’d like to fix, but the module ships with nothing but compressed versions of the Javascript code. (What were they thinking?)

A common method of packing Javascript is Dean Edwards’ packer algorithm, which has a convenient online version. You can recognize code that’s been through the packer by looking for the cute function signature at the beginning:

eval(function(p,a,c,k,e,r){...

It turns out that you can unpack code like this using Dean Edwards’ online tool. This isn’t immediately apparent, because the tool doesn’t enable this feature by default (presumably because it’s trying to pretend that packing is an effective form of code obfuscation). But if you run the handy “reEnable” bookmarklet the packer page becomes an “unpacker” page.

Once your function is unpacked you are only halfway home, because it will look like this example (from the jQuery History/Remote plugin included with Drupal jstools):

(function($){$.ajaxHistory=new function(){var c='historyReset';var k=location.hash;var e=null;var g;this.update=function(){};var h=function(){$('.remote-output').empty()};$(document).bind(c,h);if($.browser.msie){var f,initialized=false;$(function(){f=$('<iframe style="display: none;"></iframe>').appendTo(document.body).get(0);var ...

No line breaks, not so readable. So feed it through the online beautifier for Javascript and you’ll get

(function($) {
$.ajaxHistory = new
function() {
    var c = 'historyReset';
    var k = location.hash;
    var e = null;
    var g;
    this.update = function() {};
    var h = function() {
        $('.remote-output').empty()
    };
    $(document).bind(c, h);
    if ($.browser.msie) {
        var f, initialized = false;
        $(function() {
           ...

Much better.

The spammers have me outnumbered

Mike — Sun, 11 May 2008 02:11:07 -0400

Whew, just cleaned out several hundred spam comments. Those bots are prolific. I guess it’s time to see what Dries Buytaert’s anti-spam project is all about…

In the meantime, lest you suspect that I’ve made a New Year’s resolution to stop writing, I should tell you that I’ve been spending too much of my writing time posting as mechanical_fish on Hacker News, a fairly awesome site full of people who know much more about software than I.

UPDATE: From my Mollom page: “Mollom blocked 7360 spam attempts the past 10 days”. So far, so good. Mollom is hereby recommended.

UPDATE 2: That seems to have been a typo on the Mollom site: I think they meant 736 spam attempts over 10 days. Much more believeable. And still scary.

Drupal and PHPass, Macs and GPG

Mike — Sun, 30 Dec 2007 14:28:26 -0500

As I was setting up a new Drupal site, I decided to try out the hilariously named phpass module. (I cannot read that as “PH Pass” to save my life. It always comes out as… something else.)

The good news about this module is that it builds upon a PHP project called, um, phpass to add better password hashing to Drupal. The traditional way to handle passwords is to ask the user for one, compute a hash function on it, and store the hashed version. Unix systems use the Unix crypt utility to make the hash. Some newer and more naive systems, like core Drupal, use MD5 hashing, presumably because it’s newer (and, therefore, niftier by definition) and also because it’s faster.

Unfortunately, it’s a bad idea to use a fast hash function to hash your passwords, because the speed makes the brute-force attacks that much more efficient. The Right Thing to do, if we trust the professionals at Matasano Chargen, is to use an adaptive hashing scheme like bcrypt, which is tricky and slow, and can be made slower and trickier as computers get faster and faster.

Unfortuately, neither my Mac nor my Ubuntu deployment box supports CRYPT_BLOWFISH, the encryption scheme that’s needed for all-out bcrypt support. So I am using the phpass fallback scheme for now. I could try to install CRYPT_BLOWFISH using the Suhosin PHP hardening extension, but would need to test this carefully to make sure I don’t break Drupal in the process.

In the meantime, I got halfway into the Suhosin downloading process before I decided to put it off until tomorrow. Part I of the process was to finally install Gnu Privacy Guard, which I have always resisted because it seemed to be a usability horrorshow with few actual uses. I only know two people who really seem to use GPG-signed mail, let alone GPG-encrypted mail. But it turns out that there’s now a handy set of instructions for installing GPG on a Mac using the MacGPG project. And I might even get GPG working with Mail once the GPGMail utility finishes being ported to Mac OS 10.5.

Installing a git server using gitosis

Mike — Thu, 27 Dec 2007 22:47:53 -0500

I’m switching all my personal projects to git from Subversion. After watching the Peepcode git screencast, Subversion feels oh-so-2002 and I can’t wait to bury it forever.

First we have to get git running on my Ubuntu server. I tried

sudo apt-get install git-core #do not do this!

but that led to trouble down the line. Better to just fetch the source and build from that:

sudo apt-get install libexpat1-dev zlibc curl gettext
cd ~/src
wget http://www.kernel.org/pub/software/scm/git/git-1.5.4.rc2.tar.gz
cd git-1.5.4.rc2
make prefix=/usr/local all
sudo make prefix=/usr/local install

Then, following the excellent instructions on scie.nti.st, we grab the gitosis code:

cd ~/src
git clone git://eagain.net/gitosis.git

Then:

cd gitosis
sudo apt-get install python-setuptools
python setup.py install

Next we have to create a git user to own the repositories:

sudo adduser \
    --system \
    --shell /bin/sh \
    --gecos 'git version control' \
    --group \
    --disabled-password \
    --home /home/git \
    git

We copy my public ssh key into /tmp/id_rsa.pub, then run

sudo -H -u git gitosis-init < /tmp/id_rsa.pub
sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update

And that’s the end of the server-side setup! On the local machine, we check out the files that are needed to control the server. First we have to account for the fact that I run SSH on a nonstandard port: edit ~/.ssh/config and put this inside:

Host www.example.com
    Port 32767

Then you can do:

git clone git@YOUR_SERVER_HOSTNAME:gitosis-admin.git
cd gitosis-admin

This is the directory where you administer gitosis.

Social Networks: Stop Designing Out The Fun

Mike — Wed, 26 Dec 2007 12:38:44 -0500

Today in the news we find that Google is adding lame, privacy-defeating “social networking” features to popular apps without user consent. This comes on the heels of Facebook’s Beacon trainwreck, which was preceded by a similar Facebook privacy trainwreck. There are many lessons to learn from these incidents: That users really do care about privacy as soon as you start abusing their data; that web users DO NOT WANT to have the same persistent, verifiable — and easily traced — ID attached to all their online actions; that my innate distrust of Gmail is not as paranoid as I once thought. But let’s focus on a fundamental problem: Social networking is one of the most significant developments on the web today, and our leading app designers apparently have no clue about how socializing works and why we humans work so hard at it.

Stop Designing the Neutron Bomb

It would be trivial to design a better interface than DOOM if the goal was to kill the bad guys as quickly as possible: give me a 2D map of the area with icons for enemy troops and let me drop bombs on them by clicking the icons. Presto: game over in a few seconds and the good guys win every time. That’s the design you want if you are the Pentagon, but it makes for a boring game. Jakob Nielsen

Facebook and Google — and, one fears, a bunch of current and future startups — operate on the assumption that sharing links with your friends is a tedious chore. Telling your friends about your most recent purchases is a chore. Figuring out which addresses in your book correspond to your best friends is a chore. So they try to automate these chores away: In the spirit of Jakob Nielsen’s high-efficiency first-person shooter, they build one big flat list of all your friends and offer you an elegant one-button interface: SOCIALIZE.

Hello? People use social networks because they are fun. Tweaking your Friend list is fun. Personally choosing the links to be sent to your friends is fun. Choosing when and how to poke people is fun. Flirting is fun! Deciding which of your friends should be introduced to each other is a source of social anxiety, but it is also fun.

Feel free to automate the cleaning of my laundry and the removal of my trash. I don’t really enjoy those things. But stop thinking of my social life and my public persona as tasks that I want to outsource, instead of as hobbies that define my personality.

Fun is Deadly Serious

Perhaps I should stop writing right now. Once you’ve told people to stop designing out the fun, is there any more to say?

Maybe. The problem with fun is that many people don’t respect it. They think it’s frivolous, and trivial, and arbitrary. Because people share links “just for fun”, designers feel free to quietly, unilaterally change the way that shared links work. “Why are you taking Google Reader so seriously?”, they say. “Lighten up and share your data!”

These designers are insane. Fun is serious business. Just look at how animals have fun: Cats think that stalking fast-moving objects is fun, dogs think that herding and chasing are fun, gerbils think that burrowing and chewing are fun, and parrots think that — believe it or not — social networking is fun. For animals, fun is associated with the practice of vital survival skills.

Humans work the same way. Reproduction? Fun. Looking at babies, especially your own? Fun. Hunting, watching birds, playing Half-Life? It’s fun, and it’s also a way to practice the stalking skills that you’d use for hunting game or fighting wars. Chess? Strategic training for warfare. (Just ask any well-educated kid from the Middle Ages.) Gardening? Practice for farming. Stamp collecting and Pokemon? You’re practicing taxonomy, exercising the parts of your brain that evolved to remember which things you can eat, which things can eat you, which are poisonous, which are likely to be found under trees in July.

Social networking is one of the most fun things of all, because it’s a survival skill that humans are heavily invested in. We’re designed for it. Human language itself is a social networking tool.

Some fun activities are no longer vital for survival: We’ve invested lots of time, energy, and money to ensure that I can eat without knowing how to tell one plant from another, defend myself without knowing how to shoot, and stay warm without flint and tinder. Social networking is not in this category. Social networking may be more important now than ever before in human history. Many historical humans had very little interaction with strangers. They didn’t marry anyone that their parents didn’t already know, and their friends lived within a ten-minute walk. (Of course, even these people had a social network that was too complicated for a computer to manage.)

Modern social networks are very complicated. We travel a lot. We move through many different social groups during our lives. We have multiple overlapping circles of friends. We have technical friends, gaming friends, friends from work, friends from school, friendly customers, friendly teachers, friends we aspire to marry, friends with benefits, platonic friends. We have friends who we met once at a conference in 2006, friends who we went to college with but don’t see more than once every three years, and friends who we used to see every day but who just moved to Idaho. We have relatives, who may or may not be our friends. We have the relatives of friends, and the friends of relatives. We have fundamentalist Christian friends and polyamorous anarchist friends.

Incidentally, we also have enemies.

Mismanaging your social interactions can have terrible consequences. It can cost you your reputation, your spouse, your children, your inheritance, your job, your career, your freedom, and your sanity. If you make the wrong friend, and that friend turns into a stalker, it can cost you your life.

So:

Personally micromanaging your social network is fun.
The reason it’s fun is that humans are designed for it. This, in turn, is because it’s a vitally important personal skill that determines your income and your happiness. This is especially true in the 21st century.
Nobody is going to entrust their social network to a computer, not ever. Computers have not spent millions of years evolving their social skills. Computers are stupid. You have not solved the strong AI problem. If, by chance, you do invent a computer that is smarter than a human, nobody will trust that, either.
Build tools that empower people to make better decisions, not tools that try to make decisions for them.