RewriteBase in .htaccess
I needed to add RewriteBase / to the top of the .htaccess file to get the nice looking urls to work. It seems this is a general problem on many platforms, since RewriteBase / isn't always specified in default apache configs on a lot of platforms, including Apple OSX. This can be fixed with a patch to include what you added in the general case, and more installs will work out the box, without breaking existing ones that do work. I'd suggest the devs patch this in permanently.