Blackout Ireland

Entries for March 2008

March 1, 2008 at 1:52AM From a sitting room somewhere in west Dublin...

If you’re at the blog awards tomorrow, I’ll see you then. Come over and say hello, ok?

March 3, 2008 at 5:23PM Stupid hacks: nsping

Sometimes I need to check if there’s a nameserver running on some host. Here’s the stupid script I use to save myself from having to remember how.

dig @$1 -t a "" | \
    sed -n -e '/^;; Query time:/p' | \
    cut --delim=" " -f4,5

March 4, 2008 at 1:13AM My first crappy attempt at a poster for “Resign Mr. Ahern”

"Resign, Mr. Ahern" poster attempt

I’ve more ideas, and better ones, but this is the first one.

Update: For reference’s sake, it starts in Dublin at Parnell Square at 1pm and continues till 4pm.

March 4, 2008 at 2:36PM Idea: Doctests for Tcl

I was chatting with Laura over IM about testing in Tcl, and I mentioned one of my favourite features of Python, doctests, and a possible method for implementing them in Tcl.

The trick would be to define two new commands, #% and #>. Normally, these would act like regular comment commands, but when being ran in a testing environment, #% would take a command to be evaluated, the result of which would be saved and compared to the arguments passed to the next #>. Here’s a very simple example:

#% llength {a b c d e f g}
#> 7

March 4, 2008 at 3:55PM Handling of leap days in PHP and MySQL

Niall was wondering earlier about how our billing system here at work handles leap days. I wasn’t entirely sure because I wasn’t sure how exactly PHP and MySQL handle them and if they handle them the same way.

It turns out PHP and MySQL handle leap days completely differently. PHP will chose March 1st of the next year when incrementing a leap-day date, while MySQL will chose February 28th. Both can be valid ways of handling the special case, but they’re plenty of cases where you’d prefer one of them over the other. For instance, with domain renewals, it’s better to renew early rather than late because a domain might go into redemption or quarantine otherwise.

Here’s my checks. First, PHP:

echo date('c', strtotime('+1 year', strtotime('February 29th, 2008')));

And here’s MySQL:

mysql> select date_add('2008-02-29 00:00:00', interval 1 year);
| date_add('2008-02-29 00:00:00', interval 1 year) |
| 2009-02-28 00:00:00                              |
1 row in set (0.00 sec)

For the sake of reference, I tested MySQL 5.0.45 and PHP 5.2.3. YMMV.

March 12, 2008 at 7:14PM Scrum et al.

ABSTRACT: Scrum is an amazingly simple process that causes many, many changes when it is implemented. This seminar presents the basic framework of Scrum and some of the implementation issues associated with it.

March 18, 2008 at 1:27PM Messing with Standard ML and Moscow ML, part one: The core language

I was playing with Moscow ML because I’ve wanted to give Standard ML a bash for a while now, but I could never get SML/NJ to play nice for me back when I tried it first on Windows. That, and there was no documentation download.

So why Moscow ML and not SML/NJ, or Mlton, or one of the other proper Standard ML compilers? Well, even though Moscow ML is getting on a bit and uses bytecode compilation rather than native compilation, it’s simple to use, is well-documented, and, unlike Mlton, doesn’t require FreeBSD 7.0 and much as I’d like to, I haven’t upgraded yet.

It also helps that I’d already fetched it down ages ago, so the source was already in my distfiles folder. [smile]

It was dead easy to get something compiled and running. Here’s the “Hello, World” program:

(* helloworld.sml *)
val _ =
  print "Hello, world!\n";

(* and *) mark the beginning and end of comment blocks. They can be nested, so...

(* This (* is completely legal and *) makes commenting out code easy. *)

I made a really dumb mistake the first time I tried this. I’d been playing with the mosml console and forgot that in actual programs the results of expressions need to be assigned somewhere. So, my first program was like this:

(* brokenhello.sml *)
print "Hello, world!\n";

Which got me this:

% mosmlc -standalone -o hello brokenhello.sml
File "brokenhello.sml", line 2, characters 0-5:
! print "Hello, world!";
! ^^^^^
! Syntax error.

When I saw this, I stared at the screen trying to remember what I’d missed. I felt like a right dolt when I remembered.

I then rewrote the program to use the TextIO module:

(* helloworld-mod.sml *)
val _ =
  TextIO.output (TextIO.stdOut, "Hello, world!\n");

This showed that I could access modules just fine.

You’re probably wondering what val ??? = means. That’s declaration that the value of such-and-such a variable (given by ???) has the value of the expression that follows. In this case, the wildcard variable _ is being bound to the result of evaluating the “Hello, World” program.

Next I tried something a little more substantial: factorials.

(* fac1.sml *)
fun fac n = if n = 0 then 1 else n * fac (n - 1);

val _ = print (Int.toString (fac 5) ^ "\n");

Which wrote out 120, just as I’d expected. As you might guess, Standard ML’s if-expressions are just like a more readable version of C’s trinary operator.

A quick word on functions. Standard ML functions really only take one argument. To be able to take more than one, you need to either pass everything in an n-tuple or use curried function, though naturally because tuples are just another kind of value, you can mix and match both methods. The TextIO.output function above is an example of using a tuple to supply multiple arguments.

A curried functions is one that use individual one-argument functions to consume each argument. Curried functions are useful in that they allow one to partially apply functions and apply them in interesting ways. For instance, what if we wrapped TextIO.output as follows:

fun sayToStream str s = TextIO.output (str, s);

Here we’ve a two-argument curried function called sayToStream, that take a stream, str, and a string to output, s. Evaluating this function in the mosml REPL says the function has this type:

val sayToStream = fn : outstream -> string -> unit

The arrow (->) can be thought of as meaning ‘evaluates to’, so sayToStream is a function that takes an outstream and evaluates to a function that takes a string, which evaluates to unit, Standard ML’s rough equivalent of void in C-derived lanuages, but here, unit equates to a proper value rather than some notional one. You see, our above declaration of sayToStream is just a shorter, less noisy, way of saying:

val sayToStream = fn str => fn s => TextIO.output (str, s);

Where fn arg => expr defines a lambda function evaluating to expr. fn is pronounced ‘lambda’, I’m told, which would make sense if they’d used something that at least looked vaguely like a lambda such as a backslash like Haskell uses, but there you go.

We can partially evaluate sayToStream to create a function that writes its argument to standard output. Here’s what we’d get typing at the mosml REPL (- is the prompt, by the way, and > precedes the result of the preceding expression):

- val sayToStdOut = sayToStream TextIO.stdOut;
> val sayToStdOut = fn : string -> unit

So now we have sayToStdOut, a function that does just what we wanted. It’s pretty much equivalent to the toplevel print function.

Where this becomes really useful is when you want to pass the partially applied function to, say, a mapping function, or if you want to compose it with other functions. Here’s an example of the former, writing out the contents of a string array:

- List.map (sayToStream TextIO.stdOut) ["Each ", "word ", "is ", "an ", "element.\n"];
Each word is an element.
> val it = [(), (), (), (), ()] : unit list

An here’s an example of the latter, where we compose a function called shout and map it to the same array:

- val shout = (sayToStream TextIO.stdOut) o (String.map Char.toUpper);
> val shout = fn : string -> unit
- shout "hello\n";
> val it = () : unit
- List.map shout ["Each ", "word ", "is ", "an ", "element.\n"];
> val it = [(), (), (), (), ()] : unit list

o is the functional composition operator and glues two functions together. The operand order is backwards when compared with how the same operator works in maths. In maths, h = g o f would mean h(x) = f(g(x)), but in Standard ML, h = g o f means h x = g (f x), which is completely arseways. Oh, well...

Another example of the nifty things curried functions allow you to do would be O’Caml’s typesafe Printf.printf function. It takes a formatting string and returns functions that accept arguments of the correct types for each placeholder in the formatting string. The consequence of this is that the kind of exploits printf() and company are used for in C aren’t possible in O’Caml. It’s quite possible to do the same thing in Standard ML.

Keep in mind that functions are value too. That’s the reason I’ve used brackets where I have. I’ve used them where there’s an expression I want to evaluate before passing its result as an argument. fac n - 1 means something quite different from fac (n - 1).

Back to factorials. Now to try the same function, but this time using pattern matching instead:

(* fac2.sml *)
fun fac 0 = 1
  | fac n = n * fac (n - 1);

val _ = print (Int.toString (fac 5) ^ "\n");

Again, this gave me the same answer as my original fac function: 120.

Pattern matching is pretty useful. It can simplify code quite a bit by separating out the various cases of a function. Rather than having lots of conditional logic, we just make statements about what the results of evaluating the function are under different circumstances.

The pattern matching syntax above is shorthand for the following:

(* fac3.sml *)
fun fac n =
  case n of
    0 => 1
  | n => n * fac (n - 1);

Next up, I tried writing a function to join the elements of an array into a string. The function takes a function to convert each element to a string, a string to use a an element separator, and finally the list to join.

To avoid having to write any special purpose code, I decided to write a helper function that would take an extra parameter that would be prefixed onto the stringified list element. When the helper calls itself to cope with the list tail, it would then pass the separator argument as both the prefix and separator argument. When we’re initially calling the helper function, an empty string is passed in the prefix argument.

(* join1.sml *)
fun helper _ _ _ [] = ""
  | helper toString pre sep (h::t) = pre ^ (toString h) ^ (helper toString sep sep t);

fun join toString sep lst = helper toString "" sep lst;

A quick note on this: notice (h::t) in the helper function’s argument list. The :: operator is the list construction operator and in patterns can be used to decompose a list into a head element and a trailing list. As I haven’t mentioned it yet, ^ is the string concatenation operator.

Let’s try executing it in the mosml REPL:

- load "Int";
> val it = () : unit
- join Int.toString ", " [1, 2, 3, 4, 5];
> val it = "1, 2, 3, 4, 5" : string

We don’t really want the outside world to know about our helper functions. One way to hide them is to use a local pvtdecls in decls end block. This is particularly useful if the helpers are used by a number of different functions.

(* join2.sml *)
  fun helper _ _ _ [] = ""
    | helper toString pre sep (h::t) = pre ^ (toString h) ^ (helper toString sep sep t)
  fun join toString sep lst = helper toString "" sep lst

Now only join can see helper. However, as join is the only function that needs to about helper, we can declare it within join using a let decls in expr end block:

(* join3.sml *)
fun join toString sep lst =
  fun helper _ _ _ [] = ""
    | helper toString pre sep (h::t) = pre ^ (toString h) ^ (helper toString sep sep t)
  helper toString "" sep lst

Because functions declared within other functions are contained within the scope of their parent function, we don’t need to pass these values in, meaning we can simplify our helper function down like so:

(* join4.sml *)
fun join toString sep lst =
  fun helper _ [] = ""
    | helper pre (h::t) = pre ^ (toString h) ^ (helper sep t)
  helper "" lst

I could’ve approached the join function differently. Rather than using a helper function an prefixing the separator on, I could’ve treated the separator as a suffix and omitted appending the suffix in the case of an one-element list:

(* join5.sml *)
fun join _ _ [] = ""
  | join toString _ [h] = toString h
  | join toString sep (h::t) = (toString h) ^ sep ^ (join toString sep t);

But then we wouldn’t have learned as much. [smile]

[h], by the way, is a pattern that matches a single element list.

That’s enough for now. I’ll talk a bit about the type system later and about records, exceptions, references, the imperative side of the language, the module system, and all of that when I get the notion. However, if you’ve understood everything so far, you understand a fair bit of the core language.

In the meantime, you might want to read Mads Tofte’s Tips for Computer Scientists on Standard ML (PDF), which is quite readable and easy to understand, or Stephen Gilmore’s Programming in Standard ML 97: A Tutorial Introduction, which I found more difficult to follow in places, but covers everything in much more detail.

March 19, 2008 at 7:25PM Michael Bierut Talks Typography with ‘The Atlantic’


March 22, 2008 at 12:37AM On Karlin Lillington’s “The EU’s odd priorities”

Karlin Lillington recently wrote some inaccurate nonsense for the Irish Times on the EU’s approval of Google’s DoubleClick purchase. I posted a comment correcting some misinformation it contained. Here it is. Bits in italic are things I forgot when I posted the original comment.

Er, no Karlin. There’s worrying aspects to the merger, but I think you’re way off the mark: “Google’s Trojan Horse: Let the Free Ad Serving Begin”.

On IE vs. Safari, the big difference is that IE is hardwired into the underlying OS, whereas Safari isn’t. MacOS works just fine with or without Safari. The other big problem was with the way that MS’s products used unpublished Windows APIs to give them an advantage in the market over those of third parties. Not that Apple are completely clean in this regard.

And where did you get the idea that MS have been releasing code as open source? They’ve released very little as open source (the likes of WIX and WTL being minor exceptions). Are you sure you’re not thinking of their “Shared Source” Initiative? That’s the umbrella under which MS make the source selected libraries, products, or section thereof openly viewable, but not usable or forkable. For instance, take the release of the source for the .NET foundation libraries. The source was released purely as a reference. A project like Mono can’t use any of this code under any circumstances whatsoever. There’s nothing wrong with MS’s Shared Source Initiative other than its slightly deceptive name, and it’s to be lauded, but it’s most definitely not to be confused with Libre software.

A cookie is not a program--it does not execute. It’s a bit of information that the webserver sends to a browser that the browser then sends back unaltered.

As it happens, it’s quite easy to block most DoubleClick cookies, at least in browsers other than IE. For instance, there are Firefox plugins that specifically block advertising content from being fetched (e.g., Adblock). And even without them, most modern browsers allow cookies from certain domains to be blacklisted. This might be non-trivial as it requires you to know what a cookie really is and where to find the configuration panel, but it’s far from impossible.

March 27, 2008 at 5:22PM Des Bishop does “Jump Around” as Gaeilge

Surprisingly good, actually.


March 30, 2008 at 7:19PM “Sombody broke USE_GNOME!”, or Why .0 versions are a bitch

I’ve had my evenings for the last week ruined because of two little lines of code.

Yep, two lines.

Not my lines though, but I’d been wisely assuming that this was all my fault.

Some background: when I was up home in Sligo at easter, I updated my ports tree and upgraded my laptop to FreeBSD 7.0. Asides from needing to switch to the GENERIC kernel from my tweaked one that allowed me to mount large FAT32 formatted external HDDs before I could upgrade, it went off without a hitch. Before I left the house, I had portupgrade fetch down the distfiles for any ports that needed upgrading and figured that I could continue that part of the upgrade down in Carlow.

All the ports I’d installed while running FreeBSD 6.2 ran just find under FreeBSD 7.0, but I thought it was worth rebuilding everything so that next time I did an update, I wouldn’t have old 6.0 libraries cluttering up the system.

That was my mistake.

As I was later to discover, there was a problem lurking deep within the ports infrastructure, specifically the Mk/bsd.gnome.mk file. Here’s the fix, for those who can’t wait for the rest of the story:

--- bsd.gnome.mk.orig	2008-03-29 18:21:06.000000000 +0000
+++ bsd.gnome.mk	2008-03-29 18:21:36.000000000 +0000
@@ -712,6 +712,8 @@
+LIB_DEPENDS+=	${${component}_LIB_DEPENDS}
+RUN_DEPENDS+=	${${component}_RUN_DEPENDS}

 .if !defined(WITHOUT_HACK)

Those in the know should be able to figure out what happened from that. That first extra line is the difference between a 17MB INDEX-7 file and a 24MB INDEX-7 file, and the second one adds on another 1MB again. That’s quite a bit of missing data. It’s critical that after applying that patch, you do the following:

$ cd /usr/ports
$ sudo rm -rf INDEX-*
$ sudo make index
$ sudo portsdb -u
$ sudo pkgdb -aF

Otherwise the port tools won’t have the right dependencies, and will start doing stupid things like building the same port twice, regardless of whether it’s installed or not.

I ran a full update with portupgrade and discovered that when I’d ran previously, it’d only done a shallow distfile fetch, only fetching the distfiles of whichever ports had been updated, which didn’t catch cases where those ports might now be depending on other ports now too. I was annoyed, but learned my lesson not to depend on portupgrade anymore to fetch updates and to stick with portmaster for that from now on. The house I live in in Carlow has no internet connectivity--not by my choice--so I collected a list of the missing distfiles and pulled them down in work.

My second attempt at rebuilding my ports failed, but in quite subtle ways. The first inkling I got that something was wrong was that any GTK apps written in Python would hang when loading the GTK libraries. When I rebooted, dbus was screwed, and nautilus would just hang in the background, completely undead.

In retrospect, the thing I did that aggravated the situation enough to discover what the true problem was was that I cleared out the contents of /usr/local/lib/compat/pkg. I was rebuilding everything and not keeping any shared libraries, so I figured it’d be safe. Yes, I know you’re not supposed to do this, but it’s my machine and if I break it, it hurts nobody but me, and it’s not as if I didn’t understand the possible consequences.

I had no idea what was wrong and had no way to diagnose it at the time, so I decided to wipe every single port and clear any leftover junk from /usr/local that I didn’t want to keep. There was a fair bit of it, more than there ought to have been.

I decided to do a minimal install. After installing the basics, I went to reinstalling Vim. My copy is set up to depend on GTK+ 2, so I’d expected it to pull in and build it. However, it wasn’t and it and dbug-glib exploded in my face saying that it couldn’t find GLib. I noted what’d happened and configured it to build without any X11 support whatsoever, and it built fine.

Seeing as GNOME/GTK/GLib seemed to be at the root of my difficulties, I figured I’d only install anything that could have its dependencies on them removed, or which didn’t depend on them in the first place. That meant no HAL or DBus, or anything that depended on them. I configured my X server to get rid of its HAL dependency and installed the only usable window manager I had that didn’t depend on them to some degree: dwm.

Now that I’d something usable and I’d set up xterm to use a decent dark colour scheme, I set to work figuring out why everything was broken. My suspicions fell upon the USE_GNOME knob, which seemed to be the focus for all the breakage. Once I’d figured out enough of the workings behind the ports system, it lead me to Mk/bsd.gnome.mk, the make file that USE_GNOME controls. I figured out how the values in USE_GNOME relate to various dependencies, and traced through the building of a typical small port from scratch, dbus-glib. That’d when I realised that it was ignoring any library dependencies, so I patched it so that it’d collect them properly. That’s the first patch line you’ll see. That fixed dbus-glib, so I though I’d found the problem. I was stoked, so I decided to try to install something more directly useful: Meld.

I need to compare and patch files and directories of files quite a bit, so Meld is invaluable. It’s GNOME’s killer app as far as I’m concerned and about the only reason I’ve the gnome2-lite metaport installed at all. Tomboy comes a close second, but that’s mainly because I wrote a similar tool called Orpheus with wxWidgets some years back for a project, so once I’d got past the fact it depended on Mono, I grudgingly allowed it on my desktop.

The build went fine until it came to installing gnome-menus, where it crapped out complaining that /usr/local/etc/mtree/BSD.gnome.dist couldn’t be found. I scratched my head, sighed, and opened the makefile up again.

That’s where the second line comes from. It wasn’t pulling in runtime dependencies, and in this case it meant the gnomehier member of USE_GNOME as being skipped.

As I write this, everything appears to be building ok. There’s a couple of ports, specifically gvfs and the FuseFS port it depends on, I need to fetch the distfiles for which were skipped when I fetched the previously because they weren’t showing up, but that’s no biggie.

I’m sure I’m far from the only person who was affected by this. It’s amazing that two innocuous lines would have such a major effect on the system. My guess as to what happened is that somebody was editing the file with Vim, hit the d key close to the code, and then hit the up or down cursor key, inadvertently deleting those two lines. It’s happened to me in the past, so it’s plausible.

I found out the problem had been patched in the meantime, so there’s no need for me to go submitting a bug report myself. Still, I thought the experience was worth documenting, and I definitely learned a lot about the inner working of the ports system in the process!