Entries for January 2008
January 1, 2008 at 5:03AM Happy 2008!
Hey, everyone! Happy 2008!
Sorry to everyone who texted me and who I didn’t text back. The battery in my phone was almost dead, so I turned it off earlier. I’m just about to plug it back in, so to anybody who reads this before I text them, a happy new year to you!
January 2, 2008 at 6:00AM Death Cab for Cutie: The New Year
January 7, 2008 at 4:04PM Whenever I’m pulling data from a database in PHP...
...I keep wishing this stupid language supported lazy lists or would let me implement them in a non-sucky way.
January 8, 2008 at 11:37AM Is your Rails app giving the error “uninitialized constant ApplicationController::AuthenticatedSystem”?
If so, then you’re having problems with the acts_as_authenticated plugin. One of our customers here at work was having problems with this.
The first thing to do is to check if the plugin’s been installed. You can do this by checking the lib directory to make sure it contains the files authenticated_system.rb and authenticated_test_helper.rb, and the vendor directory to make sure it contains the acts_as_authenticated exists within it.
If they don’t, execute the following within your application’s root directory:
$ script/plugin install http://svn.techno-weenie.net/projects/plugins/acts_as_authenticated $ script/generate authenticated user account
And answer ‘N’ to all the questions the second command asks.
You should make sure that you upload the lib and vendor folders with your app.
January 14, 2008 at 3:32PM On the direct election of the Taoiseach
Bernie raises the idea of a directly elected Taoiseach. I posted the following comment (which appears to have ended up in /dev/null somehow) explaining why I don’t think this is either practical or desirable.
I have to say, while I agree with the idea of making mayors directly electable, the idea of making the office of Taoiseach directly electable seems somewhat, well, unworkable. The job of the US president is somewhat different from that of a head of state in a parliamentary democracy. While in a congressional democracy the head of the executive branch doesn’t need to have a power block in the upper and lower houses to do their job (though it helps), the head of government in a parliamentary democracy does. This is because the head of the executive branch in a congressional democracy combines most of the powers of the head of state with those of the head of government, meaning they have powers of veto, and so on, which heads of governments in parliamentary democracies don’t have.
Still, I’d say the election of the Taoiseach is at least a little more direct than that of the US President. They’re directly elected by their constituents, and then confirmed in the office by local delegates that have been directly elected to the Oireachtas.
[Aside: Yes, I know the Taoiseach is appointed by the President according to the constitution, but in reality, the President just confirms the selection of the Oireachtas.]
January 16, 2008 at 10:31PM Building an RPM package: a guide for the unwilling
Some day, if you’re a software developer and develop software than runs on Linux, you will end up, whether you like it or not, having to build an RPM file.
I’ve just spent an absolutely inordinate amount of time working out how to do this. It wasn’t fun, and I don’t think I’d like to ever have to do it again, so to save myself or some other poor sod similar pain, I thought I’d write up what I did here.
First, a few assumptions. Unlike most explanations of how to do this, I’m not going to be building from an tarball, so I’m not going to be covering %prep and all that just yet. Also, I’m building the RPM from a tag in a subversion repository.
Before we do anything else, we need to set up our system so we can build the RPM without being root. To do that, open the file ~/.rpmmacros in your editor of choice and enter the following:
%_topdir %(echo ~/rpmbuild)
This tells RPM to use ~/rpmbuild as its working directory. RPM requires its working directory to have a certain format, so type the following to build it:
$ cd rpmbuild; mkdir -p BUILD RPMS/i386 SOURCES SPECS SRPMS
BUILD is where the work is done, RPMS is where the finished RPM is dumped, SOURCES is where you drop tarballs if you’re building from one, SPECS is where you put your spec files, and SRPMS is where finished source RPMs are dumped.
Let’s get to writing the spec file, which I’m going to call libexample.spec. The first section of the RPM contains metadata, so let’s write up some:
Summary: My example library.
Name: libexample
Version: 0.1
Release: 1
Group: Development/Libraries
Packager: J.R. Hacker <me@example.com>
License: BSD
BuildRoot: %{_tmppath}/%{name}-root
Most of these should be pretty self-explanatory: Summary is a short human-readable summary of what the package is for; Name is its actual name; Version is the version of the library you’re building; Release is the release number of that version; Group specifies the kind of package it is, Packager is your name and contact details, and License specifies the licensing details for the package.
The last one, BuildRoot needs a little more explanation. When you’re building an RPM, you’re building a directory tree that will be extracted to the filesystem root of your machine when the package is installed. You don’t want to pollute the directory of your machine with crap, so you need somewhere you can cleanly build it. In the case above, let’s say that the value of the %{_tmppath} macro is /tmp. This would mean that BuildRoot would have the value /tmp/libexample-root. Similarly to how the Name header is referenced here as %{name}, we’ll be referencing BuildRoot as %{buildroot}.
Next comes the %description section, which contains a longer description of the package than given in the summary. Following this is normally the %prep section, which we’re going to ignore here and not use because we’re not fiddling with tarballs. Then we have the %build section. This is where the first bit of real work happens, but first a quick note on the directory structure we’re building.
We’re exporting a single directory from SVN called src. This contains a makefile that build a library using GNU libtool into a folder within it called src/.libs. The project doesn’t use the autotools, so the spec file does all the installation work after it’s built. I’m using this structure because I want to demonstrate explicitly how the contents of %{buildroot} is built.
Here’s what our %build section looks like:
%build
rm -rf src %{buildroot}
svn export https://svn.example.com/projects/libexample/tags/0.1/src/
(cd src; make)
When this section is being executed, RPM automatically changed to the %{_topdir}/BUILD directory. It’s here that we do our work. The second line cleans up our environment so that we can build it. The third exports the project’s src directory to a directory of the same name. The third then switches to this within a subshell and executes the makefile within it. Now it’s time to for the %install section, which is where we build the directory tree that’s going to end up in the RPM.
%install
# Create the directory hierarchy for building our RPM.
mkdir -p %{buildroot}%{_libdir} %{buildroot}%{_includedir}
# Copy whichever files that RPM will contain into it.
install --strip --mode=0755 src/.libs/%{name}.so.0 src/.libs/%{name}.a %{buildroot}%{_libdir}
install --mode=0755 src/%{name}.la %{buildroot}%{_libdir}
sed -i -e s/^installed=no$/installed=yes/ %{buildroot}%{_libdir}/*.la
install --mode=0644 src/*.h %{buildroot}%{_includedir}
(cd %{buildroot}%{_libdir}; ln -sf %{name}.so.0 %{name}.so)
# Build a manifest of the RPM's directory hierarchy.
echo "%%defattr(-, root, root)" >MANIFEST
(cd %{buildroot}; find . -type f -or -type l | sed -e s/^.// -e /^$/d) >>MANIFEST
The %{_libdir} and %{_includedir} macros specify the directories where your system installs libraries and header files. On a typical Red Hat derived system, these are going to have the values /usr/lib and /usr/include, though this isn’t necessarily what they’re going to be.
Within %{buildroot} we need to create the directory tree that’s going to end up in the RPM. We then need to copy anything that will end up in %{_libdir} after installation into the correct place under %{buildroot}, and to the same with any headers too. We use install to make sure that permissions and ownership for all the files are correct. The fourth last line makes a symbolic link pointing whatever linkers end up linking to our library to the version of it we’re installing. The second last and last lines build a manifest of the files within %{buildroot}. We’ll be using that later to save ourselves some work. Now that our root directory hierarchy’s been built, we can get on with the odds and ends.
Your RPM might need to have commands executed before or after installation and deinstallation. There are hooks for these, but I’m only going to cover two of them: %post, which is ran post-installation, and %postun, which is ran post-deinstallation. We’re installing libraries, which means that after they’re added and after they’re removed, we need to call ldconfig so that the dynamic linker knows about them.
%post /sbin/ldconfig %postun /sbin/ldconfig
After the RPM is built, we need to clean up our environment again. That’s what the %clean section is for:
%clean
rm -rf src %{buildroot} MANIFEST
The last section is the file manifest, which is contained within the %files section. Here you specify what files the RPM will contain and where they reside in the filesystem. Normally, it’d look something like this:
%files
%defattr(-, root, root)
%{_libdir}/%{name}.so*
%{_libdir}/%{name}.la
%{_libdir}/%{name}.a
%{_includedir}/example.h
The %defattr macro specifies the permissions and ownership of the files that follow it and takes three arguments: the permissions, the owner, and the group. If any of these don’t matter, use a hyphen. This way, the attributes already specified on the files will be used instead.
However, we were clever earlier on and built a manifest of the contents of %{buildroot}, so we can just tell RPM to use that, which means the %files section is reduced to this:
%files -f MANIFEST
And that’s it! You should now have an RPM. Here’s our finished spec file:
Summary: My example library.
Name: libexample
Version: 0.1
Release: 1
Group: Development/Libraries
Packager: J.R. Hacker <me@example.com>
License: BSD
BuildRoot: %{_tmppath}/%{name}-root
%description
A library that acts as an example of how to build an RPM.
%build
rm -rf src %{buildroot}
svn export https://svn.example.com/projects/libexample/tags/0.1/src/
(cd src; make)
%install
# Create the directory hierarchy for building our RPM.
mkdir -p %{buildroot}%{_libdir} %{buildroot}%{_includedir}
# Copy whichever files that RPM will contain into it.
install --strip --mode=0755 src/.libs/%{name}.so.0 src/.libs/%{name}.a %{buildroot}%{_libdir}
install --mode=0755 src/%{name}.la %{buildroot}%{_libdir}
sed -i -e s/^installed=no$/installed=yes/ %{buildroot}%{_libdir}/*.la
install --mode=0644 src/*.h %{buildroot}%{_includedir}
(cd %{buildroot}%{_libdir}; ln -sf %{name}.so.0 %{name}.so)
# Build a manifest of the RPM's directory hierarchy.
echo "%%defattr(-, root, root)" >MANIFEST
(cd %{buildroot}; find . -type f -or -type l | sed -e s/^.// -e /^$/d) >>MANIFEST
%post
/sbin/ldconfig
%postun
/sbin/ldconfig
%clean
rm -rf src %{buildroot} MANIFEST
%files -f MANIFEST
There’s much more to creating RPMs to what I’ve detailed here, but this should at least give you a good start.
Nota bene: This was all written pretty much trail-of-though, so there may be errors and typos. If you find any before I do, post up a comment.
References
I didn’t really find anything particularly clear and informative online about building RPMs, except the Fedora Project’s RPM Guide, the HOW-TO linked to above, and this oldish tutorial. I also found this one after the fact, and it looks pretty decent, if terse.
January 18, 2008 at 9:41PM The Cube
A rather odd, hour-long teleplay by Jim Henson.
This philosophical teleplay covers many themes such as, the nature of reality, individual versus scientific perception, self-reference, man’s relation to others and society, insanity, social projections, and race relations. It originally aired on NBC’s weekly anthology television show NBC Experiment in Television February 23, 1969. The production was produced and directed by Jim Henson, and was one of several experiments with the live-action film medium which he conducted in the 1960s, before focusing entirely on the Muppets and other puppet works. The screenplay was co-written by long-time Muppet writer Jerry Juhl (who also appears in a cameo).
[Wikipedia page for ‘The Cube’]
January 21, 2008 at 1:34PM Port management tools from my ~/bin directory
Here’s two small tools I have in my ~/bin directory. The first one, check-deps, checks the upwards and downwards dependencies for each package are consistent:
#!/usr/bin/env python
from __future__ import with_statement
import os
PKG_DB = '/var/db/pkg'
def each_dep(lines):
"""Extracts the package dependencies from the iterable."""
return (line[8:-1] for line in lines if line.startswith('@pkgdep '))
# Figure out from the dependencies which packages are required by which.
reqs = {}
for pkg in os.listdir(PKG_DB):
path = os.path.join(PKG_DB, pkg, "+CONTENTS")
if os.path.isfile(path):
with open(path) as f:
for dep in each_dep(f):
if dep not in reqs:
reqs[dep] = set()
reqs[dep].add(pkg)
# Check that the currently recorded ones are valid.
for pkg in reqs.keys():
path = os.path.join(PKG_DB, pkg, "+REQUIRED_BY")
if os.path.isfile(path):
with open(path) as f:
for line in f:
line = line[:-1]
if line in reqs[pkg]:
reqs[pkg].remove(line)
else:
print "===>\tErroneous dep '%s' in '%s'" & (line, pkg)
if len(reqs[pkg]) == 0:
del reqs[pkg]
else:
print "===>\t%s does not record:\n%s" % (pkg, "\n".join(reqs[pkg]))
if len(reqs) == 0:
print "All clean!"
The second one, check-packing, checks that the file manifest for each port is correct and that all the files are intact by comparing them against their recorded MD5 hashes:
#!/usr/bin/env python
from __future__ import with_statement
import os
import hashlib
import sys
PKG_DB = '/var/db/pkg'
def each_file(f):
"""
Generates a list of tuples, the first element being an absolute file
path, and the second being its recorded hash, from a package manifest.
"""
cwd = ''
file = ''
try:
for line in f:
if line[0] != '@':
file = os.path.join(cwd, line.strip())
elif line.startswith('@cwd '):
new_cwd = line.split(' ', 2)[1].strip()
# This check is a hack for dealing with broken manifests.
# mysql-client-5.0, nss, pth, portupgrade and zsh are
# examples.
if new_cwd != '':
cwd = new_cwd
elif file != '' and line.startswith('@comment MD5:'):
digest = line.split(':', 2)[1].strip()
yield (file, digest)
file = ''
finally:
pass
def hash_matches(path, original_hash):
"""Checks if the given file matches a hash."""
with open(path) as f:
new_hash = hashlib.md5(f.read()).hexdigest()
return new_hash == original_hash
for pkg in sorted(os.listdir(PKG_DB)):
path = os.path.join(PKG_DB, pkg, "+CONTENTS")
if os.path.isfile(path):
print "===>\tChecking %s" % (pkg,)
with open(path) as f:
for (file_path, hash) in each_file(f):
try:
if not os.path.exists(file_path):
if os.path.islink(file_path):
print "Missing: %s -> %s" % (file_path, os.path.realpath(file_path))
else:
print "Missing: %s" % (file_path,)
elif not os.path.islink(file_path) and not hash_matches(file_path, hash):
print "Altered: %s" % (file_path,)
except KeyboardInterrupt:
raise
except:
print "Unchecked: %s" % (file_path,)
They might be useful to someone.
Update (Feb 12th): Removed some stupid inefficiencies in how the files are hashed.
January 22, 2008 at 10:25AM Audiosurf - Chemical Brothers’ Star Guitar
Audiosurf is insane:
Here’s the original Michel Gondry video:
January 25, 2008 at 11:52PM Merging .iso files on Debian, Ubuntu, and most likely Linux in general
I hardly ever have to do this, so I can never remember precisely how to do it. Hopfully by noting it here, I won’t forget again.
Specifically, I want to do this so that I can create a single DVD image from the FreeBSD 6.3 CD images I have. Here’s more-or-less how it goes:
mount -t iso9660 -o loop disc1-mountpoint mount -t iso9660 -o loop disc2-mountpoint mkisofs -l -J -R -o dvd.iso disc1-mountpoint disc2-mountpoint
Don’t forget to unmount those mountpoints afterwards.
Here’s a shell script to do the work. It assumes it’s in a directory with a subdirectory called isos, which contains the source .iso images. It has no error handling, so the usual caveats apply:
#!/bin/sh
rm -rf mnt contents
mkdir mnt contents
for i in isos/*.iso; do
mount -t iso9660 -o loop $i mnt
cp -av mnt/* contents
umount mnt
done
# This bit is specific to merging FreeBSD .isos.
sed -i -e 's/^CD_VOLUME = .$/CD_VOLUME = 1/' contents/cdrom.inf
sed -i -e 's/|.$/|1/' contents/packages/INDEX
mkisofs -l -J -R -o dvd.iso contents
rm -rf mnt contents
And here’s how you merge several .iso files into a single one on FreeBSD, and how to do it for FreeBSD .iso images.
