Inklings: a tumblelog

Built on top of EPP, which is more convenient for me anyway.

Volume I opens in 221B Dodger Street, with Sherlockcat and Dr Watson preparing for dinner, when Ms Lawson, his landlady and sometime housekeeper, announces that Inspector Lestrade has asked for his assistance concering the appearance of poopy pawprints around London.

As Sherlockcat and Watson investigate, the evidence begins to point the blame at Sherlockcat himeself. However, Sherlockcat discovers that the actual perpetrator is none other than Professor Maowriarty!

With no solid evidence of Maowriarty’s guilt and the police closing in on him, Sherlockcat is forced to flee until he can find the evidence to clear his name.

Volume II opens with Dr Watson, who has been attempting to track down Sherlockcat ever since he absconded, discovering Sherlockcat in a catnip den in Shanghai. Fearing the worst for his friend, he attempts to intervene, but Sherlockcat convinces him that he is, in fact, only in the den as part of his investigation of Maowriarty, the actual cat in the mirror in the first book.

Sherlockcat discovers that the events of the first book were simply a rouse to make sure that Sherlockcat, as the only intellect Maowriarty considered his equal, was discredited and thus could not intervene in his true master plan: the cornering of the world yarn trade.

After barely escaping Shanghai with their lives, Sherlockcat and Watson track Maowriarty to Austria, where Sherlockcat confronts Maowriarty in a castle in the Austrian Alps. In the ensuing fight, both Sherlockcat and Maowriarty tumble over a balcony in to a raging waterfall, falling to their doom.

Or did they? Has Maowriarty been defeated? Will Sherlockcat return? Find out in the next book: “The Pups of the Baskervilles”.

The GeoNames geographical database covers all countries and contains over eight million placenames that are available for download free of charge.

Commercial access costs are pretty reasonable too.

Let me sing you the song of my contributions.

I haven’t listened to what mine sounds like, but it’s here.

Since I’m doing a tab clearout of my browser in work, I’d might as well note the plugin’s I’m using:

Work (Debian 7.0)

Firefox/Iceweasel 10.0.12

• Bookmark Current Tab Set
• Bookmark All - not sure why both this and the above are installed, so must see which I’m actually using. (Removed, 2013-04-02)
• Duplicate Bookmark Detector
• British English Dictionary
• Download Statusbar
• Firebug
• Flashblock - ads are one thing, but Flash ads are anathema.
• Linky (Removed, 2013-04-02)
• Live HTTP Headers - has functionality that ought to be in Firebug, but isn’t.
• MemChaser - haven’t had need of this for quite some time, so may get rid of it. (Removed, 2013-04-02)
• pwgen - for when I’m too lazy to open up a terminal. (Removed, 2013-04-02)
• Saved Password Editor - makes certain applications I use at work much less of a pain to deal with.
• Web Developer Toolbar - given how little web development I do these days (it’s almost completely backend stuff), you’d think this’d be nearly useless to me, but it’s sometimes bypassing nonsense that some sites inflict on their readers.

Wireframing tool.

[It’s just occurred to me that when I’m coding up this site’s replacement, I should really include something for tagging.]

Database migrations for SQLAlchemy.

Pathological HTTP server and client designed for fuzz testing HTTP clients and servers.

Mobile-friendly CSS framework. Source is on Github.

Simple password manager using gpg and ordinary unix directories.

Repo is here.

Yesterday I set up Review Board at work to see if it might be a better way to conduct code reviews than RhodeCode, which has code review functionality, but it’s never worked particularly well for us.

Owing problems we’ve had in the past[^3], we rarely use mod_wsgi these days and prefer run our applications as standalone daemons managed by Supervisor that listen on the loopback interface, with Apache[^1] acting as a reverse proxy using mod_proxy.

First thing I did was create a virtual environment for the application to run in:

# mkdir -p /opt/reviewboard
# cd /opt/reviewboard
# virtualenv --no-site-packages env

The server in question is running Debian Squeeze, which makes it necessary to use the --no-site-packages flag with virtualenv.

Review Board requires easy_install, so pip is a no-go:

# env/bin/easy_install ReviewBoard

It’s advisable to have a memcached instance running, so that was installed from APT.

As we authenticate off of a central LDAP server, I installed the python-ldap from PyPI into the virtual environment with pip:

# env/bin/pip install python-ldap

As we’ll need a WSGI server to run it under, I chose to use waitress. As of waitress 0.8.4, waitress comes with a command line runner I contributed called waitress-serve[^2]. It makes starting instances of waitress on the command line easier:

# env/bin/pip install waitress

With that done, it’s time to generate the site with the rb-site command:

./env/bin/rb-site install site

After asking a few questions, that will create a directory called /opt/reviewboard/site to contain the site assets.

Out of the box, reviewboard doesn’t have a way to run standalone. To get around that, I had it generate a mod_wsgi site, given that was the closest option, and took a look at the .wsgi file generated. It looked like this:

import os
import sys

os.environ['DJANGO_SETTINGS_MODULE'] = "reviewboard.settings"
os.environ['PYTHON_EGG_CACHE'] = "/opt/reviewboard/site/tmp/egg_cache"
os.environ['HOME'] = "/opt/reviewboard/site/data"
os.environ['PYTHONPATH'] = '/opt/reviewboard/site/conf'

sys.path = ['/opt/reviewboard/site/conf'] + sys.path

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

As we’ll be running this as a standalone server to be managed by Supervisor, I used this as the basis of the Supervisor configuration block for the application:

[program:reviewboard]
command=/opt/reviewboard/env/bin/waitress-serve
  --expose-tracebacks
  --url-scheme=https
  --host=127.0.0.1
  --port=8005
  --call
  django.core.wsgi:get_wsgi_application
user=nobody
stdout_logfile=/opt/reviewboard/site/logs/%(program_name)s.out.log
stderr_logfile=/opt/reviewboard/site/logs/%(program_name)s.err.log
environment=
  DJANGO_SETTINGS_MODULE="reviewboard.settings",
  PYTHON_EGG_CACHE="/opt/reviewboard/site/tmp/egg_cache",
  HOME="/opt/reviewboard/site/data",
  PYTHONPATH="/opt/reviewboard/site/conf"

Normally, waitress-serve expects a WSGI application object to be specified, but as none is created by default in a Django application, I used its --call flag so that a callable could be specified to call to get an application object.

With that set up, it’s time to set up Apache:

<VirtualHost *:80>
    ServerName reviewboard.example.com

    RewriteEngine on
    RewriteCond %{HTTPS} !^on$ [NC]
    RewriteRule . https://%{HTTP_HOST}%{REQUEST_URI} [L]
</VirtualHost>

<VirtualHost *:443>
    ServerName reviewboard.example.com

    DocumentRoot /opt/reviewboard/site/htdocs
    ProxyPass /static !
    ProxyPass /media !
    ProxyPass /errordocs !

    ProxyPreserveHost On
    ProxyPass / http://127.0.0.1:8005/
    ProxyPassReverse / http://127.0.0.1:8005/
    SetEnvIf X-Url-Scheme https HTTPS=1
    RequestHeader Set X-Forwarded-Proto https

    SSLEngine On
    SSLCertificateFile /path/to/certificate.crt
    SSLCertificateKeyFile /path/to/certificate.key
</VirtualHost>

The ProxyPass directives just following DocumentRoot ensure that any static content is handled by Apache. The block that follows that passes any other traffic through to the server we’ve bound to the loopback interface.

With all that in place, tell Supervisor and Apache to reload their configuration:

# supervisorctl update
# service apache2 graceful

And bingo! It should be working!

Caveat regarding LDAP authentication

When I set up LDAP authentication, I noticed it wasn’t working and there were some bizarre errors in the log:

WARNING:root:LDAP error: {'info': '(unknown error code)', 'desc': "Can't contact LDAP server"}

After some hunting and experimenting, I discovered that the problem was that python-ldap was doing certificate checking. To get around that, I opened up the reviewboard.accounts.backends module, found the LDAPBackend class, and in the authenticate() function, inserted the following statement following the import ldap statement:

ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_ALLOW)

And it was able to authenticate off of LDAP.

[^3]: YMMV: mod_wsgi is still a solid piece of software, but it just didn’t suit our deployment scenarios. [^1]: We may switch to Nginx for this for lower overhead, but it’s not a priority. [^2]: It doesn’t currently support WSGI middleware though. I didn’t feel strongly about adding support for middleware at the time, but I may change my mind in the future. It wouldn’t be a difficult thing to add.

2013-05-27: Updated for waitress 0.8.4: the waitress-serve package is no longer required as it’s been merged into waitress proper.