Xen: Running a Sid DomU with PyGrub on a Squeeze Dom0 //at 03:07 //by abe

I’m running one Debian Sid and one Jessie (Testing) Xen guest domain on a Debian Squeeze (Oldstable) Xen 4.0 running host server.

Recently I had to reboot one these virtual machines after more than a year of uptime. But the new 3.14 kernel from Debian Experimental didn’t boot. Neither did 3.13 from Debian Unstable. Nor did any other kernel image newer then the 3.5-trunk (from Debian Experimental back than) work.

Everytime pygrub bailed out with this error message:

Error: (2, 'Invalid kernel', 'xc_dom_find_loader: no loader found\n')

(Yes, the parentheses and the “\n” were part of the error message.)

After some searching on the web I found hints that this message may be caused by an unsupported compression type in the kernel image.

And indeed, if I unpack the “vmlinuz” with the extract-vmlinux tool which is part of Linux’ source code (but not yet part of any binary package in Debian), and use the extract file in grub’s menu.lst (which is then read by pygrub) instead, the DomU boots Linux kernel 3.14 again, even on a Squeeze-running Dom0.

Tools for CLI Road Warriors: Remote Shells //at 19:44 //by abe

Most of my private online life happens on netbooks and besides the web browser, SSH is my most used program — especially on netbooks. Accordingly I also have hosts on the net to which I connect via SSH. My most used program there is GNU Screen.

So yes, for things like e-mail, IRC, and Jabber I connect to a running screen session on some host with a permanent internet connection. On those hosts there is usually one GNU Screen instance running permanently with either mutt or irssi (which is also my Jabber client via a Bitlbee gateway).

But there are some other less well-known tools which I regard as useful in such a setup. The following two tools can both be seen as SSH for special occassions.

autossh

I already blogged about autossh, even twice, so I’ll just recap the most important features here:

autossh is a wrapper around SSH which regularily checks via two tunnels connect to each other on the remote side if the connection is still alive, and if not, it kills the ssh and starts a new one with the same parameters (i.e. tunnels, port forwardings, commands to call, etc.).

It’s quite obvious that this is perfect to be combined with screen’s -R and -d options.

I use autossh so often that I even adopted its Debian package.

mosh

Since last week there’s a new kid in town^WDebian Unstable: mosh targets the same problems as autossh (unreliable networks, roaming, suspending the computer, etc.) just with a completely different approach which partially even obsoletes the usage of GNU Screen or tmux:

While mosh uses plain SSH for authentication, authorization and key exchange the final connection is an AES-128 encrypted UDP connection on a random port and is independent of the client’s IP address.

This allows mosh to have the following advantages: The connection stays even if you’re switching networks or suspending your netbook. So if you’re just running a single text-mode application you don’t even need GNU Screen or tmux. (You still do if you want the terminal multiplexing feature of GNU Screen or tmux.)

Another nice feature, especially on unreliable WLAN connections or laggy GSM or UMTS connections is mosh’s output prediction based on its input (i.e. what is typed). Per line it tries to guess which server reaction a key press would cause and if it detects a lagging connection, it shows the predicted result underlined until it gets the real result from the server. This eases writing mails in a remote mutt or chatting in a remote irssi, especially if you noticed that you made a typo, but can’t remember how many backspaces you would have to type to fix it.

Mosh needs to be installed on both, client and server, but the server is only activated via SSH, so it has no port open unless a connection is started. And despite that (in Debian) mosh is currently just available in Unstable, the package builds fine on Squeeze, too. There’s also an PPA for Ubuntu and of course you can also get the source code, e.g. as git checkout from GitHub.

mosh is still under heavy development and new features and bug fixes get added nearly every day.

Thanks to Christine Spang for sponsoring and mentoring Keith’s mosh package in Debian.

Update: I gave a lightning talk about Mosh and AutoSSH in German at Easterhegg 2012. The slides are available online.

Tools for CLI Road Warriors: Tunnels //at 19:49 //by abe

Sometime the network you’re connected to is either untrusted (e.g. wireless) or castrated in some way. In both cases you want a tunnel to your trusted home base.

Following I’ll show you three completely different tunneling tools which may helpful while travelling.

sshuttle

sshuttle is a tool somewhere in between of automatic port forward and VPN. It tunnels arbitrary TCP connections and DNS through an SSH tunnel without requiring root access on the remote end of the SSH connection.

So it’s perfect for redirecting most of your traffic through an SSH tunnel to your favourite SSH server, e.g. to ensure your local privacy when you are online via a public, unencrypted WLAN (i.e. easy to sniff for everyone).

It runs on Linux and MacOS X and only needs a Python interpreter on the remote side. Requires root access (usually via sudo) on the client side, though.

It’s currently available at least in Debian Unstable and Testing (Wheezy) as well as in Ubuntu since 11.04 Natty.

Miredo

Miredo is an free and open-source implementation of Microsoft’s NAT-traversing Teredo IPv6 tunneling protocol for at least Linux, FreeBSD, NetBSD and MacOS X.

Miredo includes not only a Teredo client but also a Teredo server implementation. The developer of Miredo also runs a public Miredo server, so you don’t even need to install a server somewhere. If you run Debian or Ubuntu you just need to do apt-get install miredo as root and you have IPv6 connectivity. It’s that easy.

So it’s perfect to get a dynamic IPv6 tunnel for your laptop or mobile phone independently where you are and without the need to register any IPv6 tunnel or configure the Miredo client.

I usually use Miredo on my netbooks to be able to access my boxes at home (which are behind an IPv4 NAT router which is also an SixXS IPv6 tunnel endpoint) from whereever I am.

iodine

iodine is likely the most undermining tool in this set. It tunnels IPv4 over DNS, allowing you to make arbitrary network connections if you are on a network where nothing but DNS requests is allowed (i.e. only DNS packets reach the internet).

This is often the case on wireless LANs with landing page. They redirect all web traffic to the landing page. But the network’s routers try to avoid poisoning the client’s DNS cache with different DNS replies as they would get after the user is logged in. So DNS packets usually pass even the local network’s DNS servers unchanged, just TCP and other UDP packets are redirected until logging in.

With an iodine tunnel, it is possible get a network connection to the outside on such a network anyway. On startup iodine tries to automatically find the best parameters (MTU, request type, etc.) for the current environmenent. However that may fail if any DNS server in between imposes DNS request rate limits.

To be able to start such a tunnel you need to set up an iodine daemon somewhere on the internet. Choose a server which is not already a DNS server.

iodine is available in many distributions, e.g. in Debian and in Ubuntu.

Illegal attempt to re-initialise SSL for server (theoretically shouldn’t happen!) //at 02:52 //by abe

After dist-upgrading my main Hetzner server from Lenny to Squeeze, Apache failed to come up, barfing the following error message in the alphabetically last defined and enabled virtual host’s error log:

[error] Illegal attempt to re-initialise SSL for server (theoretically shouldn't happen!)

Well this is not theory but the real world and it did happen — and it took me a while to find out what was wrong with the configuration despite it worked with Lenny’s Apache version.

To avoid that others have to search as long as I had to, here’s the solution:

Look at all enabled sites, pick out those which have a VirtualHost on port 443 defined and verify that all these VirtualHost containers do have their own “SSLEngine On” statement. If at least one is missing, you’ll run into the above mentioned error message.

And it won’t necessarily show up in the error log of those VirtualHosts which are missing the statement but only in the last VirtualHost (or the last VirtualHost on port 443).

To find the relevant site files, I used the following one-liner:

grep -lE 'VirtualHost.*443' sites-enabled/*[^~] | \
  xargs grep -ci "SSLEngine On" | \
  grep :0

Should work for all sites which have defined just one VirtualHost on port 443 per file.

I suspect that the raise of SNI made Apache’s SSL implementation more picky with regards to VirtualHosts.

Oh, and kudos to this comment to an article on Debian-Administration.org because it finally pointed me in the right direction. :-)

Useful but Unknown Unix Commandline Options: sort -h //at 01:12 //by abe

The GNU coreutils command “du” knows about the option “-h” to output human readable (or at least human friendly) values with unit prefixes, e.g. k, M or G.

The GNU coreutils command “sort” also can sort by numbers for quite a long time using the option “-n”, but that doesn’t work on the output of “du -h”. So you usually just did one of the following commands, but couldn’t easily combine them:

$ du -h
$ du | sort -n

For approximately a year, GNU sort now knows about another command line option named “-h”. You guessed it probably: “sort -h” can sort human readable values with SI prefixes, e.g.

$ du -h | sort -h | tail -15
34M     ./ttf-mplus-033/debian/ttf-mplus
34M     ./ttf-mplus-033/debian/ttf-mplus/usr
34M     ./ttf-mplus-033/debian/ttf-mplus/usr/share
34M     ./ttf-mplus-033/debian/ttf-mplus/usr/share/fonts
34M     ./ttf-mplus-033/debian/ttf-mplus/usr/share/fonts/truetype
34M     ./ttf-mplus-033/debian/ttf-mplus/usr/share/fonts/truetype/ttf-mplus
35M     ./ttf-mplus-034
57M     ./ttf-mplus-029
60M     ./php5-5.2.6/ext
60M     ./ttf-mplus-030
63M     ./ttf-mplus-031
65M     ./ttf-mplus-032
67M     ./ttf-mplus-033
81M     ./php5-5.2.6
1.5G    .
$

You can get this feature already in Debian Unstable (Sid) and Testing (Squeeze, the upcoming stable release), and Ubuntu Maverick and Natty, but not yet in the current Debian Stable release (Lenny) nor in the last Ubuntu LTS release (Lucid Lynx).

ratpoison and focus follows mouse //at 00:22 //by abe

I use ratpoison as window manager on my ASUS EeePC netbook “nemo” for more than two years now. But although I’m very happy with ratpoison in the EeePC, there are two feature wishes which have been refused by upstream: One is more flexibel window name matching for the unmanage command. The other one is “focus follows mouse” between ratpoison frames.

Well, I always guessed that it was possible, but it took until now to find outhow to implement “focus follows mouse” for ratpoison.

There’s an ancient but still useful tool called Not a Window Manager (nawm) which is a small awk-like interpreter offering mostly window handling functions.

The following .nawmrc implements “focus follows mouse” in nawm:

window newwin;  # stores window to raise
window lastwin; # stores previous window to prevent race conditions
leave {
    lastwin = currentwindow;
}
enter {
    newwin = pointerwindow();
    if (name(newwin) != "" && newwin != lastwin) {
        raise newwin;
        sync;
    }
}

The leave hook is necessary to prevent flapping between two windows if switched between them via ratpoison’s commands.

I also had to add the following hook to my .ratpoisonrc to work around some cases where ratpoison’s own window switching didn’t work anymore. Only happened with more than one frame — with one frame banishing the mouse cursor was annoying, so I filtered that case:

addhook switchwin exec if [ `ratpoison -c fdump|fgrep -o frame|wc -l` -gt 1 ]; then ratpoison -c banish; fi

Unfortunately nawm has been removed from Debian Sid about a year ago due to being buggy and orphaned. There was not upstream development for seven years or so either.

So for the moment you can get nawm either from Debian Lenny or from snapshot.debian.org.

But I had to fix a segfault in nawm when calling name() on a window without name to be able to use it at all, so you will probably have to rebuild it anyway with the following patch:

diff -u nawm-0.0.20030130/builtins.c nawm-0.0.20030130-patched/builtins.c
--- nawm-0.0.20030130/builtins.c        2010-10-25 06:00:02.000000000 +0200
+++ nawm-0.0.20030130-patched/builtins.c        2010-10-25 04:15:25.000000000 +0200
@@ -546,8 +546,12 @@
     *name = gcstrdup("");
   else
     {
-      *name = gcstrdup((char *)nm);
-      XFree(nm);
+      if ((char *)nm) {
+        *name = gcstrdup((char *)nm);
+        XFree(nm);
+      } else {
+        *name = gcstrdup("");
+      }
     }
 }

And yes, I’m thinking about adopting and reintroducing the nawm package into Debian Sid.

But I’d prefer if anyone could give me a hint how to do this with more current and still maintained tools (or a patch against ratpoison :-). I looked into suckless-tools, but I haven’t found anything in there which provides hooks on X events. And the Perl module Tk seems to be able to set X event hooks, but only within the application being written itself.

New upstream versions of xrootconsole and keynav in Debian Experimental //at 16:45 //by abe

I recently uploaded new upstream versions of two neat small X tools to Debian Experimental:

• xrootconsole displays on a transparent or shaded layer on the root window what it gets as input on STDIN, from a FIFO or from a file, and
• keynav, a way to control your mouse cursor efficiently with the keyboard.

Both packages introduce several new features and I’d be happy if users of these packages in Debian Sid or Debian Sid users curious about them could test the versions in Debian Experimental.

xrootconsole 0.6 + patches

xrootconsole saw no love since 2006 with the last maintainer upload having been in 2002. Nevertheless it never got kicked out of Debian just because of this. The package had been forcibly orphaned just less than a year ago. So it’s no big wonder that this new upstream version I packaged was released already back in 2004. :-)

But besides packaging a new upstream version, bumping Standards-Version and debhelper compatibility, fixing tons of lintian warnings and some bugs, I also added two patches which add new features not (yet) available upstream:

• UTF-8 support: Upstream xrootconsole just drops all bytes where the 8th bit is set which only allows ASCII. Miroslav Jezbera submitted a patch to allow at least displaying localized text containing single byte 8 bit characters as used in all ISO-Latin encodings. Inspired by the patch to add UTF-8 support to ratmenu, I wrote a patch to add UTF-8 support for xrootconsole, too.
• ANSI color support: Last year, Julien Viard de Galbert, who is also active in Debian, posted a patch in his blog to support ANSI colors as produced by many log colorizers like e.g. loco (RIP), colortail, lwatch, or ccze. (Didn’t get colortail and lwatch to work with xrootconsole yet, though.) I included this patch and made it compatible with my UTF-8 support patch. The patch raises the memory consumption per displayed character by one byte, but effectively I just saw an overall memory usage increase of about 25% which seems acceptable.

Of course I informed upstream about these feature patches, but I haven’t got any feedback yet.

But I got feedback from Julien Viard de Galbert, and he’ll join me in packaging xrootconsole as co-maintainer.

keynav 0.20101014.3067

I adopted the Debian package of keynav recently, subscribed to the keynav mailing list (well, it’s a Google group), got a nice welcome mail from the very friendly upstream developer Jordan Sissel who offered to help with any keynav issues.

I told him what features I’d like to see in keynav to fit better the setup where I’m using it. And just a few days later there was a new upstream release including both features I suggested, some more neat new features, and one bug fix. Jordan Sissel writes in the upstream changelog (emphasis and italic text by me):

• Added ‘restart’ command. Makes keynav restart. Useful for binding a key to reload the config.
• Added ‘loadconfig’ command. This lets you include additional config files to load on the command line or in one of the default keynavrc files. (requested by Axel Beckert) … and already in use here.
• keynav will now restart if it receives SIGHUP or SIGUSR1
• Map ‘Enter’ by default to ‘warp,click 1,end’ (requested by Axel Beckert)
• Fix a bug causing the point under the mouse cursor to not click through the keynav window in certain conditions. Reported via mailing list by Eric Van Dewoestine and Krister Svanlund.

Both packages will be reuploaded to Debian Sid (Unstable) after the release of Debian Squeeze (currently “testing”).