xen-tools back in Debian Unstable //at 13:12 //by abe

After xen-tools had been removed from Debian Unstable by the upstream developer last December due to no more upstream development, I took over upstream development as well as Debian package maintainership in January.

Since then a lot happened:

• Development moved from Mercurial to a set of git repositories at gitorious.org.
• Dmitry Nedospasov joined my effort to revive xen-tools.
• Via gitorious’ clone tracking we noticed that there were useful patches in clones of xen-tools and incorporated most of them back into our master branch.
• We created a second mailing list xen-tools-dev which is mainly thought for communication between the xen-tools developers.
• The xen-tools.org website and the xen-tools mailing lists moved from Steve’s to my server. The website has been slightly overhauled and the mailing lists now use Mailman.
• Support for all recent Debian and Ubuntu releases including Squeeze and Lucid.
• Full support for pygrub and DomU distribution kernels installed in the DomU. Now you easily can install DomUs whose udev doesn’t fit to your Dom0 kernel.
• Full support for cdebootstrap in addition to debootstrap.
• Tons of bugs fixed
• I’m on the quest to find the holy grail of bug tracking systems. ;-) More on this in another blog posting.

But despite all the things which were to fix after Steve discontinued the xen-tools developement, Steve Kemp did a great job to bring xen-tools so far initially. I’m really happy that this software exists and that’s the reason why I won’t let it die so fast. :-)

xen-tools 4.2 beta 1 released

Last weekend I released xen-tools 4.2 beta 1 which I also uploaded to Debian Unstable. And thanks to the active ftp-master team, xen-tools went through the NEW queue in less than 12 hours and is now available again in Debian Unstable.

4.2 beta 1 still has some known bugs, the worst ones are in xen-delete-image. But no fear, it doesn’t delete more than wanted, just not enough or things which don’t exist anyway. :-) There are also some smaller issues to fix for a final 4.2 release.

We may also create an PPA for Ubuntu Lucid in the future to work around the lack of xen-tools in Ubuntu Lucid.

What about Fedora/RedHat/CentOS support?

Since rinse is out of date with regards to current Fedora releases, rpmstrap seems to be out of date for even longer (it has been removed from Debian before the Lenny release) and febootstrap neither seems to work out of the box nor seems to support bootstrapping to mount points (because it wants to delete them first and fails), bootstrapping Fedora ist currently not supported. The “copy” installation method should work though.

rinse still is able to bootstrap CentOS 4 and 5, but the installation seems incomplete. We’ll try to work around these issues for the final 4.2 release.

We haven’t looked at mach and mock so far, but according to Lucas, both are out of date, too.

Sure, implementing Lucas’ howto in form of a script would be a possible way, but we think it would just create yet another not so well supported clone of rinse, rpmstrap, mach, mock or febootstrap.

Because of that, Dmitry works on support for the installation of Xen DomUs via kickstart.

Fate of xen-shell?

Xen-shell has also been removed from Debian Unstable last December because Steve stopped its developement, too.

But in comparison to xen-tools itself, we didn’t resurrect xen-shell since none of us currently has use for it.

So if you are currently using xen-shell and intend to continue to do so, you should hurry up if you want to see it in Debian Squeeze. We can also help you, but we surely can’t and won’t do it alone nor will we take the majority of the necessary workload.

Perfect Team: autossh and GNU Screen //at 13:11 //by abe

SSH is definitely one of my most often used tools, not only for system administration at work but also on the road with my netbook, an EeePC 701 running Debian Sid.

On the road, it often happens that I have a flaky WLAN or UMTS connection, so I often have to kill (via <Enter>~.) and reconnect my SSH session due to a changed IP address or so.

First step against problems arising from using SSH over unreliable network connections is of course GNU Screen. Second step is use SSH keys and ssh-agent to not needing to type the password on every reconnect.

But it’s still very annoying to kill the SSH connection and call ssh again manually. For luck there is autossh, a wrapper around SSH which regularily checks via two tunnels connect to each other on the remote side if the connection is still alive, and if not, it kills the ssh and starts a new one with the same parameters (i.e. tunnels, port forwardings, commands to call, etc.).

It’s quite obvious that this is perfect to be combined with screen’s -R and -d options (Reattach if a detached screen is around, else start a new screen; detach a currently attached screen), so I found myself very often typing (or fetching it from the commandline history :-):

autossh -t sym.noone.org 'screen -Rd'

-t is necessary to allocate a terminal device on the remote machine which is not done by default if you directly call a command via ssh.

In comparision to OpenSSH, autossh needs the single quotes, because otherwise it would parse -Rd as options to parse to ssh and bail out. That’s not a real problem, but when you’re used to just type ssh -t sym.noone.org screen -Rd without any quotes, you’ll run into this then and when.

Update, 25-May-2010, 14:55: As Carsten Hey points out, autossh also supports the -- option to declare that all following options and parameters must be passed to ssh itself. (End of Update)

Typing that often and mistyping it then and when cries for an shell alias or an shell function. So I came up with the following shell function:

asc() {
    autossh -x -a -t "$@" 'screen -RdU'
}

I used a function instead of an alias in case of autossh will in future regard all parameters given after the command as part of the command as ssh does.

The additional options -x and -a disable X and SSH Agent forwarding which both don’t work if you reattach to an already running screen.

And if you’re using Zsh as I do, you can even add some more format string magic to set the window title more or less to the expanded alias, eh, function:

function asc() {
    # Set the title to something more obvious, e.g. the expanded
    # alias, eh, function
    print -Pn "\e]0;%n@%m: autossh -t $* 'screen -RdU'\a";
    autossh -x -a -t "$@" 'screen -RdU'
}
compdef asc=ssh

Update, 25-May-2010, 14:59: As Hauke points out in a comment, Zsh users should also declare that asc should have the same tab completion as ssh itself. The example above has been updated accordingly. (End of Update)

In the meanwhile on the EeePC I use asc on the commandline more often than ssh itself. And I nearly no more type autossh. (The most common exception here is autossh hostname tail -F /path/to/some/logfile.)

Using that function you can also add common ssh options for tunneling, etc. — I use it most often like this:

asc -D 1080 sym.noone.org

This opens a SOCKS proxy on localhost, port 1080 and that way I can surf via the host I’m connecting to by SSH.

There’s one small drawback though: You didn’t expect that I can just invent some new three letter command without a namespace clash, did you? There is a free game called Advanced Strategic Command whose binary (and Debian package) is named asc, too. If you have that game installed, you can always call it using its full path, e.g. /usr/games/asc on Debian.

P.S.: My whole grml based .zshrc is also available via git at git.noone.org as well as on github.

abe@debian.org //at 23:58 //by abe

On Wednesday I got DAM approval and since Saturday late evening I’m officially a Debian Developer. Yay! :-)

My thanks go to

• Christoph Berg (Myon) whom I know for more than a decade since we studied together, and who’s career in Debian was way faster than mine, but who on the other hand probably knows me better than nobody else in Debian — which made him the perfect advocate;
• Bernd Zeimetz (bzed) whom I know from my times at DaLUG and who was the friendliest Application Manager I could imagine — he’s probably also one of the fastest (8 days from application to AM report :-);
• Luca Capello (gismo), who was the most demanding but also most inspiring sponsor I ever had and who became a very good friend after we found each other over my package conkeror.
• Arne Wichmann (Y_Plentyn) for being my first drop-in center for Debian questions (like “can I directly dist-upgrade from 2.0 to 3.0?” :-);
• Martin Zobel-Helas (zobel) who was always encouraging me to continue exploring new sides of Debian;
• Gerfried Fuchs (Rhonda) just for being there (and for being a package maintainer with good relations to upstream ;-);
• my coworkers at the IT Services Group of the Department of Physics at ETH Zurich, who always found new challenges in Debian for me to solve;
• … and all those others who offered to also advocate me (e.g. Otavio Salvador) or sponsored my packages so far (or at least offered to do so), e.g. Alexander Wirt (formorer), Martin F. Krafft (madduck), Robert Jördens (jordens), …

As Bernd cited in his AM report, my earliest activity within the Debian community I can remember was organising the Debian booth at LinuxDay.lu 2003, where I installed Debian 3.0 Woody on my Hamilton Hamstation “hy” (a Sun SparcStation 4 clone).

I wrote my first bugreport in November 2004 (#283365), probably during the Sarge BSP in Frankfurt. And my first Debian package was wikipedia2text, starting to package it August 2005 (ITP #325417).

My only earlier documented interest in the Debian community is subscribing to the lists debian-apache@l.d.o and debian-emacsen@l.d.o in June 2002.

I though remember that I started playing around with Debian 2.0 Hamm, skipping 2.1 (for whatever reasons, I can’t remember), using 2.2 quite regularily and started to dive into with Woody which also ran on my first ThinkPad “bijou”. I installed it over WLAN with just a boot floppy at the Chemnitzer Linux-Tage. :-)

Anyway, this has led to what it had to lead — to a new Debian Developer. :-)

The first package I uploaded with my newly granted rights was a new conkeror snapshot. This version should work out of the box on Ubuntu again, so that conkeror in Ubuntu should not lag that much behind Debian Sid anymore.

In other News

Since Wednesday I own a Nokia N900 and use it as my primary mobile phone now. Although it’s not as free as the OpenMoko (see two other recent posts by Lucas Nussbaum and by Tollef Fog Heen on Planet Debian) it’s definitely what I hoped the OpenMoko will once become. And even if I can’t run Debian natively on the N900 (yet), it at least has a Debian chroot on it. :-)

A few weeks ago, I took over the organisation of this year’s Debian booth at FOSDEM from Wouter Verhelst who’s busy enough with FOSDEM organisation itself.

Last Monday the organiser of the BSD DevRoom at FOSDEM asked on #mirbsd for talk suggestions and they somehow talked me into giving a talk about Debian GNU/kFreeBSD. The slides should show up during the next days on my Debian GNU/kFreeBSD talks page. I hope, I’ll survive that talk despite giving more or less a talk saying “Jehova!”. ;-)

What a week.

Spam in SMTP not via SMTP //at 18:50 //by abe

While examining the mail queue after a big mail server migration, I found the following reason for a bounce (hostnames replaced according to RFC2606):

550-5.1.1 - 
550-5.1.1 -
550-5.1.1 TO LEARN WHY YOUR EMAIL WAS REJECTED PLEASE GO HERE: 
550-5.1.1 - 
550-5.1.1 http://www.example.com/answers/dWtsb3R0b3NAdWtsb3R0ZXJ5LmNvLnVrPgA=AAA=/
550-5.1.1 - 
550-5.1.1 Cheap, Reliable Webhosting
550-5.1.1 http://www.example.com/a/hostgator/
550-5.1.1 -
550-5.1.1 Round-Trip Flights under $200 from Priceline!
550-5.1.1 http://www.example.com/a/pricelinertf/
550-5.1.1 -
550-5.1.1 Free Skype-to-Skype calls on your mobile
550-5.1.1 http://www.example.com/a/skype/
550-5.1.1 -
550 5.1.1 -

What’s next? Advertisements in HTTP headers? Oh, I forgot, they already exist and are called “referrer spam”.

192.168.noone.org //at 21:41 //by abe

About a year ago, Eric Poscher invented the IP address blog and installed his one at http://192.168.epe.at/. Every hour his netbook notes down the IP address of the interface which currenntly the default route goes through and if it has an internet connection, it uploads the list of IP adresses it had. Additionally, he filters the list to IP addresses in 192.168.0.0/16.

June this year he published the source code behind his IP blog under GNU General Public License">GPL and Creative Commons. I modified his script slighty to just write down the new IP address if it’s different to the previous one, but without any filter. This makes the list much more colorful (and my online times less traceable :-) as you can see at http://192.168.noone.org/.

But the biggest disadvantage of Eric’s code design is not the fact that it’s a (quite nice to read :-) shell script but that it doesn’t save the list of IPs separately and is not able to regenerate everything if you want to change the design, but always just adds a line to the HTML page.

So I rewrote the whole thing in Perl last Saturday while sitting the dog of my parents. If you change the templates and call the script again, it regenerates the whole list with the new templates. The code is also under GNU General Public License">GPL, the HTML parts are under Creative Commons, too.

And hey, this is one of the very few (if not only) applications which are much more fun with IPv4 than with IPv6. ;-)

Still happy with the ASUS EeePC 701 //at 18:32 //by abe

Recently Eric asked on the LUG Vorarlberg mailing list about netbook experience. I wrote a lengthy reply summarizing my experiences with the ASUS EeePC 701. And I thought this is something I probably should share with more people than only one LUG:

I ordered an ASUS EeePC 701 (4G) with US keyboard layout at digitec in Spring 2008, got it approximately one month later and posted a first resumé after one month in my blog.

I’m still very happy with the EeePC 701, despite two commonly mentioned drawbacks (the small screen resolution and the small SSD – which I both don’t see as real problems) and some other minor issues.

What matters

• Very robust and compact case. And thanks to a small fan being the only moving part inside, the EeePC 701 is also very robust against mobile use.
• Very pleasing always-in-my-daypack size (despite the 7" screen it’s the typical 9" netbook size) and easily held with one hand.
• Black. No glossy display. Neither clear varnish nor piano laquer. Short: No bath room tile. Textured surface, small scratches don’t stick out and don’t matter.
• Debian (previously Lenny, now Sid) runs fine on it, even the webcam works out-of-the-box.
• Despite all those neat features, it was fscking cheap at that time. And it was available without Windows.

Nice to have

• There’s power on the USB sockets even if the EeePC is turned off but the power supply is plugged in.
• The speakers are impressingly good and loud for their size. (But my demands with regards to audio are probably not too high, so audiophiles shouldn’t run to ebay because of this. ;-)
• It has three external USB sockets.

What doesn’t matter

• The small 7" 800×480 screen: I like small fonts and do most things inside a terminal anyway. And even with 800×480, those terminals are still much bigger than 80×25 characters. Only some applications and webpages have no heart for small screens.
• The small disk size: Quite a lot of programs fit on 4 GB of disk space. Additionally I use tmpfs a lot. And music and video files are either on a external 500 GB Western Digital 2.5" “My Passport” disk (which I need quite seldomly) or much more come via sshfs and IPv6 from my home server anyway. :-)
• The small keyboard: I just don’t have any problems with the size or layout (right shift right of the cursor up key, etc.) of the keyboard. Well, maybe except that any standard sized keyboard feels extremely large after having used the EeePC exclusively for some weeks. ;-)
• The to 630 MHz underclocked 900 MHz Intel Celeron: It’s enough for most of the things I do with the EeePC. Also the original 512 MB RAM are somehow ok, but for using tmpfs, but no swap space at all, 1 GB or 2 GB are surely the better choice.
• A battery runtime of 2.5h to 3h is fine for me.

What’s not so nice

• The “n” key needs to be pressed slighty stronger than other keys, otherwise no “n” appears. So if one of my texts in average misses more “n” than other letters, I typed it on the EeePC. ;-)
• Home, End, Page-Up, and Page-Down need the Fn key. This means that these keys can only be used with two hands (or oe very big hand and I have quite small hands). This is usually no problem and you get used to it. It’s just annoying if you hold the EeePC with one hand and try to type with the other.
• What looks like a single mouse button is a seesaw and therefore two mouse buttons below one button. This makes it quite hard to press both at the same time, e.g. for emulating a middle mouse button press. It usually works in about half of all cases I tried it. My solution was to bind some key combination to emulate a middle mouse button in my window manager, ratpoison:

  bind y ratclick 2

  And that mouse button bar already fell off two times.
• The battery reports only in 10% steps, and reporting in percentage instead of mAh is an ACPI standard violation because reporting in percentage is only allowed for non-rechargable batteries. It also doesn’t report any charging and discharging rates. But in the meanwhile nearly all battery meter can cope with these hardware bugs. This was quite a problem in the early days.
• Now, after approximately 1.5 years, the battery slowly fritzes: When charging there are often only seconds between 10% and 40%. Rigorously using up all power of the battery helped a little bit. Looks like some kind of memory effect althought the battery is labeled Li-Ion and not Ni-MH and Li-Ion batteries are said to have no memory effect.
• The SD card reader only works fine if you once completed the setup of the original firmware or set the corresponding BIOS switch appropriately. No idea why.

Similar models

Technically, most of this also counts for the EeePC 900SD (not 901) which only differs in screen, resolution and disk size as well as CPU, but not on the the case. So same size, same robustness, same battery, same mainboard, bigger screen, resolution, disk and faster CPU. (The 901 has a different CPU, a different battery, and a different, glossy and partially chromed case.) See Wikipedia for the technical specifications of all EeePC models.

ASUS’ only big FAILure

Stopping to sell most EeePCs with Linux and cowardly teaming up with Microsoft after having shown big courage to come out with a Linux only netbook. Well, you probably already know, but it’s better without Windows…

So basically you no more get these really neat netbooks from ASUS anymore and you get nearly no netbooks with Linux from ASUS in the stores anymore. It’s a shame.

Would I buy it again?

Sure.

Well, maybe I would also buy the 900SD or 702 (8G) instead of the 701, but basically they’re very similar. See Wikipedia for the differences between these EeePC models. And of course I still prefer the versions without Windows.

But despite the low price, the EeePC 701 is surprisingly robust and still works as on the first day (ok, except battery, the mouse button bar and the “n” key ;-), so I recently bought a second power supply (only white ones were available *grrrr*) and ordered a bigger third party battery plus an adapter to load the battery directly from the (second) power supply without EeePC inbetween.

What desktop do I use on the EeePC?

None.

I use ratpoison as window manager, uxterm, urxvt, and yeahconsole as terminal emulators (running zsh with grml based .zshrc even as root’s login shell :-), wicd-curses as network manager and xmobar (previously dzen2) with i3status as text-only panel. Installed editors are GNU Emacs 23, GNU Zile and nvi. (No vim. :-)

And of course a netbook wouldn’t be a netbook if it wouldn’t have a lot of network applications installed. For me the most important ones are: ssh, scp, autossh, sshfs, miredo, conkeror, git, hg, and rsync.

/var/cache/apt/ on tmpfs //at 16:44 //by abe

My ASUS EeePC 701 (4G) “nemo” running Debian Sid has a 4 GB SSD as main disk, which is on the one hand quite full (mostly with software I use, but also local working copies of software I work on) and on the other hand an SSD, so I always try to reduce the amount of write to disk without losing convenience. Similar issues have systems which run off a CF or SD card or maybe even an USB stick.

Since I ordered a 2 GB RAM bar together with the EeePC, I not bound to the 512 MB which it had originally. But on the other hand I seldom needed more than 1 GB of RAM. Usually I needed between 400 MB and 1 GB of RAM. So it’s quite obvious to use tmpfs on as many places as possible.

Making /tmp, /var/run and /var/lock tmpfs were the most obvious directories to mount as tmpfs. Especially /var/run on tmpfs brought up a few bugs a while ago (mostly init.d scripts relying on /var/run/$PACKAGENAME/’s existence), but it’s no hassles to use nowadays. Even in Debian Stable such bugs got fixed.

Next target to explore for was /var/cache. According to the FHS, /var/cache is intended for cached data from applications. […] The application must be able to regenerate or restore the data. So it should be safe to put anything under /var/cache on tmpfs.

One directory in there which gets written quite often and with a lot of data on Debian Unstable is /var/cache/apt and its subdirectories, especially /var/cache/apt/archives. If you update your Sid installation daily, all new or updated .debs will be downloaded to /var/cache/apt first.

So I put /var/cache/apt on tmpfs by putting the following line into /etc/fstab:

tmpfs /var/cache/apt tmpfs defaults,noexec,nosuid,nodev,mode=0755 0 0

But despite FHS stating that anything under /var/cache must be reproducible by the application, apt is puking and refusing to work:

!447 Z31 ?0 L1 root@nemo:pts/0 (-zsh) 16:13:10 [~] # apt-get update
E: Archive directory /var/cache/apt/archives/partial is missing.
!448 Z32 ?100 L1 root@nemo:pts/0 (-zsh) 16:13:17 [~] # 

If you create /var/cache/apt/archives/partial, it will also argue about /var/cache/apt/partial.

Of course the workaround is simple: Just put mkdir -p /var/cache/apt/partial /var/cache/apt/archives/partial in /etc/rc.local.

But nevertheless, this is a bug in apt – which already has been reported by madduck earlier this year (#523920). Unfortunately the APT maintainers have not yet even commented on this FHS violation and therefore also a Debian Policy (Section 9.1.1) violation.

One more thought about /var/cache/apt vs only /var/cache/apt/archives: apt-file also caches its data under /var/cache/apt. So if you want to use apt-file after a reboot and have /var/cache/apt mounted as tmpfs, you have to run apt-file update first and it will download all Contents files (can be dozens of megabytes) and not only the differences to previously downloaded Contents files.

So if you use apt-file a lot, you probably go better with making only /var/cache/apt/archives tmpfs and not whole /var/cache/apt.