GNU Screen 4.2.0 in Debian Experimental //at 20:22 //by abe

About a month ago, on 20th of March, GNU Screen had its 27th anniversary.

A few days ago, Amadeusz Sławiński, GNU Screen’s new primary upstream maintainer, released the status quo of Screen development as version 4.2.0 (probably to distinguish it from all those 4.1.0 labeled development snapshots floating around in most Linux distributions nowadays).

I did something similar and uploaded the status quo of Debian’s screen package in git as 4.1.0~20120320gitdb59704-10 to Debian Sid shortly afterwards. That upload should hit Jessie soon, too, resolving the following two issues also in Testing:

• #740301: proper systemd support – Thanks Josh Triplett for his help!
• #735554: fix for multiuser usage – Thanks Martin von Wittich for spotting this issue!

That way I could decouple these packaging fixes/features from the new upstream release which I uploaded to Debian Experimental for now. Testers for the 4.2.0-1 package are very welcome!

Oh, and by the way, that upstream comment (or ArchLinux’s according announcement) about broken backwards compatibility with attaching to running sessions started with older Screen releases doesn’t affected Debian since that has been fixed in Debian already with the package which is in Wheezy. (Thanks again Julien Cristau for the patch back then!)

While there are bigger long-term plans at upstream, Amadeusz is already working on the next 4.x release (probably named 4.2.1) which will likely incorporate some of the patches floating around in the Linux distributions’ packages. At least SuSE and Debian offered their patches explicitly for upstream inclusion.

So far already two patches found in the Debian packages have been obsoleted by upstream git commits after the 4.2.0 release. Yay!

Updates (8th of May 2014): 4.2.0 in Testing, Upstream released 4.2.1

screen 4.2.0-2 migrated to testing now.

Upstream released 4.2.1 in the meanwhile with most Debian patches applied. Despite being a minor update, it was necessary to bump it’s internal message version, so vanilla 4.2.1 clients can’t connect to vanilla 4.2.0 servers. Accordingly it may take a moment until 4.2.1 hits Debian as I need to sort out some stuff before uploading that version.

Xen: Running a Sid DomU with PyGrub on a Squeeze Dom0 //at 03:07 //by abe

I’m running one Debian Sid and one Jessie (Testing) Xen guest domain on a Debian Squeeze (Oldstable) Xen 4.0 running host server.

Recently I had to reboot one these virtual machines after more than a year of uptime. But the new 3.14 kernel from Debian Experimental didn’t boot. Neither did 3.13 from Debian Unstable. Nor did any other kernel image newer then the 3.5-trunk (from Debian Experimental back than) work.

Everytime pygrub bailed out with this error message:

Error: (2, 'Invalid kernel', 'xc_dom_find_loader: no loader found\n')

(Yes, the parentheses and the “\n” were part of the error message.)

After some searching on the web I found hints that this message may be caused by an unsupported compression type in the kernel image.

And indeed, if I unpack the “vmlinuz” with the extract-vmlinux tool which is part of Linux’ source code (but not yet part of any binary package in Debian), and use the extract file in grub’s menu.lst (which is then read by pygrub) instead, the DomU boots Linux kernel 3.14 again, even on a Squeeze-running Dom0.

PDiffs are still useful //at 03:26 //by abe

… probably just not as default.

I do agree with Richi and with Michael that disabling PDiffs by default gives the big majority of Debian Testing or Unstable users a speedier package list update.

I’m though not sure, if disabling PDiffs by default would

• also have an performance impact on our mirrors — it surely would have a traffic impact on the mirrors;
• really bring a benefit for Debian Stable users as Debian Stable changes seldomly and hence there are not that many PDiffs to download and apply — at least I can’t remember being annoyed by PDiffs anywhere else than on Debian Testing and Debian Unstable. Even the repositories with security updates don’t change that often.

Additionally I want to remind you that PDiffs per se are nothing bad and should be continued to be supported:

• Because there are still areas, even in “civilized” countries, where only small bandwidth is available and where using PDiffs reduces the download time a lot. Yes, also in Germany. BTDT. Only until recently there was no Fibre, no DSL, very bad UTMS reception and otherwise just EDGE at my parents’ home. (LTE was available far too expensive until recently.) And I was very happy about not having to download 30 MB or such just for seeing if there are updates at all, because 25 kB/s was the fastest download rate I could get (peaks, not average).
• Because it seems to be in fashion with big ISP near-to-monoplists, especially in Germany, to cut off your nice bandwidth if you transfer too many Megabytes. Keyword “Drosselkom”. If you happen to be a customer of such a shitty ISP, you may be happy to reduce your traffic amount by using PDiffs instead of downloading the full package list every time.

So yes, disabling PDiffs by default is probably ok, but the feature must be kept available for those who haven’t 100 MBit/s fibre connection into their homes or are sitting just one hop away from the next Debian mirror (like me at work :-).

Oh, and btw., for the very same reasons I’m also a big fan of debdelta which is approximately the same as PDiffs, just not for package lists but for binary packages. Using debdelta I was able to speed up my download rates over EDGE to up to virtual 100 kB/s, i.e. by factor four (depending on the packages). Just imagine a LibreOffice minor update at 15 kB/s average download rate. ;-)

And all these experiences were not made with a high-performance CPU but with the approximately 5 year old Intel Atom processor of my ASUS EeePC 900A. So I used PDiffs and debdelta even despite having a slight performance penalty on applying the diffs and deltas.

unburden-home-dir uploaded to Sid //at 02:54 //by abe

Most popular web browsers cause quite a lot of I/O on a user’s home directory and their cache’s also take up quite some disk space – with Google’s Chrome/Chromium you can’t even configure how much disk space should be used for the cache.

This causes unnecessary network traffic and no more makes sense if the home directory itself comes over the network, e.g. via NFS or Samba. And on laptops it spins up the disks and unnecessarily costs battery power and therefore lowers the potential battery life.

Such caches also costs scarce disk space on SSDs or flash cards as common in laptops, netbooks and other mobile devices, and often get backed up without any real use.

To take some of this burden off our NFS servers at work I started to develop an Xsession.d hook which moves off such caches to the local disk and puts in symbolic links instead into the user’s home directory when the user locally logs in.

This hook quickly became a standalone Perl script named unburden-home-dir and the Xsession.d hook just a wrapper around it. Due to some unsolved issues I didn’t feel it’s good enough for Debian Unstable, so I uploaded it just to Debian Experimental back then.

Pietro Abate’s recent blog posting about unburden-home-dir on Planet Debian gave me the right kick to make another try to solve the remaining issues.

And the mental distance gained over the time indeed helped and I could fix the remaining issues. So I added some polish to the package and uploaded it to Debian Unstable.

If you used the previous version from experimental, you have to take care of a few things:

• Previously some configuration files sported unburden_home_dir as base name while others used unburden-home-dir as base name as that’s also the package name. Now all configuration files use the package name, i.e. unburden-home-dir as base name.
• “Conffiles” under /etc/ should be renamed by dpkg automatically, but per-user configuration files ($HOME/.unburden_home_dir and $HOME/.unburden_home_dir_list) must be manually renamed to $HOME/.unburden-home-dir and $HOME/.unburden-home-dir.list.
• By adding UNBURDEN_HOME=yes to $HOME/.unburden-home-dir every user can decide himself if he wants the Xsession.d hook to be used when he logs in under X. On managed workstations with many users this eases testing of unburden-home-dir with just a few users a lot.

You can follow the development of unburden-home-dir also on GitHub and on Gitorious as well as on Ohloh.

Enjoy!

Daily Snapshot .debs of Conkeror //at 22:57 //by abe

Keeping track with packaging software which is under heavy development can be time-consuming. I noticed this while packaging Conkeror, because there was quite a demand for up-to-date packages, especially from upstream themself.

So recently on the IRC channel #conkeror the idea of automatically built Debian packages came up. After a few hours of experimenting and a few days of steadily optimizing, I can proudly present daily built snapshot packages of Conkeror for currently Lenny and Sid, ready to be included in your sources.list:

deb     http://noone.org/conkeror-nightly-debs lenny main
deb-src http://noone.org/conkeror-nightly-debs lenny main

deb     http://noone.org/conkeror-nightly-debs sid main
deb-src http://noone.org/conkeror-nightly-debs sid main

The binary package conkeror-spawn-process-helper is currently only built for the i386 architecture, but other architectures may follow.

The packages probably work also on any other Debian based distribution (e.g. Ubuntu) which includes XULRunner version 1.9.

Surely they are not of the usual Debian quality, but they should do it for staying up-to-date with the Conkeror development just by using your favourite APT frontend.

The script which generates those packages is also available in the Conkeror git repository at repo.or.cz.

The APTable archive is generated with reprepro. Packages and the repository are signed with the passphrase-less GnuPG key 373B76B4 which is used only for the Conkeror nightly builds. (If anyone knows a better solution for automatic builds than a passphrase-less key, please tell me. :-)

P.S.: I really like the new keybindings “<<”, “>>” and “G”. :-)

How to move a git submodule //at 20:31 //by abe

If you try to move a git submodule with git mv, you’ll get the following error message:

$ git mv old/submodule new/submodule
fatal: source directory is empty, source=old/submodule, destination=new/submodule

There’s a patch against git to supoort submodule moving, but it doesn’t seem to be applied yet, at least not in the version which is currently in Debian Sid.

What worked for me to solve this issue was the following (as posted on StackOverflow):

1. Edit .gitmodules and change the path of the submodule appropriately, and put it in the index with git add .gitmodules.
2. If needed, create the parent directory of the new location of the submodule: mkdir new.
3. Move all content from the old to the new directory: mv -vi old/submodule new/submodule.
4. Remove the old directory with git rm --cached old/submodule.

Looked like this for me afterwards:

 # On branch master
 # Changes to be committed:
 #   (use "git reset HEAD <file>..." to unstage)
 #
 #       modified:   .gitmodules
 #       renamed:    var/lib/dokuwiki/tpl -> var/lib/dokuwiki/lib/tpl
 #

Finally commit the changes. HTH.

Useful but Unknown Unix Commandline Options: sort -h //at 01:12 //by abe

The GNU coreutils command “du” knows about the option “-h” to output human readable (or at least human friendly) values with unit prefixes, e.g. k, M or G.

The GNU coreutils command “sort” also can sort by numbers for quite a long time using the option “-n”, but that doesn’t work on the output of “du -h”. So you usually just did one of the following commands, but couldn’t easily combine them:

$ du -h
$ du | sort -n

For approximately a year, GNU sort now knows about another command line option named “-h”. You guessed it probably: “sort -h” can sort human readable values with SI prefixes, e.g.

$ du -h | sort -h | tail -15
34M     ./ttf-mplus-033/debian/ttf-mplus
34M     ./ttf-mplus-033/debian/ttf-mplus/usr
34M     ./ttf-mplus-033/debian/ttf-mplus/usr/share
34M     ./ttf-mplus-033/debian/ttf-mplus/usr/share/fonts
34M     ./ttf-mplus-033/debian/ttf-mplus/usr/share/fonts/truetype
34M     ./ttf-mplus-033/debian/ttf-mplus/usr/share/fonts/truetype/ttf-mplus
35M     ./ttf-mplus-034
57M     ./ttf-mplus-029
60M     ./php5-5.2.6/ext
60M     ./ttf-mplus-030
63M     ./ttf-mplus-031
65M     ./ttf-mplus-032
67M     ./ttf-mplus-033
81M     ./php5-5.2.6
1.5G    .
$

You can get this feature already in Debian Unstable (Sid) and Testing (Squeeze, the upcoming stable release), and Ubuntu Maverick and Natty, but not yet in the current Debian Stable release (Lenny) nor in the last Ubuntu LTS release (Lucid Lynx).