Showing packages newer than in archive with aptitude //at 22:14 //by abe

I happens quite often that I install a manually built, newer version of some package on a machine. Occassionally I forget to remove it or to downgrade it to the version in the APT repo.

$ apt-show-versions | fgrep newer

easily finds those packages.

But usually when doing such a check, I want this list of packages in my aptitude TUI to have a look at the other versions of that package and to take actions. And I don’t want to manually search for each of the package manually.

This can be done with the following “one-liner”:

# aptitude -o "Aptitude::Pkg-Display-Limit=( `apt-show-versions | fgrep newer | awk -F '[ :]' '{printf "~n ^"$1"$ | "}' | sed -e 's/| *$//'` )"

It uses apt-show-version’s output, searches for the right packages, takes the first column and transforms it into an aptitude search pattern matching all packages whose name is exactly one of the listed packages.

But this solution is quite ugly and slow. So I wondered if this is also doable with pure aptitude search patterns which likely would also be faster.

And after some playing around I found the following working aptitude search term:

~i ?any-version(!~O.) !~U !~o

This matches all packages which which are installed and which have a version which has no origin, i.e. no associated APT repository. Since this also matches all hold packages as well as all packages not available in any archive, I use !~U !~o to exclude those packages from that list again.

Since nobody can remember that nor wants to type that everytime needed, I added the following alias to my setup:

alias aptitude-newer-than-in-archive='aptitude -o "Aptitude::Pkg-Display-Limit=~i ?any-version(!~O.) !~U !~o"'

Only caveat so far:

It seems to also match packages from APT repos which haven’t set an “Origin”. This should not happen with any Debian or Ubuntu APT repository, but seems to happen occasionally with privately run APT repositories.

And using ~A instead of ~O, i.e. ~i ?any-version(!~A.), does not work for this case either, despite it matches installed packages of which versions not in any available archive exist. But unfortunately aptitude seems to remember in some way if a package was in some archive in the past, so this only shows packages installed with dpkg -i, but not packages removed from e.g. unstable but with older versions still being available in stable.

Up to date Aptitude Documentation Online //at 12:51 //by abe

Aptitude ships documentation in 7 languages as HTML files. However the latest version available online was 0.4.11.2 from 2008 and hosted on the server by the previous, now unfortunately inactive Aptitude maintainer, and only covered 5 languages.

This lack of up to date online documentation even caused others to put more up to date versions online. Nevertheless they age, too, and the one I’m aware is not up to date for Wheezy.

So the idea was born to keep an up to date version online on Aptitude’s Alioth webspace (which currently redirects to a subdirectory of the previous maintainer’s personal website). But unfortunately we, the current Aptitude Team, are still lacking administrative rights on Aptitude’s Alioth project, which would be necessary to assign new team members who could work on that.

As an intermediate step, there’s now a (currently ;-) up to date Aptitude User’s Manual online in all 7 languages at

http://people.debian.org/~abe/aptitude/

and English at

http://people.debian.org/~abe/aptitude/en/

As this location could also suffer from the same MIA issues as any other “personal” copy, the plan is to move this to somewhere under http://aptitude.alioth.debian.org/ as soon as we have full access to Aptitude’s Alioth project.

Our plans for then are:

• Redirects from http://people.debian.org/~abe/aptitude/ to e.g. http://aptitude.alioth.debian.org/doc/, so that all your links to or bookmarks of http://people.debian.org/~abe/aptitude/ are still valid. (This unfortunately won’t work for jump marks to specific sections, just per file, i.e. chapter.)
• Set up a cron-job, which keeps the documentation in sync with the version of Aptitude in Unstable (and maybe also with Aptitude in Stable).

P.S.: Anyone interested in doing a German translation of the Aptitude User’s Manual? Sources are in DocBook, i.e. XML, and available via Git.

Automatically hardlinking duplicate files under /usr/share/doc with APT //at 20:43 //by abe

On my everyday netbook (a very reliable first generation ASUS EeePC 701 4G) the disk (4 GB as the product name suggests :-) is nearly always close to full.

TL;DWTR? Jump directly to the HowTo. :-)

So I came up with a few techniques to save some more disk space. Installing localepurge was one of the earliest. Another one was to implement aptitude filters to do interactively what deborphan does non-interactively. Yet another one is to use du and friends a lot – ncdu is definitely my favourite du-like tool in the meantime.

Using du and friends I often noticed how much disk space /usr/share/doc takes up. But since I value the contents of /usr/share/doc a lot, I condemn how Nokia solved that on the N900: They let APT delete all files and directories under /usr/share/doc (including the copyright files!) via some package named docpurge. I also dislike Ubuntu’s “solution” to truncate the shipped changelog files (you can still get the remainder of the files on the web somewhere) as they’re an important source of information for me.

So when aptitude showed me that some package suddenly wanted to use up quite some more disk space, I noticed that the new package version included the upstream changelog twice. So I started searching for duplicate files under /usr/share/doc.

There are quite some tools to find duplicate files in Debian. hardlink seemed most appropriate for this case.

First I just looked for duplicate files per package, which even on that less than four gigabytes installation on my EeePC found nine packages which shipped at least one file twice.

As recommended I rather opted for an according Lintian check (see bugs. Niels Thykier kindly implemented such a check in Lintian and its findings are as reported as tags “duplicate-changelog-files” (Severity: normal, from Lintian 2.5.2 on) and “duplicate-files” (Severity: minor, experimental, from Lintian 2.5.0 on).

Nevertheless, some source packages generate several binary packages and all of them (of course) ship the same, in some cases quite large (Debian) changelog file. So I found myself running hardlink /usr/share/doc now and then to gain some more free disk space. But as I run Sid and package upgrades happen more than daily, I came to the conclusion that I should run this command more or less after each aptitude run, i.e. automatically.

Having taken localepurge’s APT hook as example, I added the following content as /etc/apt/apt.conf.d/98-hardlink-doc to my system:

// Hardlink identical docs, changelogs, copyrights, examples, etc

DPkg
{
Post-Invoke {"if [ -x /usr/bin/hardlink ]; then /usr/bin/hardlink -t /usr/share/doc; else exit 0; fi";};
};

So now installing a package which contains duplicate files looks like this:

~ # aptitude install perl-tk
The following NEW packages will be installed:
  perl-tk 
0 packages upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 2,522 kB of archives. After unpacking 6,783 kB will be used.
Get: 1 http://ftp.ch.debian.org/debian/ sid/main perl-tk i386 1:804.029-1.2 [2,522 kB]
Fetched 2,522 kB in 1s (1,287 kB/s)  
Selecting previously unselected package perl-tk.
(Reading database ... 121849 files and directories currently installed.)
Unpacking perl-tk (from .../perl-tk_1%3a804.029-1.2_i386.deb) ...
Processing triggers for man-db ...
Setting up perl-tk (1:804.029-1.2) ...
Mode:     real
Files:    15423
Linked:   3 files
Compared: 14724 files
Saved:    7.29 KiB
Duration: 4.03 seconds
localepurge: Disk space freed in /usr/share/locale: 0 KiB
localepurge: Disk space freed in /usr/share/man: 0 KiB
localepurge: Disk space freed in /usr/share/gnome/help: 0 KiB
localepurge: Disk space freed in /usr/share/omf: 0 KiB

Total disk space freed by localepurge: 0 KiB

Sure, that wasn’t the most space saving example, but on some installations I saved around 100 MB of disk space that way – and I still haven’t found a case where this caused unwanted damage. (Use of this advice on your own risk, though. Pointers to potential problems welcome. :-)

aptitude-gtk will likely vanish //at 01:06 //by abe

As Christian already wrote, there’s an Aptitude revival ongoing. We already saw this young team releasing aptitude 0.6.5 about 6 weeks ago, more commits have been made, and now we’re heading towards an 0.6.6 release quickly.

But this revival mostly covers the well-known and loved curses interface (TUI) of aptitude and not the seldomly installed GTK interface, which unfortunately never really took off:

While aptitude itself (i.e. the curses and commandline interface) is installed on nearly 99% of all Debian installations which take part in Debian’s “Popularity Contest” statistics, aptitude-gtk is only installed on 0.42% of all these installations.

One reason is likely that aptitude-gtk still hasn’t all the neat features of the curses interface. And another reason is probably that it’s still quite buggy.

Since nobody from the current Aptitude Team has the experience, leisure or time to resurrect (or even complete) aptitude-gtk, the plan is to stop building aptitude-gtk from the aptitude source package soon, i.e. to remove it from Debian for now.

Like the even less finished Qt interface of aptitude, its code will stay in the VCS, but will be unmaintained unless someone steps up to continue aptitude-gtk (or aptitude-qt, or both), maybe even as its own source package.

So if you like aptitude-gtk so much that you’re still using it and want to continue using it, please think about contributing by joining the Aptitude Team and getting aptitude’s GUI interface(s) back in shape.

Another option would be to find a mentor so that resurrecting (one of) aptitude’s GUI interfaces could become (again) a potential project at Debian’s participation at Google’s Summer of Code.

Please direct any questions about aptitude-gtk or aptitude-qt to the Aptitude Development Mailing List. Or even better, join the discussion in this thread.

Finding libraries not marked as automatically installed with aptitude //at 17:26 //by abe

This is a direct followup on my blog posting Finding packages for deinstallation on the commandline with aptitude.

In the meantime on more alias for finding obosolete packages made it into my zsh configuration. It’s an alias to find installed libraries, …-data, …-common and other usually only automatically installed packages, which are not marked as being installed automatically nevertheless:

alias aptitude-review-unmarkauto-libraries='aptitude -o "Aptitude::Pkg-Display-Limit=( ^lib !-dev$ !-dbg$ !-utils$ !-tools$ !-bin$ !-doc$ !^libreoffice | -data$ | -common$ | -base$ !^r-base ) !~M"'

And yes, this pattern is slightly larger than those from the previous posting, so here’s the used filter in a little bit more readable way:

(
  ^lib
    !-dev$
    !-dbg$
    !-utils$
    !-tools$
    !-bin$
    !-doc$
    !^libreoffice | 
  -data$ | 
  -common$ | 
  -base$
    !^r-base
)
!~M

It matches all non-automatically installed packages whose name starts with “lib”, but is neither a debug symbols package, a development header package, a documentation package, a package containing supplied commands, nor a libreoffice package.

Additionally it matches all non-automatically installed packages ending in -data, -common, or -base, but excludes r-base packages.

Of course you can then mark any erroneously unmarked library by pressing “M” (Shift-m).

If you press “g” for “Go” afterwards and wonder why nothing to remove shows up, be reminded that the filter limit is active in this view, too. So press “l” for “Limit” and then Ctrl-u to erase the current filter limit of this view and press enter to set the new (now empty) filter, et voilà…

Hope this is of help for some others, too.

Finding packages for deinstallation on the commandline with aptitude //at 20:18 //by abe

Although I often don’t agree with Erich (especially if GNOME is involved ;-), he recently posted something on Planet Debian which I found very helpful.

I also own a netbook where disk space is sacre. It’s an ASUS EeePC 701 with just 4GB disk space. And it runs Debian Sid, so dependencies change often, leaving packages installed which formerly had hard dependencies on, but are now left with just recommendations pointing to it.

Quite a few times I asked myself if it’s possible to find those packages and if so, how to do it. Well, I don’t have to ask myself that anymore, since Erich recently posted the appropriate filter patterns for my favourite package manager aptitude for this task in his posting “Finding packages for deinstallation”. Thanks, Erich!

Since those filters aren’t very easy to remember, I’d like to extend the usefulness of his posting towards the commandline. I for myself added the following aliases to my shell setup:

alias aptitude-just-recommended='aptitude -o "Aptitude::Pkg-Display-Limit=!?reverse-depends(~i) ~M !?essential"'
alias aptitude-also-via-dependency='aptitude -o "Aptitude::Pkg-Display-Limit=~i !~M ?reverse-depends(~i) !?essential"'

As youam suggested on IRC, I also added the filter !?essential since we won’t touch essential packages when cleaning up the list of installed packages anyway.

Hope this helps further.

Group packages by origin in aptitude //at 00:06 //by abe

I always wondered how others recognise non-Debian packages in the aptitude package tree. I also missed the additional priority level in the hierachy well-known from good old dselect.

For the last one, I quickly found out that you can set the priority as subsection — it’s straight forward after you’ve read the documentation: Just add ,priority at the end of the default grouping method for package views under “Options → UI Options” in the aptitude menu.

Getting the origin as given in the Release file of the repository a package originates from is a little bit more difficult. You need to use the pattern() group function with the appropriate search pattern: pattern(~O)

Since already the default default grouping method for package views doesn’t fit into the dialog, I nowadays just edit /etc/apt/apt.conf directly for changes on aptitude’s default grouping method for package views. It now looks like this on several of my machines:

Aptitude::UI {
  Default-Grouping "filter(missing),status,section(subdir,passthrough),pattern(~O),section(topdir),priority";
};

In aptitude this looks like this:

[...]
  --- text - Text processing utilities
  --\ utils - Various system utilities
    --- Debian
    --- Mowgli
    --- volatile.debian.org
  --\ web - Web browsers, servers, proxies, and other tools
    --- Debian
    --- Opera Software ASA
  --\ x11 - The X window system and related software
    --\ Debian
      --- contrib - Programs which depend on software not in Debian
      --\ main - The main Debian archive
        --- Priority optional
        --- Priority extra
      --- non-free - Programs which are not free software
    --- Mowgli
[...]

Unfortunately this doesn’t work with all non-Debian repositories since a few repository maintainer, e.g. those from Emdebian, arrogate to just keep “Debian” as their packages’ origin. This could be solved, if there’s a possibility to group by e.g. repository URL (host and/or path).

Another problem I haven’t solved yet is that grouping by origin does neither work with locally created nor virtual packages nor tasks — probably since all of them lack an origin. Those branches are just empty or don’t even show up anymore with this configuration. I probably have to dig a little bit more in the aptitude documentation to resolve this.

Now playing: E-Rotic — Max don’t have sex with your ex