Showing packages newer than in archive with aptitude //at 22:14 //by abe

I happens quite often that I install a manually built, newer version of some package on a machine. Occassionally I forget to remove it or to downgrade it to the version in the APT repo.

$ apt-show-versions | fgrep newer

easily finds those packages.

But usually when doing such a check, I want this list of packages in my aptitude TUI to have a look at the other versions of that package and to take actions. And I don’t want to manually search for each of the package manually.

This can be done with the following “one-liner”:

# aptitude -o "Aptitude::Pkg-Display-Limit=( `apt-show-versions | fgrep newer | awk -F '[ :]' '{printf "~n ^"$1"$ | "}' | sed -e 's/| *$//'` )"

It uses apt-show-version’s output, searches for the right packages, takes the first column and transforms it into an aptitude search pattern matching all packages whose name is exactly one of the listed packages.

But this solution is quite ugly and slow. So I wondered if this is also doable with pure aptitude search patterns which likely would also be faster.

And after some playing around I found the following working aptitude search term:

~i ?any-version(!~O.) !~U !~o

This matches all packages which which are installed and which have a version which has no origin, i.e. no associated APT repository. Since this also matches all hold packages as well as all packages not available in any archive, I use !~U !~o to exclude those packages from that list again.

Since nobody can remember that nor wants to type that everytime needed, I added the following alias to my setup:

alias aptitude-newer-than-in-archive='aptitude -o "Aptitude::Pkg-Display-Limit=~i ?any-version(!~O.) !~U !~o"'

Only caveat so far:

It seems to also match packages from APT repos which haven’t set an “Origin”. This should not happen with any Debian or Ubuntu APT repository, but seems to happen occasionally with privately run APT repositories.

And using ~A instead of ~O, i.e. ~i ?any-version(!~A.), does not work for this case either, despite it matches installed packages of which versions not in any available archive exist. But unfortunately aptitude seems to remember in some way if a package was in some archive in the past, so this only shows packages installed with dpkg -i, but not packages removed from e.g. unstable but with older versions still being available in stable.

How to make wget honour Content-Disposition headers //at 16:12 //by abe

Download links often point to CGI scripts which actually generate (or just fetch, i.e. proxy) the actual file to be downloaded, e.g. URLs like http://www.example.com/download.cgi?file=foobar.txt.

Most of such CGI scripts send the real file name in the Content-Disposition header as specified in the MIME Specification.

All browsers I know (well, at least those I use regularily :-) handle that perfectly and propose the file name sent in the Content-Disposition header as file name for saving the downloaded name which is usually exactly what I want.

All browsers do that, …, just not my favourite commandline download tool GNU Wget … Downloading the above URL with wget would look like this with default settings:

$ wget 'http://www.example.com/download.cgi?file=foobar.txt'
--2013-10-02 16:04:16--  http://www.example.com/download.cgi?file=foobar
Resolving www.example.com (www.example.com)... 93.184.216.119, 2606:2800:220:6d:26bf:1447:1097:aa7
Connecting to www.switch.ch (www.example.com)|2606:2800:220:6d:26bf:1447:1097:aa7|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2020 (2.0K) [text/plain]
Saving to: `download.cgi?file=foobar.txt'

100%[============================================>] 2,020       --.-K/s   in 0s

2013-10-02 16:04:24 (12.5 MB/s) - `download.cgi?file=foobar.txt' saved [2020/2020]

Meh!

But luckily Wget can do that, it’s just not enabled by default — because it’s an experimental and possibly buggy feature, at least according to the man page. Well, works for me! :-)

You can easily enabled it by default for either your user or the whole system by placing the following line in your ~/.wgetrc or /etc/wgetrc:

content-disposition = on

Given the CGI script sends an appropriate Content-Disposition header, the above output now looks like this:

$ wget 'http://www.example.com/download.cgi?file=foobar.txt'
--2013-10-02 16:04:16--  http://www.example.com/download.cgi?file=foobar
Resolving www.example.com (www.example.com)... 93.184.216.119, 2606:2800:220:6d:26bf:1447:1097:aa7
Connecting to www.switch.ch (www.example.com)|2606:2800:220:6d:26bf:1447:1097:aa7|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2020 (2.0K) [text/plain]
Saving to: `foobar.txt'

100%[============================================>] 2,020       --.-K/s   in 0s

2013-10-02 16:04:24 (12.5 MB/s) - `foobar.txt' saved [2020/2020]

Now Wget does what I mean!

You can also set this as flag on the commandline, but typing wget --content-disposition … everytime is surely not what I want. ;-)

git $something -p //at 16:09 //by abe

git add -p is one of my favourite git features. It lets you selectively add the local changes hunk by hunk to the staging area. This is especially nice if you want to commit one change in a file, but not a second one, you also already did.

Recently I noticed that you can also selectively revert changes already in the staging area using git reset -p HEAD. The user interface is exactly the same as for git add -p.

Today I discovered another selective undo in git by just trying it out of curiosity if that works, too: Undoing local changes selectively using git checkout -p. Maybe less useful than those mentioned above, but nevertheless most times quicker than firing up your favourite editor and undoing the changes manually.

Another nice git feature which I discovered by accidentially using it (this time even unwittingly) is git checkout - which behaves like cd -, just for branches instead of directories, i.e. it switches back to the previously checked out branch. Very useful for quickly changing between two branches again and again.

Finding libraries not marked as automatically installed with aptitude //at 17:26 //by abe

This is a direct followup on my blog posting Finding packages for deinstallation on the commandline with aptitude.

In the meantime on more alias for finding obosolete packages made it into my zsh configuration. It’s an alias to find installed libraries, …-data, …-common and other usually only automatically installed packages, which are not marked as being installed automatically nevertheless:

alias aptitude-review-unmarkauto-libraries='aptitude -o "Aptitude::Pkg-Display-Limit=( ^lib !-dev$ !-dbg$ !-utils$ !-tools$ !-bin$ !-doc$ !^libreoffice | -data$ | -common$ | -base$ !^r-base ) !~M"'

And yes, this pattern is slightly larger than those from the previous posting, so here’s the used filter in a little bit more readable way:

(
  ^lib
    !-dev$
    !-dbg$
    !-utils$
    !-tools$
    !-bin$
    !-doc$
    !^libreoffice | 
  -data$ | 
  -common$ | 
  -base$
    !^r-base
)
!~M

It matches all non-automatically installed packages whose name starts with “lib”, but is neither a debug symbols package, a development header package, a documentation package, a package containing supplied commands, nor a libreoffice package.

Additionally it matches all non-automatically installed packages ending in -data, -common, or -base, but excludes r-base packages.

Of course you can then mark any erroneously unmarked library by pressing “M” (Shift-m).

If you press “g” for “Go” afterwards and wonder why nothing to remove shows up, be reminded that the filter limit is active in this view, too. So press “l” for “Limit” and then Ctrl-u to erase the current filter limit of this view and press enter to set the new (now empty) filter, et voilà…

Hope this is of help for some others, too.

Finding packages for deinstallation on the commandline with aptitude //at 20:18 //by abe

Although I often don’t agree with Erich (especially if GNOME is involved ;-), he recently posted something on Planet Debian which I found very helpful.

I also own a netbook where disk space is sacre. It’s an ASUS EeePC 701 with just 4GB disk space. And it runs Debian Sid, so dependencies change often, leaving packages installed which formerly had hard dependencies on, but are now left with just recommendations pointing to it.

Quite a few times I asked myself if it’s possible to find those packages and if so, how to do it. Well, I don’t have to ask myself that anymore, since Erich recently posted the appropriate filter patterns for my favourite package manager aptitude for this task in his posting “Finding packages for deinstallation”. Thanks, Erich!

Since those filters aren’t very easy to remember, I’d like to extend the usefulness of his posting towards the commandline. I for myself added the following aliases to my shell setup:

alias aptitude-just-recommended='aptitude -o "Aptitude::Pkg-Display-Limit=!?reverse-depends(~i) ~M !?essential"'
alias aptitude-also-via-dependency='aptitude -o "Aptitude::Pkg-Display-Limit=~i !~M ?reverse-depends(~i) !?essential"'

As youam suggested on IRC, I also added the filter !?essential since we won’t touch essential packages when cleaning up the list of installed packages anyway.

Hope this helps further.

Planet Commandline officially online //at 22:25 //by abe

Around the first bunch of postings in my Useful but Unknown Unix Tools, Tobias Klauser of inotail and Symlink fame came up with the idea of making a Planet (i.e. a blog aggregator) of all the comandline blogs and blog categories out there.

A first Planet Venus running prototype based on the template and style sheets of Planet Symlink was quickly up and running.

I just couldn’t decide if I should use an amber or phosphor green style for this new planet. Marius Rieder finally had the right idea to solve this dilemma: Offer both, an amber and a phosphor green style. Christian Herzog pointed me to the right piece of code at A List Apart. So here is it, available in you favourite screen colors:

Planet Commandline

For a beginning, the following feeds are included:

• Myon’s Unix blog category
• Evgeni Golov’s Desktop in a Shell column
• My Useful but Unknown Unix Tools column
• My Shell blog category
• The GRML Development Blog
• Chrütertee (Swiss-German)
• Commandline Magic’s identi.ca feed

Which leads us to the discussion what kind of feeds should be included in Planet Commandline.

Of course, all blogs or blog categories which (nearly) solely post neat tips and tricks about the command line in English are welcome.

Microblogging feeds containing (only) small but useful command line tips are welcome, too, if they neither permanently contain dozens of posts per day nor have a low signal-to-noise ratio. Unfortunately most identi.ca groups do, so they’re not suitable for such a planet.

What I’m though unsure about are non-English feeds. Yes, there’s one in already, but I noticed this only after including Beat’s Chrütertee and his FreeBSD command line tips are really good. So if it doesn’t go overboard, I think it’s ok. If there are too many non-English feeds, I’ll probably split Planet Commandline off into at least three Planets: One with all feeds, one with English only and one with all non-English feeds or maybe even one feed per language. But for now that’s still a long way off.

Another thing I’m unsure about are more propgram specific blogs like the impressive Mastering Emacs blog “about mastering the world’s best text editor”. *g* (Yeah, I didn’t include that one yet. But as soon someone shows me the vi-equivalent of that blog, I’ll include both. Anyone thinks, spf13’s vim category is up to that?)

Oh, and sure, any shell-specific (zsh, tcsh, bash, mksh, busybox) tips & tricks blogs don’t count as program-specific blogs like some $EDITOR, $BROWSER, or $VCS specific blogs do. :-)

Of course I’m happy about further suggestions for feeds to include in Planet Commandline. Just remember that the feed should provide (at least nearly) exclusively command line tips, tricks or howtos. Suggestions for links to other commandline related planets are welcome, too.

Cool new feature in OpenSSH 5.7: scp between two remote hosts //at 02:55 //by abe

Just a few days after OpenSSH 5.7 was released upstream, our (Debian’s as well as Ubuntu’s) tireless OpenSSH and GRUB maintainer Colin Watson uploaded a first package of OpenSSH 5.7 to Ubuntu Natty and to Debian Experimental.

Besides the obvious new thing, the implementation of Elliptic Curve Cryptography which promises better speed and shorter keys while staying at the same level of security, one other item of his changelog entry stuck out and caught my attention:

• scp(1): Add a new -3 option to scp: Copies between two remote hosts are transferred through the local host.

That’s something I always wondered why it didn’t “just work”. While it still doesn’t seem to detect such a situation by default, it’s now at least possible to copy stuff from on remote box to another without ugly port forwarding and tunneling hacks.

Further cool stuff in the changelog:

• sftp(1)/sftp-server(8): add a protocol extension to support a hard link operation. It is available through the “ln” command in the client. The old “ln” behaviour of creating a symlink is available using its “-s” option or through the preexisting “symlink” command.

Colin++