How to make wget honour Content-Disposition headers //at 16:12 //by abe

Download links often point to CGI scripts which actually generate (or just fetch, i.e. proxy) the actual file to be downloaded, e.g. URLs like http://www.example.com/download.cgi?file=foobar.txt.

Most of such CGI scripts send the real file name in the Content-Disposition header as specified in the MIME Specification.

All browsers I know (well, at least those I use regularily :-) handle that perfectly and propose the file name sent in the Content-Disposition header as file name for saving the downloaded name which is usually exactly what I want.

All browsers do that, …, just not my favourite commandline download tool GNU Wget … Downloading the above URL with wget would look like this with default settings:

$ wget 'http://www.example.com/download.cgi?file=foobar.txt'
--2013-10-02 16:04:16--  http://www.example.com/download.cgi?file=foobar
Resolving www.example.com (www.example.com)... 93.184.216.119, 2606:2800:220:6d:26bf:1447:1097:aa7
Connecting to www.switch.ch (www.example.com)|2606:2800:220:6d:26bf:1447:1097:aa7|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2020 (2.0K) [text/plain]
Saving to: `download.cgi?file=foobar.txt'

100%[============================================>] 2,020       --.-K/s   in 0s

2013-10-02 16:04:24 (12.5 MB/s) - `download.cgi?file=foobar.txt' saved [2020/2020]

Meh!

But luckily Wget can do that, it’s just not enabled by default — because it’s an experimental and possibly buggy feature, at least according to the man page. Well, works for me! :-)

You can easily enabled it by default for either your user or the whole system by placing the following line in your ~/.wgetrc or /etc/wgetrc:

content-disposition = on

Given the CGI script sends an appropriate Content-Disposition header, the above output now looks like this:

$ wget 'http://www.example.com/download.cgi?file=foobar.txt'
--2013-10-02 16:04:16--  http://www.example.com/download.cgi?file=foobar
Resolving www.example.com (www.example.com)... 93.184.216.119, 2606:2800:220:6d:26bf:1447:1097:aa7
Connecting to www.switch.ch (www.example.com)|2606:2800:220:6d:26bf:1447:1097:aa7|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2020 (2.0K) [text/plain]
Saving to: `foobar.txt'

100%[============================================>] 2,020       --.-K/s   in 0s

2013-10-02 16:04:24 (12.5 MB/s) - `foobar.txt' saved [2020/2020]

Now Wget does what I mean!

You can also set this as flag on the commandline, but typing wget --content-disposition … everytime is surely not what I want. ;-)

Planet Commandline officially online //at 22:25 //by abe

Around the first bunch of postings in my Useful but Unknown Unix Tools, Tobias Klauser of inotail and Symlink fame came up with the idea of making a Planet (i.e. a blog aggregator) of all the comandline blogs and blog categories out there.

A first Planet Venus running prototype based on the template and style sheets of Planet Symlink was quickly up and running.

I just couldn’t decide if I should use an amber or phosphor green style for this new planet. Marius Rieder finally had the right idea to solve this dilemma: Offer both, an amber and a phosphor green style. Christian Herzog pointed me to the right piece of code at A List Apart. So here is it, available in you favourite screen colors:

Planet Commandline

For a beginning, the following feeds are included:

• Myon’s Unix blog category
• Evgeni Golov’s Desktop in a Shell column
• My Useful but Unknown Unix Tools column
• My Shell blog category
• The GRML Development Blog
• Chrütertee (Swiss-German)
• Commandline Magic’s identi.ca feed

Which leads us to the discussion what kind of feeds should be included in Planet Commandline.

Of course, all blogs or blog categories which (nearly) solely post neat tips and tricks about the command line in English are welcome.

Microblogging feeds containing (only) small but useful command line tips are welcome, too, if they neither permanently contain dozens of posts per day nor have a low signal-to-noise ratio. Unfortunately most identi.ca groups do, so they’re not suitable for such a planet.

What I’m though unsure about are non-English feeds. Yes, there’s one in already, but I noticed this only after including Beat’s Chrütertee and his FreeBSD command line tips are really good. So if it doesn’t go overboard, I think it’s ok. If there are too many non-English feeds, I’ll probably split Planet Commandline off into at least three Planets: One with all feeds, one with English only and one with all non-English feeds or maybe even one feed per language. But for now that’s still a long way off.

Another thing I’m unsure about are more propgram specific blogs like the impressive Mastering Emacs blog “about mastering the world’s best text editor”. *g* (Yeah, I didn’t include that one yet. But as soon someone shows me the vi-equivalent of that blog, I’ll include both. Anyone thinks, spf13’s vim category is up to that?)

Oh, and sure, any shell-specific (zsh, tcsh, bash, mksh, busybox) tips & tricks blogs don’t count as program-specific blogs like some $EDITOR, $BROWSER, or $VCS specific blogs do. :-)

Of course I’m happy about further suggestions for feeds to include in Planet Commandline. Just remember that the feed should provide (at least nearly) exclusively command line tips, tricks or howtos. Suggestions for links to other commandline related planets are welcome, too.

Useful but Unknown Unix Tools //at 01:04 //by abe

Most of my talks are either talks about commandline basics (Commandline Helpers about GNU Coreutils and Findutils usage; Understanding Shell Quoting) or collections of small tips and tricks, e.g. about SSH, History Expansion, or commandline tools. Valentin Haenel called this second type of talks “nugget talks”. I like that term.

My newest talk (German only so far, next occasion at the LUGS meeting in Zurich on 16th of December 2010) is such a nugget talk and is about useful but unknown tools. While preparing that talk I noticed that the amount of neat but less known tools I collected for possible inclusion into that talk would be enough for a whole day or so. I noticed that I’ll never be able to give a talk about all that stuff.

To not letting these tips rot in that file uncommented, I decided that I’ll write (more or less) short blog postings about these tools, similar to and inspired by Myon’s blog postings about Cool Unix Features as well as the no more existing or at least broken Debian Package of the Day and Debian Package a Day.

While Myon mostly posted long-time but still unknown features, my postings will mostly cover useful but unknown packages available in Debian and most debian-derived distributions like grml and Ubuntu. Some postings may also just highlight some features of common tools or packages like e.g. GNU Coreutils.

You can subscribe to these tips via RSS or just view all of them in your web browser at http://noone.org/blog/English/Computer/Debian/CoolTools/.

Multiple Move & Co. //at 00:08 //by abe

nion’s blog made me notice that many people don’t know mmv (multiple move), which approximately works like this:

mmv '*.htm' #1.html
mmv '*.foo.*' #1.#2.bla

Additionally, mmv also can copy, link or even append files when called as mcp, mln or mad respectively with the appropriate command line options.

When I told nion in IRC on #debian.de about mmv, HE pointed me to the Perl script /usr/bin/rename, which is in Debian’s perl package and therefore installed on nearly every Debian system by default. It moves files by applying perl subsitutions to file names:

rename 's/\.htm$/.html/' *.htm
rename 'y/A-Z/a-z/' *

Being curious, if the newly found tool is not only available in Debian, I looked on a SuSE 9.0 box and indeed, I also found there a /usr/bin/rename. But — surprise, surprise — it’s not a Perl script but an ELF binary. And although it does similar things than mmv and Debian’s rename, it is the simplest of the three commands:

rename .htm .html *.htm
rename foo foo00 foo?
rename foo foo0 foo??

Note to my self: Nice add-on for your command line efficiency talk.

BarCamp Zurich — Resumé //at 02:02 //by abe

The BarCamp Zurich 2006 is over. On the way there I thought about what I would do during time slots with no interesting talks. But when I tried to make up my personal schedule, I noticed that I rather would have the opposite problem: Too many interesting talks at the same time… Well, to many interesting talks at all, although I only went to tech talks and left out the biz talks.

I first went to the Podcasting & Co. talk by Timo Hetzel, since I never heard or made a podcast, but was curious about podcasts in general. Besides statistics and rankings he spoke about where people listen to podcast (most listeners seem to do that during commuting), what people like in podcasts, why companies podcast, etc. And that a very big share of all podcast listeners use iTunes as podcast client and except juice (never heard of it before) all other podcast clients seem to be irrelevant.

My conclusion: I haven’t missed anything not having listened to or made podcasts neither do I need to listen or make podcasts in the future. They’re irrelevant. To me. :-)

Then I had to choose between the talks AJAX@localhost (PDF) by Harry Fuecks and Realtime Collaborative Text Editing and SubEthaEdit by the Coding Monkeys. I heard about realtime collaborative editing once know that it’s a challenging task for the developer. I also know what AJAX is (and that I would only use or recommend it for bells and whistles, but not for content in general), but “AJAX@localhost” sounded like writing normal applications using AJAX. It sounded interesting and evil at the same time. I had to go there! ;-) Others had similar expectations after reading the talk’s title, so I was quite surprised that it was about something completely different, namely about debugging AJAX on the localhost but under conditions usually only appearing if you’re running AJAX application not from localhost but from somewhere on the net: You may have different lags with every request, so some requests may reach the server before others, which may screw up the whole AJAX application, if the developers didn’t think about it and only tested it on localhost. (Hence the talk’s title…)

My conlusion: I will use and recommend AJAX even more seldom, since there seem to be even more design misconceptions than I thought before. But I’ll once have a look at the Webtuesday meeting, he mentioned.

For the third time-slot, I didn’t need long to decide where to go: I already knew a little bit about Microformats and I wanted to know more. Tag Trade also sounded interesting, but the second part of the talk’s title, Paid Learning sounded like business and so I had no scruples to cold-shoulder that talk. I probably didn’t learn anything really new in the microformats talk, but my knowledge about microformats is now more concrete, and after talking with Cédric Hüsler later during a break, I would even trust myself to start and define a new microformat.

Then I went to the HG Caféteria together with Gürkan and two German guys. While waiting in the queue, we were talking about our jobs and our favourite Linux distributions. I got some rhubarb pie and a rum truffles, assuming that the Caféteria uses no alcohol in their products like all other SV restaurant I know. But this one seemed to have quite a lot of alcohol, since it felt like my breath was burning… Well, this resulted in my second SV feedback form submission…

Next I went to Alex Schröder’s talk about multilingual websites, Oddmuse and the Emacs Wiki, although also the talk A-Life about simulating evolution sounded promising. Alex asked the listeners about their experiences with multilingual websites and showed what Oddmuse offers as partial solution to the general multilingualism problems. But regarding the comments from the auditorium, there probably won’t be a perfect solution until computers can translate perfectly…

The next talk I visited was Gabor’s talk about his master thesis Organizing E-Mail which resulted in a soon to be released Mozilla Thunderbird extension called BuzzTrack. From the other concepts he showed, I found Microsoft’s SNARF (Social Network and Relationship Finder) and IBM’s Thread Arcs most interesting as well as the fact that there is no e-mail client seems to have a majority at all.

Directly after Gabor I had my own talk about Understanding Shell Quoting, so I also couldn’t go to Adrian Heydecker’s talk about Learning with Hypertext and Search Engines. I had only about three and a half listeners of whom several to my surprise where here because they didn’t know what “shell quoting” is.

I really didn’t expect that.

But that seems to be one of the differences between a BarCamp and a Linux Conferences: People come here to see something new, something they haven’t heard about before. On Linux events most people come, because they already heard about some special topic and want to know more or learn something about it. On Linux event my shell talks usually were attracting many visitors while at a BarCamp, talks presenting an idea, a concept or a tool seem to much more interesting for the attendees. So for the next BarCamp I perhaps exhume my Website Meta Language talk which never seemed to hit the nerve of Linux event attendees, since it tried to “sell” a different concept of generating website than most were used to.

At least one listener excepted the talk to be named “shell escaping”, but IMHO escaping is only one quoting technic and it’s not only used for quoting. But perhaps I should take the word “escaping” in the title though for the next time.

Happily most of the listeners seem to have learned something new from the talk and Silvan Gebhardt was really happy about his new knowledge about ssh ~ escapes, although I mainly talked about how to quote them than how to use them. :-)

During the last slot I visited the session about the upcoming BarCamp Alsace 2 and the yet to be planned BarCamp Rhine, a BarCamp to be held on a ship traveling from Basel in Switzerland down the Rhine, stopping in Strasbourg, Karlsruhe, Rhein-Main-Area and perhaps even Cologne and Amsterdam.

Contrary to my initial thoughts, the day was over very fast and I had no single boring minute during the BarCamp. Wow!

After we’ve been kicked out of the building by ETH janitors, we joined again at the Bar N-68. On the way there I met Urban Müller who attended BarCamp Zurich, too. We talked quite a lot and it was very interesting to see behind the scenes of e.g. map.search.ch. Later I joined the French speaking table, talking with Gregoire Japiot from WineCamp France and Alex Schröder.

Around 9pm I left the N-68 as one of the last BarCampers, tired but with new knowledge, new ideas, new acquaintances and a new hobby: BarCamping. What a luck that BarCamps aren’t that often, otherwise I couldn’t afford this new hobby. ;-)

As a relaxing end I met with Alex Schröder and Christophe Ducamp on Sunday morning for brunch in the restaurant Gloria in the Industriequartier. When we were leaving the Gloria I noticed their book board with a lots of BookCrossing books and I took “The Da Vinci Code” with me, since I saw the movie and people were telling me that the book is much better. I’ll see…

Next Shell Quoting Talks //at 12:33 //by abe

There are a several events coming up where I plan to hold my Shell Quoting Talk: First, there will be the BarCamp Zurich on October, the 28th at ETH Zürich HG and then there will be the 8th Linuxday.at on November, the 18th at the HTL at Dornbirn (Vorarlberg, Austria) organised by the LUG Vorarlberg. It’s also possible that, in addition to the Shell Quoting talk, I’ll also give a talk for beginners about Commandline Helpers. (Probably all the talks will be held in German.)

I’m quite curious on both events, for very different reasons. On the one hand, a BarCamp is something completely new for me and it sounds like a very interesting mixture of a real life Wikipedia meeting and a flash mob to me.

On the other hand, this year’s Linuxday.at will have several new facettes for me: First there were several changes in the organising team, so I wonder if and in that case how much this will change the face of the event. Then it’s the first Linxuday.at since I live in Zurich, which means it’s the first Linuxday without 1000km travelling during that weekend, so I also have some time to meet friends in the area in advance to or after the event. Yeah!

Nice Shell Bloomer //at 16:39 //by abe

While looking for users which still have “.” in their path, I found the following nice bloomer:

PATH=``$PATH:.:$HOME/bin''

It’s obvious what the user tried to do. But why the fuck does this (more or less man or info page alike) quoting syntax work?

It took me a moment to realise that this kind of “quoting” works in nearly all Unix shells: The two backquotes as well as the two single quotes become an empty string and are therefor completely useless in this case.

The user probably read some uglily localized man or info page (like the German ones in Debian Sarge) and did some copy and paste to his .bashrc. And since it “worked” he didn’t see any reason to change it again.