Friday·29·November·2013
PDiffs are still useful //at 03:26 //by abe
… probably just not as default.
I do agree with Richi and with Michael that disabling PDiffs by default gives the big majority of Debian Testing or Unstable users a speedier package list update.
I’m though not sure, if disabling PDiffs by default would
- also have an performance impact on our mirrors — it surely would have a traffic impact on the mirrors;
- really bring a benefit for Debian Stable users as Debian Stable changes seldomly and hence there are not that many PDiffs to download and apply — at least I can’t remember being annoyed by PDiffs anywhere else than on Debian Testing and Debian Unstable. Even the repositories with security updates don’t change that often.
Additionally I want to remind you that PDiffs per se are nothing bad and should be continued to be supported:
- Because there are still areas, even in “civilized” countries, where only small bandwidth is available and where using PDiffs reduces the download time a lot. Yes, also in Germany. BTDT. Only until recently there was no Fibre, no DSL, very bad UTMS reception and otherwise just EDGE at my parents’ home. (LTE was available far too expensive until recently.) And I was very happy about not having to download 30 MB or such just for seeing if there are updates at all, because 25 kB/s was the fastest download rate I could get (peaks, not average).
- Because it seems to be in fashion with big ISP near-to-monoplists, especially in Germany, to cut off your nice bandwidth if you transfer too many Megabytes. Keyword “Drosselkom”. If you happen to be a customer of such a shitty ISP, you may be happy to reduce your traffic amount by using PDiffs instead of downloading the full package list every time.
So yes, disabling PDiffs by default is probably ok, but the feature must be kept available for those who haven’t 100 MBit/s fibre connection into their homes or are sitting just one hop away from the next Debian mirror (like me at work :-).
Oh, and btw., for the very same reasons I’m also a big fan of debdelta which is approximately the same as PDiffs, just not for package lists but for binary packages. Using debdelta I was able to speed up my download rates over EDGE to up to virtual 100 kB/s, i.e. by factor four (depending on the packages). Just imagine a LibreOffice minor update at 15 kB/s average download rate. ;-)
And all these experiences were not made with a high-performance CPU
but with the approximately 5 year old Intel Atom processor of my ASUS
EeePC 900A. So I used PDiffs and debdelta even despite having a slight
performance penalty on applying the diffs and deltas.
Tagged as: APT, Aptitude, ASUS, debdelta, Debian, Discussion, DSL, EDGE, EeePC, nemo2, Other Blogs, PDiffs, Planet Debian, Sid, Testing, UMTS
// show without comments // write a comment
Related stories
Tuesday·26·November·2013
Showing packages newer than in archive with aptitude //at 22:14 //by abe
I happens quite often that I install a manually built, newer version of some package on a machine. Occassionally I forget to remove it or to downgrade it to the version in the APT repo.
$ apt-show-versions | fgrep newer
easily finds those packages.
But usually when doing such a check, I want this list of packages in my aptitude TUI to have a look at the other versions of that package and to take actions. And I don’t want to manually search for each of the package manually.
This can be done with the following “one-liner”:
# aptitude -o "Aptitude::Pkg-Display-Limit=( `apt-show-versions | fgrep newer | awk -F '[ :]' '{printf "~n ^"$1"$ | "}' | sed -e 's/| *$//'` )"
It uses apt-show-version
’s output, searches for the right
packages, takes the first column and transforms it into an aptitude
search pattern matching all packages whose name is exactly one of the
listed packages.
But this solution is quite ugly and slow. So I wondered if this is also doable with pure aptitude search patterns which likely would also be faster.
And after some playing around I found the following working aptitude search term:
~i ?any-version(!~O.) !~U !~o
This matches all packages which which are installed and which have a
version which has no origin, i.e. no associated APT repository. Since
this also matches all hold packages as well as all packages not
available in any archive, I use !~U !~o
to exclude those
packages from that list again.
Since nobody can remember that nor wants to type that everytime needed, I added the following alias to my setup:
alias aptitude-newer-than-in-archive='aptitude -o "Aptitude::Pkg-Display-Limit=~i ?any-version(!~O.) !~U !~o"'
Only caveat so far:
It seems to also match packages from APT repos which haven’t set an “Origin”. This should not happen with any Debian or Ubuntu APT repository, but seems to happen occasionally with privately run APT repositories.
And using ~A
instead of ~O
, i.e. ~i
?any-version(!~A.)
, does not work for this case either, despite
it matches installed packages of which versions not in any available
archive exist. But unfortunately aptitude seems to remember in some
way if a package was in some archive in the past, so this only shows
packages installed with dpkg -i
, but not packages removed
from e.g. unstable but with older versions still being available in
stable.
Tagged as: alias, apt-show-versions, aptitude, awk, CLI, Debian, filter, grep, one-liner, Package Management, Quoting, UUUCO
// show without comments // write a comment
Related stories
Next Debian Meetup in Zurich //at 20:16 //by abe
The first Debian meetup in Zurich last month was quite a success and I look forward to further Debian meetups in Zurich — every first Tuesday of the month.
The next meetup will be
on Tuesday, 2013-Dec-03 starting at 18:30 CET
at St. Gallerhof,
Konradstrasse 2, 8005 Zurich
Please note that this is a different location than last month.
Everybody who is interested in Debian is welcome to join us. Registering is not necessary.
There was also some interest in Debian meetups in other Swiss cities,
namely Bern and somewhere around Lac Leman. In case you want a Debian
meetup elsewhere in Switzerland, too, or if you’re interested in any
Debian meetup in Switzerland, feel free to join the Swiss
Debian Community Mailing List and help organising other Swiss
Debian meetups.
Tagged as: Bern, Berne, Debian, Debian ZH, Geneva, Genf, Local Group, Meeting, Meetup, Other Blogs, Planet Debian, Stammtisch, Switzerland, Zürich
// show without comments // write a comment
Related stories
Saturday·02·November·2013
Debian-Stammtisch in Zürich //at 12:49 //by abe
Wir (Michael Stapelberg und ich) wollen in Zürich einen regelmässigen Debian-Stammtisch, eine Debian-Lokalgruppe etablieren.
Das erste Treffen findet statt:
- am Dienstag, den 5. November 2013, ab 19 Uhr (CET)
- im Gloria, Josefstrasse 59, 8005 Zürich.
Jeder, der sich für Debian interessiert, ist eingeladen. Eine Anmeldung ist nicht notwendig.
Als regelmässigen Termin visieren wir den ersten Dienstag im Monat an.
Diesen Termin haben wir gewählt, weil er unseren Recherchen nach keine Terminkonflikte mit den Lokalgruppen der LUGS in Zürich und Winterthur, der FSFE in Zürich, dem CCC ZH, den Tuxeros oder dem Webtuesday gibt. In Kauf genommen haben wir Kollisionen mit wöchentlichen Treffs in der Hackerspaces Ruum42 in St. Gallen und Reaktor23 in Waldshut-Tiengen.
Es kam aber dennoch bereits die erste Meldung bzgl. potentieller Terminkonflikte rein. Wir diskutieren gerne noch über den Termin, sowohl am Stammtisch selbst, aber auch auf der Debian.ch Community-Mailingliste. Bevorzugte Sprache auf der Mailingliste ist Englisch, Deutsch ist aber auch in Ordnung.
Potentielle Terminänderungen werden primär auf der Debian.ch Community-Mailingliste und im Debian-Wiki auf der Lokalgruppen-Seite bekanntgegeben.
Die Termine wollen wir aber auch über die LUGS-Termine-Liste, die es auch als iCal oder E-Mail-Reminder gibt, publizieren. Danach sollten die Termine auch auf Freie Termine erscheinen.
Hui, ist das lange her, daß ich hier was auf Deutsch
geschrieben habe. :-)
Tagged as: CCC, CCCZH, Debian, Debian ZH, Freie Termine, FSFE, Gloria, Hackerspaces, Linux, Local Group, LUGS, Meeting, Meetup, Reaktor23, Regulars, Ruum42, sECuRE, Stammtisch, Tuxeros, Webtuesday, Winterthur, Zürich
// show without comments // write a comment
Related stories
Sunday·06·October·2013
Searching in Screen’s copy mode //at 23:43 //by abe
I’m using GNU Screen daily for definitely more than a decade and I became maintainer of Debian’s screen package nearly exactly two years ago. Nevertheless it still happens occassionally that I discover features yet unknown to me. Recently I had one of these moments again:
I looked for a specific line in the long output of a command which has run
inside a Screen session. For that I entered Screen’s copy mode with
Ctrl-A [
and scrolled around with arrow keys and page-up
and -down keys.
But didn’t find it. I thought, it would be cool if I can search for
the string I’m looking for. Intuïtively I typed /
,
the search string and pressed enter. And it worked! It jumped to the
next occurrence of that string.
Of course I immediately had to check if tmux has such a feature, too. And it indeed has, but it seems to be a less sophisticated implementation:
Feature | Key-binding in GNU Screen | Key-binding in Tmux |
---|---|---|
Switch into copy/scroll mode (needed for the remainder) |
Ctrl-A [ |
Ctrl-B [ |
Search for string once, forward | / + string + Enter |
Ctrl-S + string + Enter |
Search for string once, backward | ? + string + Enter |
Ctrl-R + string + Enter |
Search for string again, forward | / Enter |
Ctrl-S Enter |
Search for string again, backward | ? Enter |
Ctrl-R Enter |
Incremental search for string, forward | Ctrl-S + string |
- |
Incremental search for string, backward | Ctrl-R + string |
- |
(Incremental) search for next occurrence, forward | Ctrl-S again |
- |
(Incremental) search for next occurrence, backward | Ctrl-R again |
- |
Being able to do incremental search like with GNU Emacs gave me yet
another reason for continuing to use Screen and not to switch Tmux.
;-)
Tagged as: copy mode, feature, GNU Screen, screen, search, tmux
// show without comments // write a comment
Related stories
Wednesday·02·October·2013
How to make wget honour Content-Disposition headers //at 16:12 //by abe
Download links often point to CGI scripts which actually generate (or
just fetch, i.e. proxy) the actual file to be downloaded, e.g. URLs
like http://www.example.com/download.cgi?file=foobar.txt
.
Most of such CGI scripts send the real file name in the Content-Disposition
header as specified
in the MIME Specification.
All browsers I know (well, at least those I use regularily :-) handle that perfectly and propose the file name sent in the Content-Disposition header as file name for saving the downloaded name which is usually exactly what I want.
All browsers do that, …, just not my favourite commandline download tool GNU Wget … Downloading the above URL with wget would look like this with default settings:
$ wget 'http://www.example.com/download.cgi?file=foobar.txt' --2013-10-02 16:04:16-- http://www.example.com/download.cgi?file=foobar Resolving www.example.com (www.example.com)... 93.184.216.119, 2606:2800:220:6d:26bf:1447:1097:aa7 Connecting to www.switch.ch (www.example.com)|2606:2800:220:6d:26bf:1447:1097:aa7|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 2020 (2.0K) [text/plain] Saving to: `download.cgi?file=foobar.txt' 100%[============================================>] 2,020 --.-K/s in 0s 2013-10-02 16:04:24 (12.5 MB/s) - `download.cgi?file=foobar.txt' saved [2020/2020]
Meh!
But luckily Wget can do that, it’s just not enabled by default — because it’s an experimental and possibly buggy feature, at least according to the man page. Well, works for me! :-)
You can easily enabled it by default for either your user or the whole
system by placing the following line in your ~/.wgetrc
or /etc/wgetrc
:
content-disposition = on
Given the CGI script sends an appropriate Content-Disposition header, the above output now looks like this:
$ wget 'http://www.example.com/download.cgi?file=foobar.txt' --2013-10-02 16:04:16-- http://www.example.com/download.cgi?file=foobar Resolving www.example.com (www.example.com)... 93.184.216.119, 2606:2800:220:6d:26bf:1447:1097:aa7 Connecting to www.switch.ch (www.example.com)|2606:2800:220:6d:26bf:1447:1097:aa7|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 2020 (2.0K) [text/plain] Saving to: `foobar.txt' 100%[============================================>] 2,020 --.-K/s in 0s 2013-10-02 16:04:24 (12.5 MB/s) - `foobar.txt' saved [2020/2020]
Now Wget does what I mean!
You can also set this as flag on the commandline, but typing
wget --content-disposition …
everytime is surely
not what I want. ;-)
Tagged as: CGI, CLI, Content-Disposition, download, howto, HTTP, Shell, UUUCO, wget
// show without comments // write a comment
Related stories
Thursday·02·May·2013
New web browsers in Wheezy //at 16:14 //by abe
Since there is so much nice new stuff in Debian Wheezy, I have to split up my contributions to Mika’s #newinwheezy game on Planet Debian.
Here’s the next bunch, this time web browsers:
- dillo
- The FLTK-based lightweight GUI web browser Dillo comes with its own rendering engine (no JavaScript, incomplete CSS support) was already in Debian before, but was removed before the release of Debian Squeeze, because Dillo 2 relied on FLTK 2.x which had an unclear license situation back then and never made it into Debian. In the meanwhile Dillo 3 relies on FLTK 1.3 as FLTK upstream abandoned the 2.0 branch and continued development on the 1.3 branch. So I brought Dillo back into Debian with its 3.0.x release.
- netsurf
- The RiscOS-originating lightweight GUI web browser Netsurf was already in Debian, too, but didn’t make it into Debian Squeeze as it needed the Lemon parser generator (part of the SQLite source) to build back then and a change in Lemon caused Netsurf to no more build properly in the wrong moment. Netsurf supports CSS 2.1, but has no JavaScript support either. I’d consider its rendering engine more complete than Dillo’s.
- surf and xxxterm
- Surf and XXXTerm are both simple and minimalistic webkit-based browsers. Surf is easy to embed in other applications and XXXTerm features vi-like keybindings for heavy keyboard users.
To be continued… ;-)
Tagged as: Debian, dillo, netsurf, newinwheezy, surf, webbrowser, Wheezy, xxxterm
// show without comments // write a comment
Related stories
New SSH-related stuff in Wheezy //at 15:28 //by abe
Mika had the nice idea of doing a #newinwheezy game on Planet Debian, so let’s join:
There are (at least) two new SSH related tools new in Debian Wheezy:
- mosh
- is the “mobile shell”, an UDP based remote shell terminal which works better than SSH in case of lag, packet loss or other forms of bad connection. I wrote about mosh in more detail about a year ago. mosh is also available for Debian Squeeze via squeeze-backports.
- sshuttle
- is somewhere between port-forwarding and VPN. It allows forward arbitrary TCP connections over an SSH connection without the need to configure individual port forwardings. It does not need root access on the server-side either. I wrote about sshuttle in more detail about a year ago.
To be continued… ;-)
Tagged as: Debian, mika, mosh, newinwheezy, Planet Debian, SSH, sshuttle, Wheezy
// show without comments // write a comment
Related stories
Sunday·10·March·2013
Rendering Markdown, Asciidoc and Friends automatically while Editing //at 15:41 //by abe
Partially because of Markdown being Github’s markup format of choice, I enjoy writing documents in simple markup formats more and more.
There’s though one common annoyance with these formats compared to writing plain HTML…
The Annoyance
They need to be rendered (i.e. more or less compiled) before you can view your outpourings rendered, e.g. in the web browser. So the workflow usually is:
- Saving the current file in your favourite editor
- Switch to terminal with commandline
- Cursor up, Enter
- Switch to your favourite web browser
- Hit the reload button
Using a Specialized Editor with Live Preview
One choice would be to use a specific editor with live rendering. The one I know in Debian (from Wheezy on) is ReText (Debian package retext). It supports Markdown and reStructuredText.
But as with most simple GUI editors, I miss there many of the advanced editing commands possible with Emacs.
Using Emacs’ Markdown Mode
Then there is the Markdown Mode
for Emacs (part of Debian’s emacs-goodies-el package), where
you can get a “preview” by pressing C-c C-c p
. But for
some reason this takes several seconds, opens a new buffer
and window with the rendered HTML code and then starts
(hardcoded) Firefox (which is not my preferred web browser). And if you do that a
second time without closing Firefox first, it won’t just reload the
file but will open a new tab. You might think that just hitting reload
should suffice. But no, the new tab has a different file name, so
reload doesn’t help. Additionally it may not use my preferred Markdown
implementation. Meh.
Well, I probably could fix all those issues with Markdown Mode, it’s only Emacs Lisp. Heck, the called command is even configurable. But fixing at least four issues to fix one workflow annoyance? Maybe some other time, but not as long there are other nice choices…
Using inotifywait to Render on Write
So everytime you save the currently edited file, you immediately want to rerender the same HTML file from it. This can be easily automated by using Linux’ inotify kernel subsystem which notices changes to the filesystem, and reports those to applications which ask for it.
One such tool is inotifywait
which can either output all
or just specific events, or just exit if the first requested event
occurs. With the latter it’s easy to write a while loop on the
commandline which regenerates a file after every write access. I use
either Pandoc or Asciidoc for that since both generate full HTML pages
including header and footer, but you can use that also with Markdown
to render just the HTML body. Most browsers render it correctly
anyway:
while inotifywait -q -e modify index.md; do pandoc -s -f markdown -t html -o index.html index.md; done while inotifywait -q -e modify index.txt; do asciidoc index.txt; done while inotifywait -q -e modify index.md; do markdown index.md > index.html; done
This solution is even editor- and build-system-agnostic (But not operating-system-agnostic.)
inotifywait is part of inotify-tools, a useful set of commandline tools to interface with inotify. They’re packaged in Debian as inotify-tools, too.
Using mdpress for Markdown plus Impress.js based Slides
The ruby-written mdpress is a special case of the previous case. It’s
a commandline tool to convert Markdown into Impress.js based slide
shows and it has an option named --automatic
which causes
it to keep running and automatically update the presentation as soon
as changes are made to the Markdown file.
mdpress is not yet in Debian, but there’s an ITP for it and
Impress.js itself recently entered Debian as libjs-impress.
Nevertheless, two dependencies (highlight.js,
ITP‘ed, ruby-launchy, ITP‘ed) are still missing in Debian.
Tagged as: Asciidoc, Emacs, emacs-goodies-el, GitHub, HTML, Impress.js, inotify, inotify-tools, inotifywait, ITP, Major-Mode, Markdown, mdpress, oneliner, Pandoc, reST, ReText, Ruby, slides, Wheezy
// show without comments // write a comment
Related stories
Up to date Aptitude Documentation Online //at 12:51 //by abe
Aptitude ships documentation in 7 languages as HTML files. However the latest version available online was 0.4.11.2 from 2008 and hosted on the server by the previous, now unfortunately inactive Aptitude maintainer, and only covered 5 languages.
This lack of up to date online documentation even caused others to put more up to date versions online. Nevertheless they age, too, and the one I’m aware is not up to date for Wheezy.
So the idea was born to keep an up to date version online on Aptitude’s Alioth webspace (which currently redirects to a subdirectory of the previous maintainer’s personal website). But unfortunately we, the current Aptitude Team, are still lacking administrative rights on Aptitude’s Alioth project, which would be necessary to assign new team members who could work on that.
As an intermediate step, there’s now a (currently ;-) up to date Aptitude User’s Manual online in all 7 languages at
http://people.debian.org/~abe/aptitude/
and English at
http://people.debian.org/~abe/aptitude/en/
As this location could also suffer from the same MIA issues as any other “personal” copy, the plan is to move this to somewhere under http://aptitude.alioth.debian.org/ as soon as we have full access to Aptitude’s Alioth project.
Our plans for then are:
- Redirects from http://people.debian.org/~abe/aptitude/ to e.g. http://aptitude.alioth.debian.org/doc/, so that all your links to or bookmarks of http://people.debian.org/~abe/aptitude/ are still valid. (This unfortunately won’t work for jump marks to specific sections, just per file, i.e. chapter.)
- Set up a cron-job, which keeps the documentation in sync with the version of Aptitude in Unstable (and maybe also with Aptitude in Stable).
P.S.: Anyone interested in doing a German translation of the Aptitude
User’s Manual? Sources are in DocBook, i.e. XML, and available via Git.
Tagged as: Alioth, aptitude, Debian, documentation, link, online
// show without comments // write a comment