automounter vs procmail //at 00:10 //by abe

At work we use .procmailrc files generated by CGIpaf to let non-technical users create forwards, out-of-office mails, etc. and any combination thereof. This also has the advantage that we can filter out double bounces and spam (which also prevents us from being listed in spammer blacklists).

Unfortunately autofs (seems independent if autofs4 or autofs5 is used) seems to be unreliable if there are bursts of mount or umount requests, resulting either in “File or directory not found” error message while trying to access the home directory of a user, or “Directory not empty” error messages if the automounter tries to remove the mount point after unmounting. In that case a not mounted directory owned by root is left over.

In the end both cases lead to procmail behaving as if that user does not have a .procmailrc – which looks like sporadically lost mails to those who forward all mails. (The mails then can be found in the local default INBOX for that user.)

Additionally there are similar issues when the NFS servers are not available.

The most effective countermeasure we found so far was adding tests to the global /etc/procmailrc to check if the user’s home directory exists and belongs to the correct user:

# -----------------
# Global procmailrc
# -----------------

# For debugging, turn off if everything works well
VERBOSE=1
LOGFILE=/var/log/procmail.log

# This only works with bourne shells, $SHELL defaults to the user's
# login shell. And by experience dash seems not work, so we use bash.
OLDSHELL=$SHELL
SHELL=/bin/bash

# temporary failure (see EX_TEMPFAIL in /usr/include/sysexits.h) if
# $LOGNAME is not set for some reason. (Just to be sure our paths
# later on are not senseless.
:0
* ? test -z "$LOGNAME"
{
    LOG="Expected variable LOGNAME not set. "
    EXITCODE=75
    :0
    /dev/null
}

# temporary failure (see EX_TEMPFAIL in /usr/include/sysexits.h) if
# $HOME is not readable. ~$LOGNAME does not seem to work, so this uses
# a hard wired /home/.
:0
* ? test ! -r /home/$LOGNAME
{
    LOG="Home of user $LOGNAME not readable: /home/$LOGNAME "
    EXITCODE=75
    :0
    /dev/null
}

# temporary failure (see EX_TEMPFAIL in /usr/include/sysexits.h) if
# $HOME has wrong owner. ~$LOGNAME does not seem to work, so this uses
# a hard wired /home/.
:0
* ? test ! -O /home/$LOGNAME
{
    LOG="Home of user $LOGNAME has wrong owner: /home/$LOGNAME "
    EXITCODE=75
    :0
    /dev/null
}

[…]

If you want to store a copy of these mails for debugging purposes on every delivery attempt, replace /dev/null with some Maildir or mbox only accessible for root.

One small but important part was to explicitly declare bash as shell for executing the tests, otherwise mails for users with tcsh or zsh as login shell filled up the mail queue and never get delivered (if the SHELL variable never gets fixed).

Only drawback so far: This leads to more lagging e-mail on e-mail bursts also for those users who have no .procmailrc – because procmail can’t check if there’s really no .procmailrc.

Extensive procmail documentation can be found online at the Procmail Documentation Project as well as in the man pages procmail(1), procmailrc(5) and procmailex(5).

grep everything //at 09:43 //by abe

During the OpenRheinRuhr I noticed that a friend of mine didn’t know about zgrep and friends. So I told him what other grep variations I know and he told me about some grep variations I didn’t know about.

So here’s our collection of grep wrappers, derivatives and variations. First I’ll list programs which search for text in different file formats:

And then there are also greps for special patterns on more or less normal files:

One question is though still unanswered for us: Is there some kind of meta-grep which chooses per file the right grep from above by looking at the MIME type of the according files, similar to xdg-open.

Other tools which have grep in their name, but are too special to properly fit into the above lists:

• ext3grep: Tool to help recover deleted files on ext3 filesystems
• xautomation: Includes a tool named visgrep to grep for subimages inside other images.

Includes contributions by Frank Hofmann and Faidon Liambotis.

Screen and Emacsclient: Automatically switching to the Emacs window //at 10:38 //by abe

For a very long time, I use mutt with emacsclient as configured editor and a single GNU Emacs instance started from either .screenrc or .Xsession, depending on the system. And I’m very used to switching the virtual desktop or the screen window after starting a mail in mutt.

Since Debian 5.0 Lenny and Emacs 22, Emacs automatically grabs the focus and switches to the right virtual desktop. So after telling mutt recipient and subject of a new e-mail, it invokes emacsclient and immediately the focus has moved to the running Emacs instance. Because I was used to switch one virtual desktop to the right at that point, I often found my self two desktops to the right until I got used to it. :-)

I usually hate applications which grab the focus without being asked. But in this case I basically asked for it. And there’s no delay like with starting up an application which has to read in some database first – think of Liferea or Rhythmbox which take many seconds to minutes to start up, even on my 2.2 GHz dual core ThinkPad.

In the meantime I got so used to that automatic desktop switch that I forget to switch the screen window in the second scenario where I use this combination: My screen doesn’t automatically switch to the Emacs window (window 1) after I told mutt recepient and subject in window 2.

Knowing that screen is quite scriptable, I found out that only a very small change is needed to my mutt configuration to get that desktop feature to my everyday screen session. I simply replaced the editor setting in my .muttrc with the following line:

set editor="screen -X select 1;emacsclient"

Now mutt tells screen to switch to window 1 (where Emacs is running) and then tells Emacs to open the appropriate file to edit my new mail.

Update Friday, 2009-04-24, 18:22

Even though Zack surely is right with his comment about the multi-terminal feature of the upcoming GNU Emacs 23, I still have Etch (and therefore GNU Emacs 21) on the server where I have my screen session.

So the next step was to switch back to the mutt window (window 2) after I’m finished with editing the mail. Since mutt gives the the file to edit as argument to the contents of $editor, simply adding ;screen -X select 2 at the end of $editor doesn’t suffice.

So I wrote a small shell script (named ~/.mutt/editor.sh) as wrapper which calls all the commands and passes the parameters to the right command:

#!/bin/sh

screen -X select 1
emacsclient -a ~/.mutt/alteditor.sh "$@"
screen -X select 2

Of course, $editor is now set to that script:

set editor="/home/abe/.mutt/editor.sh"

Emacsclient of GNU Emacs 21 already supports the -a option to call another editor in case of not being able to connect to a running Emacs instance. Since I don’t want to switch to another screen window in that case, I wrote a second shell script (named ~/.mutt/alteditor.sh) which switches back to the mutt window and then calls GNU Zile, my preferred low-end emacs clone:

#!/bin/sh

screen -X select 2
zile "$@"
screen -X select 1

I love it!

New virtual home //at 03:10 //by abe

There were quite a lot of small changes in my personal e-mail, web server and DNS infrastructure during the last months. While for more than ten years all my mails came together on a Linux account at the Students Representatives of Computer Science (German: Fachschaftsrat Informatik, short: FS Info) at the University of Saarland and all but one domain was hosted at my former employer, now nearly everything (including this blog) has moved to a Hetzner root server run by me and some friends and running Debian Sarge for AMD64. Most of these moves happend divided in small steps during January and February this year.

Only one move happend a little earlier than expected: Without any notice all external mail to my FS Info account got lost in a black hole one hop in front of the FS Info mail server. It looks like the RBG of the Dept. of CS at the University of Saarland didn’t like my face —eh— mails anymore and just dropped them. No bounces, no notices. Not that I knew them as friendly and kind, but blackholing active e-mail accounts without notice leads to dismissal at other places. They should really go and hear alphascorpii’s BOFH talk.

So my long-time e-mail address abe@fsinfo.cs.uni-sb.de is no more valid and any mail there won’t get read or answered. But I don’t mind if you send any junk there, it’s not my job anymore.

For luck the hoster of my deuxchevaux.org domain, Internett, which I used for all non-university e-mail, changed the forward within a good hour, so I only lost incoming mails of about one and a half day (which was the time until I noticed the blackholing), namely most mails written to all of my addresses at 30th and 31st of January 2007. If you wrote me around that time and miss the answers, you now know why. The few people I expected mail from, already have been contacted.

So for the future, write emails only to abe@ either deuxchevaux.org or — if you can’t remember how to spell that (no, you’re not the only one ;-) — noone.org (“no one” as in “nobody”, not “no. one” as in “the best” ;-)

And so since then none of my coworkers can make fun of me because I don’t have root on the box where my mail is received. ;-)

One of the moves also gave me the possibility to get “my own” Jabber server, so the time with those two jabber addresses (abe@jabber.noxa.de and abe@amessage.de), where usually at least one had some problems with some gateways are also over. If you want to contact me using Jabber, use abe@noone.org, equal to my easier to remember e-mail address. ;-)

So there’s still left to move: web, mail and DNS of deuxchevaux.org and beckert.at (domain of my parents) and my old web pages on fsinfo.cs.uni-sb.de as well as Planet Symlink. And of course the redesign of my web pages, which I’ve planned since more than a half decade… ;-)

The infrastructure in my real-life home also changed: My gateway to the world is now a 266 MHz MIPS based ASUS WL500g Premium (which I named “pluriel” after the multifunctional Citroën C3 Pluriel) running FreeWRT 1.0.1 which replaced tryane, an as noisy (power supply fan) as compact Unisys Aquanta CP mini-desktop with a passively cooled 200 MHz Pentium MMX.

BarCamp Zurich — Resumé //at 02:02 //by abe

The BarCamp Zurich 2006 is over. On the way there I thought about what I would do during time slots with no interesting talks. But when I tried to make up my personal schedule, I noticed that I rather would have the opposite problem: Too many interesting talks at the same time… Well, to many interesting talks at all, although I only went to tech talks and left out the biz talks.

I first went to the Podcasting & Co. talk by Timo Hetzel, since I never heard or made a podcast, but was curious about podcasts in general. Besides statistics and rankings he spoke about where people listen to podcast (most listeners seem to do that during commuting), what people like in podcasts, why companies podcast, etc. And that a very big share of all podcast listeners use iTunes as podcast client and except juice (never heard of it before) all other podcast clients seem to be irrelevant.

My conclusion: I haven’t missed anything not having listened to or made podcasts neither do I need to listen or make podcasts in the future. They’re irrelevant. To me. :-)

Then I had to choose between the talks AJAX@localhost (PDF) by Harry Fuecks and Realtime Collaborative Text Editing and SubEthaEdit by the Coding Monkeys. I heard about realtime collaborative editing once know that it’s a challenging task for the developer. I also know what AJAX is (and that I would only use or recommend it for bells and whistles, but not for content in general), but “AJAX@localhost” sounded like writing normal applications using AJAX. It sounded interesting and evil at the same time. I had to go there! ;-) Others had similar expectations after reading the talk’s title, so I was quite surprised that it was about something completely different, namely about debugging AJAX on the localhost but under conditions usually only appearing if you’re running AJAX application not from localhost but from somewhere on the net: You may have different lags with every request, so some requests may reach the server before others, which may screw up the whole AJAX application, if the developers didn’t think about it and only tested it on localhost. (Hence the talk’s title…)

My conlusion: I will use and recommend AJAX even more seldom, since there seem to be even more design misconceptions than I thought before. But I’ll once have a look at the Webtuesday meeting, he mentioned.

For the third time-slot, I didn’t need long to decide where to go: I already knew a little bit about Microformats and I wanted to know more. Tag Trade also sounded interesting, but the second part of the talk’s title, Paid Learning sounded like business and so I had no scruples to cold-shoulder that talk. I probably didn’t learn anything really new in the microformats talk, but my knowledge about microformats is now more concrete, and after talking with Cédric Hüsler later during a break, I would even trust myself to start and define a new microformat.

Then I went to the HG Caféteria together with Gürkan and two German guys. While waiting in the queue, we were talking about our jobs and our favourite Linux distributions. I got some rhubarb pie and a rum truffles, assuming that the Caféteria uses no alcohol in their products like all other SV restaurant I know. But this one seemed to have quite a lot of alcohol, since it felt like my breath was burning… Well, this resulted in my second SV feedback form submission…

Next I went to Alex Schröder’s talk about multilingual websites, Oddmuse and the Emacs Wiki, although also the talk A-Life about simulating evolution sounded promising. Alex asked the listeners about their experiences with multilingual websites and showed what Oddmuse offers as partial solution to the general multilingualism problems. But regarding the comments from the auditorium, there probably won’t be a perfect solution until computers can translate perfectly…

The next talk I visited was Gabor’s talk about his master thesis Organizing E-Mail which resulted in a soon to be released Mozilla Thunderbird extension called BuzzTrack. From the other concepts he showed, I found Microsoft’s SNARF (Social Network and Relationship Finder) and IBM’s Thread Arcs most interesting as well as the fact that there is no e-mail client seems to have a majority at all.

Directly after Gabor I had my own talk about Understanding Shell Quoting, so I also couldn’t go to Adrian Heydecker’s talk about Learning with Hypertext and Search Engines. I had only about three and a half listeners of whom several to my surprise where here because they didn’t know what “shell quoting” is.

I really didn’t expect that.

But that seems to be one of the differences between a BarCamp and a Linux Conferences: People come here to see something new, something they haven’t heard about before. On Linux events most people come, because they already heard about some special topic and want to know more or learn something about it. On Linux event my shell talks usually were attracting many visitors while at a BarCamp, talks presenting an idea, a concept or a tool seem to much more interesting for the attendees. So for the next BarCamp I perhaps exhume my Website Meta Language talk which never seemed to hit the nerve of Linux event attendees, since it tried to “sell” a different concept of generating website than most were used to.

At least one listener excepted the talk to be named “shell escaping”, but IMHO escaping is only one quoting technic and it’s not only used for quoting. But perhaps I should take the word “escaping” in the title though for the next time.

Happily most of the listeners seem to have learned something new from the talk and Silvan Gebhardt was really happy about his new knowledge about ssh ~ escapes, although I mainly talked about how to quote them than how to use them. :-)

During the last slot I visited the session about the upcoming BarCamp Alsace 2 and the yet to be planned BarCamp Rhine, a BarCamp to be held on a ship traveling from Basel in Switzerland down the Rhine, stopping in Strasbourg, Karlsruhe, Rhein-Main-Area and perhaps even Cologne and Amsterdam.

Contrary to my initial thoughts, the day was over very fast and I had no single boring minute during the BarCamp. Wow!

After we’ve been kicked out of the building by ETH janitors, we joined again at the Bar N-68. On the way there I met Urban Müller who attended BarCamp Zurich, too. We talked quite a lot and it was very interesting to see behind the scenes of e.g. map.search.ch. Later I joined the French speaking table, talking with Gregoire Japiot from WineCamp France and Alex Schröder.

Around 9pm I left the N-68 as one of the last BarCampers, tired but with new knowledge, new ideas, new acquaintances and a new hobby: BarCamping. What a luck that BarCamps aren’t that often, otherwise I couldn’t afford this new hobby. ;-)

As a relaxing end I met with Alex Schröder and Christophe Ducamp on Sunday morning for brunch in the restaurant Gloria in the Industriequartier. When we were leaving the Gloria I noticed their book board with a lots of BookCrossing books and I took “The Da Vinci Code” with me, since I saw the movie and people were telling me that the book is much better. I’ll see…

Bei manchem Spam ist man froh, daß SpamAssassin ihn nicht erwischt… //at 01:38 //by abe

Mir ist zwar noch nicht klar, warum der SpamAssassin diesen Spam nicht erwischt hat, aber ich bin froh drüber. Denn diese automatisierte Übersetzung ist zum totlachen:

Date: Wed, 21 Sep 2005 13:17:30 +0000
Subject: LifeProfit Inc.
X-Spam-Status: No, hits=1.6 required=5.0 tests=BAYES_44,DNS_FROM_RFCI_DSN,
        HTML_MESSAGE,MIME_BOUND_NEXTPART,MIME_HTML_NO_CHARSET,NO_REAL_NAME
        autolearn=no version=2.64

LifeProfit Inc. kann Ihnen helfen, Ihre Traume zu zwingen, sich, bekommend
das monatliche Gehalt, das das Internet einwirkt, zu verwirklichen.

Um mit LifeProfit Inc. zu arbeiten, brauchen:
? Ein B?rger Deutschlands zu sein
? das Vorhandensein der Rechnung in der Bank
? die Vereinigung des Computers zum Internet
? ist ein wenig es Ihre freie Zeit

Wie es arbeitet:

Arbeitend f?r uns wie die Bezahlung, Die den Manager bearbeitet, werden Sie
Teil Aus der neuen Tendenz im weltumfassenden Banksystem. Die Pflichten Die
Lage ist einfach, und das Einkommen hangt nur von Ihren Bem?hungen ab.

Sie Sollen - die Zahlungen der Sendung zwischen unseren Kunden, die Ihr
Interesse von jeder Operation bekommen, machen.
Hauptsachlich werden es 5 %, mit jeder Sendung, die Ihr Interesse z?chten
wird. Unsere Kunden bekommen 1000 EURO zu 20000.

www.lifeprofit.com

LifeProfit Inc. sorgt sich um Ihr Bl?hen. Machen Sie Ihr Leben besser
zusammen mit der LifeProfit Inc.!

Und daß Postbank und Deutsche Bank jetzt zur Deutschen Postbank fusioniert haben, weiß ich auch erst seit dieser netten Mail:

From support@deutsche-bank.de Tue Sep 20 21:53:14 2005
From: PostBank <support@deutsche-bank.de>
Date: Tue, 20 Sep 2005 15:45:19 -0400
Subject: PostBank online banking
X-Mailer: Microsoft Outlook, Build 10.0.2616
X-Spam-Level: ****
X-Spam-Status: No, score=4.7 required=5.0 tests=BAYES_50,FORGED_OUTLOOK_HTML,
        FORGED_OUTLOOK_TAGS,HTML_MESSAGE,MIME_HTML_ONLY,RATWARE_RCVD_PF
        autolearn=no version=3.0.2

[-- Attachment #1 --]
[-- Type: text/html, Encoding: quoted-printable, Size: 3.3K --]

[-- Autoview using /usr/bin/lynx -dump -force_html '/home/abe/Mail/tmp/muttu0uPo7' --]

   Sehr geehrte Damen und Herren,
   In Verbindung damit, daß wir in unserem Land in eine schwierige
   Situation mit online - Banking geraten sind, wurde uns empfohlen, alle
   online - Konten von unserer Bank zu kontrollieren, um die
   "Tageskonten" festzustellen, die von den Betrügern dazu benutzt
   werden, das gestohlene Geld zu waschen. Demzufolge bitten wir unsere
   Kunden inständig, das Formular einer Kontobestätigung, das sich auf
   unserer offiziellen Web-Seite befindet, auszufüllen.
   Jene Konten, die bis zum 27.08.05 auf diesem Formular nicht angegeben
   werden, werden bis zur Feststellung der Bedingungen ihrer Eröffnung
   und Benutzung blockiert. Diese Revision betrifft sowohl die
   Privatkunden, als auch die Firmenkunden.

   Wir bitten um Verzeihung für die Unannehmlichkeiten, die wir Ihnen
   bereitet haben, wir hoffen auf Ihre Hilfe und gegenseitiges
   Verständnis.
   Mit freundlichen Grüßen,
   Sicherheitsabteilung,
   Postbank.de
      © 2005 Deutsche Postbank AG

References

   Visible links
   Hidden links:
   1. http://postbanking.net/

[-- Attachment #2: Enkidu.gif --]
[-- Type: image/gif, Encoding: base64, Size: 2.8K --]

[-- image/gif is unsupported (use 'v' to view this part) --]

[-- Attachment #3: fray.jpg --]
[-- Type: image/jpeg, Encoding: base64, Size: 37K --]

[-- image/jpeg is unsupported (use 'v' to view this part) --]

Klasse, nicht?