Stoppt die Vorratsdatenspeicherung! Jetzt klicken &handeln! Willst du auch an der Aktion teilnehmen? Hier findest du alle relevanten Infos und Materialien:
Jump to menu and information about this site.

Wednesday·14·March·2012

SSH Multiplexer: parallel-ssh //at 03:10 //by abe

from the one-long-line-but-one-line dept.

There are many SSH multiplexers in Debian and most of them have one or two features which make them unique and especially useful for that one use case. I use some of them regularily (I even maintain the Debian package of one of them, namely pconsole :-) and I’ll present then and when one of them here.

For non-interactive purposes I really like parallel-ssh aka pssh. It takes a file of hostnames and a bunch of common ssh parameters as parameters, executes the given command in parallel in up to 32 threads (by default, adjustable with -p) and waits by default for 60 seconds (adjustable with -t). For example to restart hobbit-client on all hosts in kiva.txt, the following command is suitable:

$ parallel-ssh -h kiva.txt -l root /etc/init.d/hobbit-client restart
[1] 19:56:03 [FAILURE] kiva6 Exited with error code 127
[2] 19:56:04 [SUCCESS] kiva
[3] 19:56:04 [SUCCESS] kiva4
[4] 19:56:04 [SUCCESS] kiva2
[5] 19:56:04 [SUCCESS] kiva5
[6] 19:56:04 [SUCCESS] kiva3
[7] 19:57:03 [FAILURE] kiva1 Timed out, Killed by signal 9

(Coloured “Screenshots” done with ANSI HTML Adapter from the package aha.)

You easily see on which hosts the command failed and partially also why: On kiva6 hobbit-client is not installed and therefore the init.d script is not present. kiva1 is currently offline so the ssh connection timed out.

If you want to see the output of the commands, you have a two choices. Which one to choose depends on the expected amount of output:

If you don’t expect a lot of output, the -i (or --inline) option for inline aggregated output is probably the right choice:

$ parallel-ssh -h kiva.txt -l root -t 10 -i uptime
[1] 20:30:20 [SUCCESS] kiva
 20:30:20 up 7 days,  5:51,  0 users,  load average: 0.12, 0.08, 0.06
[2] 20:30:20 [SUCCESS] kiva2
 20:30:20 up 7 days,  5:50,  0 users,  load average: 0.19, 0.08, 0.02
[3] 20:30:20 [SUCCESS] kiva3
 20:30:20 up 7 days,  5:49,  0 users,  load average: 0.10, 0.06, 0.06
[4] 20:30:20 [SUCCESS] kiva4
 20:30:20 up 7 days,  5:49,  0 users,  load average: 0.25, 0.17, 0.14
[5] 20:30:20 [SUCCESS] kiva6
 20:30:20 up 7 days,  5:49, 10 users,  load average: 0.16, 0.08, 0.02
[6] 20:30:21 [SUCCESS] kiva5
 20:30:21 up 7 days,  5:49,  0 users,  load average: 3.11, 3.36, 3.06
[7] 20:30:29 [FAILURE] kiva1 Timed out, Killed by signal 9

If you expect a lot of output you can give directories with the -o (or --outdir) and -e (or --errdir) option:

$ parallel-ssh -h kiva.txt -l root -t 20 -o kiva-output lsb_release -a
[1] 20:36:51 [SUCCESS] kiva
[2] 20:36:51 [SUCCESS] kiva2
[3] 20:36:51 [SUCCESS] kiva3
[4] 20:36:51 [SUCCESS] kiva4
[5] 20:36:53 [SUCCESS] kiva6
[6] 20:36:54 [SUCCESS] kiva5
[7] 20:37:10 [FAILURE] kiva1 Timed out, Killed by signal 9
$ ls -l kiva-output
total 24
-rw-r--r-- 1 abe abe  98 Aug 28 20:36 kiva
-rw-r--r-- 1 abe abe   0 Aug 28 20:36 kiva1
-rw-r--r-- 1 abe abe  98 Aug 28 20:36 kiva2
-rw-r--r-- 1 abe abe  98 Aug 28 20:36 kiva3
-rw-r--r-- 1 abe abe  98 Aug 28 20:36 kiva4
-rw-r--r-- 1 abe abe 102 Aug 28 20:36 kiva5
-rw-r--r-- 1 abe abe 100 Aug 28 20:36 kiva6
$ cat kiva-output/kiva5
Distributor ID:	Debian
Description:	Debian GNU/Linux 6.0.2 (squeeze)
Release:	6.0.2
Codename:	squeeze

The only annoying thing IMHO is that the host list needs to be in a file. With zsh, bash and the original ksh (but neither tcsh, pdksh nor mksh), you can circumvent this restriction with one of the following command lines:

$ parallel-ssh -h <(printf "host1\nhost2\nhost3\n…") -l root uptime
[…]
$ parallel-ssh -h <(echo host1 host2 host3 … | xargs -n1) -l root uptime
[…]

And in zsh there’s an even easier way to type this:

$ parallel-ssh -h <(print -l host1 host2 host3 …) -l root uptime
[…]

In addition to parallel-ssh the pssh package also contains some more ssh based tools:

  • parallel-scp and parallel-rsync for parallel copying files onto a set of hosts.
  • parallel-slurp for fetching files in parallel from a list of hosts.
  • parallel-nuke to kill a bunch of processes in parallel on a set of machines.

I though think that parallel-ssh is by far the most useful tool from the pssh package. (Probably no wonder as it’s the most generic one. :-)

Friday·28·January·2011

Cool new feature in OpenSSH 5.7: scp between two remote hosts //at 02:55 //by abe

from the always-wanted dept.

Just a few days after OpenSSH 5.7 was released upstream, our (Debian’s as well as Ubuntu’s) tireless OpenSSH and GRUB maintainer Colin Watson uploaded a first package of OpenSSH 5.7 to Ubuntu Natty and to Debian Experimental.

Besides the obvious new thing, the implementation of Elliptic Curve Cryptography which promises better speed and shorter keys while staying at the same level of security, one other item of his changelog entry stuck out and caught my attention:

  • scp(1): Add a new -3 option to scp: Copies between two remote hosts are transferred through the local host.

That’s something I always wondered why it didn’t “just work”. While it still doesn’t seem to detect such a situation by default, it’s now at least possible to copy stuff from on remote box to another without ugly port forwarding and tunneling hacks.

Further cool stuff in the changelog:

  • sftp(1)/sftp-server(8): add a protocol extension to support a hard link operation. It is available through the “ln” command in the client. The old “ln” behaviour of creating a symlink is available using its “-s” option or through the preexisting “symlink” command.

Colin++

Tag Cloud

2CV, aha, Apache, APT, aptitude, ASUS, Automobiles, autossh, Berlin, bijou, Blogging, Blosxom, Blosxom Plugin, Browser, BSD, CDU, Chemnitz, Citroën, CLI, CLT, Conkeror, CX, deb, Debian, Doofe Parteien, E-Mail, eBay, EeePC, Emacs, Epiphany, Etch, ETH Zürich, Events, Experimental, Firefox, Fläsch, FreeBSD, FVWM, Galeon, Gecko, git, GitHub, GNOME, GNU, GNU Coreutils, GNU Screen, Google, GPL, grep, grml, gzip, Hackerfunk, Hacks, Hardware, Heise, HTML, identi.ca, IRC, irssi, Jabber, JavaShit, Kazehakase, Lenny, Liferea, Linux, LinuxTag, LUGS, Lynx, maol, Meme, Microsoft, Mozilla, Music, mutt, Myon, München, nemo, Nokia, nuggets, Open Source, Opera, packaging, Pentium I, Perl, Planet Debian, Planet Symlink, Quiz, Rant, ratpoison, Religion, RIP, Sarcasm, Sarge, Schweiz, screen, Shell, Sid, Spam, Squeeze, SSH, Stöckchen, SuSE, Symlink, Symlink-Artikel, Tagging, Talk, taz, Text Mode, ThinkPad, Ubuntu, USA, USB, UUUCO, UUUT, VCFe, Ventilator, Vintage, Wahlen, Wheezy, Wikipedia, Windows, WML, Woody, WTF, X, Xen, zsh, Zürich, ÖPNV

Calendar

← 2017 →
Months
AugSep Oct Nov Dec
 August →
Mo Tu We Th Fr Sa Su
 
18 19 20
21 22 23 24 25 26 27
28 29 30 31      

Tattletale Statistics

Blog postings by posting time
Blog posting times this month



Search


Advanced Search


Categories


Recent Postings

2 most recent of 2 postings total shown.


Recent Comments

Hackergotchi of Axel Beckert

About...

Debian GNU/Linux is my favourite Linux distribution, being stable, flexible, consistent and having a great community. Although I'm not the biggest bug report writer, I try to contribute by staffing the Debian booth at events, carrying the necessary hardware there or even organising the whole booth.


RSS Feeds


Identity Archipelago


Picture Gallery


Button Futility

Valid XHTML Valid CSS
Valid RSS Any Browser
GeoURL
This content is licensed under a Creative Commons License (SA 3.0 DE). Some rights reserved. Hacker Emblem
Get Mozilla Firefox! Powered by Linux!
Typed with GNU Emacs Listed at Tux Mobil
XFN Friendly Button Maker

Blogroll

Blog or not?


People I know personally


Other blogs I like or read


Independent News


Interesting Planets


Web comics I like and read

Stalled Web comics I liked


Blogging Software

Blosxom Plugins I use

Bedside Reading

Just read

  • Bastian Sick: Der Dativ ist dem Genitiv sein Tod (Teile 1-3)
  • Neil Gaiman and Terry Pratchett: Good Omens (borrowed from Ermel)

Currently Reading

  • Douglas R. Hofstadter: Gödel, Escher, Bach
  • Neil Gaiman: Keine Panik (borrowed from Ermel)

Yet to read

  • Neil Stephenson: Cryptonomicon (borrowed from Ermel)

Always a good snack

  • Wolfgang Stoffels: Lokomotivbau und Dampftechnik (borrowed from Ermel)
  • Beverly Cole: Trains — The Early Years (getty images)

Postponed