Join 3,556 readers in helping fund MetaFilter (Hide)


Telnet Times Ten Thousand
April 12, 2012 8:16 AM   Subscribe

If you've ever worked with the command prompt on a Unix-based computer, you're likely familiar with SSH (Secure SHell), which is a program and a protocol that allows you (yes, you!) to securely access a remote system. While SSH has certainly earned the "Secure" portion of its namesake over the years, it's functionality as a shell has ironically received very little attention, and has begun to show signs of age and obsolescence: SSH doesn't work very well on mobile connections, and its support for Unicode is buggy and incomplete. A group of MIT researchers think they've found solutions to these problems, and have created Mosh as a potential successor to SSH, which fixes many of the old protocol's annoyances and shortcomings, while retaining all of SSH's security features.
posted by schmod (77 comments total) 52 users marked this as a favorite

 
The roaming problem is long solved. See GNU screen for more details.
posted by Blazecock Pileon at 8:20 AM on April 12, 2012 [7 favorites]


Are we sure that SSH is obsolete?

This is quite honestly the first hint of that I've heard. It's always seemed like one of the least controversial corners of the command line world, I mean this isn't vi versus emacs or something!

Can anyone elaborate on that? Have I been missing out on drama?
posted by Stagger Lee at 8:22 AM on April 12, 2012 [3 favorites]


The GNU screen problem is also long solved. See tmux for more details.
posted by felix at 8:23 AM on April 12, 2012 [23 favorites]


Yeah, learning screen pretty much changed my command-line life. Possibly second only to VIM in terms of being indispensable to my workflow.
posted by kmz at 8:23 AM on April 12, 2012 [5 favorites]


Yeah, I've been doing screen + ssh for a while now and it completely rules for mobile email. Better than "true" mobile email, IMO, since I don't have to have my email split or synced or anything.

But what's this about ssh being a terrible shell? Doesn't it just dump you into your normal shell? I'm not having any trouble typing unicode into my ssh connection here.
posted by DU at 8:25 AM on April 12, 2012 [1 favorite]


The GNU screen problem is also long solved. See tmux for more details.

Screen does everything I need, but that's interesting and I'll check it out. Thanks!
posted by Blazecock Pileon at 8:26 AM on April 12, 2012


Unlike SSH, mosh's UDP-based protocol handles packet loss gracefully, and sets the frame rate based on network conditions. Mosh doesn't fill up network buffers, so Control-C always works to halt a runaway process.
OK, this one is actually pretty useful. So is the "server runs as user" thing.
posted by DU at 8:27 AM on April 12, 2012 [4 favorites]


What about ssh Unicode support is buggy and incomplete?

Mosh uses ssh, for authentication and setting up the UDP server. It's an interesting replacement for parts of ssh but not for the public key part. If you care about the difference between screen and tmux, then Mosh may be an interesting third way.
posted by Nelson at 8:27 AM on April 12, 2012 [1 favorite]


It's not that SSH is obsolete, it's that wireless sucks. And SSH over terrible wireless sucks quite a bit. High-latency connections also suck. (Slow connections suck too, but that goes without saying.) This is a fix for parts of that.
posted by fragmede at 8:28 AM on April 12, 2012 [3 favorites]


Yeah, I heard about this one and it's a little bit of a head-scratcher for me. SSH is useful in part because so many things support it.
posted by verb at 8:28 AM on April 12, 2012


This post bills mosh as a successor to SSH, but yet it uses SSH to handle authentication and login? I don't understand.
posted by NathanBoy at 8:29 AM on April 12, 2012 [1 favorite]


The upside to mosh (and, full disclosure, I know a bunch of the contributors) is that you don't have to log back in like with screen. So, I open a remote shell at home, take my laptop to work and unsuspend, and it's already talking to my server like nothing happened. Head to the airport, go cross country, same deal. It's pretty shiny, and surprisingly nice once you try it. Doesn't eliminate the need for screen, though, since screen (a) lets you have multiple "windows", and (b) survives even if you have to restart your local terminal. I use mosh to replace ssh, but still run screen once I've connected. (Just replace "ssh host -t screen -rd" with "mosh host -- screen -rd".)

I think the bigger ooh-shiny is that it's a lot more robust against a connection with lots of dropped packets than SSH - there's a lot less perception of lag. Which makes using a terminal over a slow or laggy or drop-prone connection tolerable, as opposed to infuriating.
posted by spaceman_spiff at 8:30 AM on April 12, 2012 [7 favorites]


I'm cautiously excited about this, if only because I am the type of weirdo who ssh'es from their phone (and while screen is awesome, if the connection is dropped due to an IP change you still have to log back in, right? or is there something I'm missing?).

Also I hate that "everything is hieroglyphics" state with the white hot heat of a thousand suns.
posted by en forme de poire at 8:31 AM on April 12, 2012 [1 favorite]


This post bills mosh as a successor to SSH, but yet it uses SSH to handle authentication and login? I don't understand.

From what I can tell, it's not really a replacement for SSH, more a wrapper/enhancer.
posted by kmz at 8:31 AM on April 12, 2012


Or on preview, what spaceman_spiff said.
posted by en forme de poire at 8:32 AM on April 12, 2012


The claim is that it sucks to use SSH interactively over a high latency connection. You don't see what you type immediately, for instance, which isn't catastrophically bad but really fucks with your typing speed and accuracy, especially if you're used to touch-typing looking at the screen.

It looks like they've done some interesting stuff to deal with that. On the one hand, you see the characters you type right away (like with local echo mode in telnet). But on the other hand, there's a visual difference between characters that are just being locally echoed and those that have made the "round trip" to the server and back (which IIRC local echo on telnet didn't do), so you can keep track of how badly "behind" the server is while still getting the visual feedback that tells you you're typing things correctly in real time.

I would have killed for that feature back in 1994. I honestly just don't spend enough time working with a shitty connection these days, but for those who do I suspect this will be pretty awesome.
posted by nebulawindphone at 8:35 AM on April 12, 2012 [5 favorites]


nebulawindphone has it. There are many ways to optimize your experience over the internet. For instance, modern browsers download more than two files at a time, because the internet these days is fast as hell. Google is optimizing TCP all over their clusters because they know exactly how their data centers are laid out.
This is a good example of "wait, I know what the user wants from this particular connection, let's just make it happen!" (See also putty's "disable nagle algorithm")
posted by Phredward at 8:44 AM on April 12, 2012


I use SSH and screen (and screen -x!) whenever I'm in front of a computer, and have always idly wished for stateless SSH to support roaming across IPs. I freaking love this idea.
posted by These Premises Are Alarmed at 9:02 AM on April 12, 2012 [1 favorite]


It's a remote object synchronization protocol over an encrypted UDP 'session'. You SSH to your host and run a 'mosh' server with an encryption key then the ssh session exits and the mosh server keeps listening on a UDP port. Then on your machine the 'mosh' client is started with the same encryption key. Now the client and server can communicate with encrypted UDP packets. That's not terribly magic, think IPSEC. The tricks are: a) the server doesn't care what IP talks to it, an incoming packet either decrypts with the key or it's trash; b) the real magic is the object synchronization protocol. Think like a Terminal-Emulator-Object each on the client and server that are kept in sync. Like an online FPS shooter game that's keeping the world in a consistent state but much better. mosh checks/sends updates as needed back and forth at 1/50th second intervals making sure the objects on the client/server are in the same state. That's the rough gist of it based on what I've read so far.

The difference between mosh and a VPN being that a session isn't tied to a client IP, the client IP can change since they aren't checking and it's just encrypted UDP packets flying back and forth. And the session key is setup during the initial mosh server start (protected by a short SSH session).

Nothing at all to do with screen/tmux and I have no clue what they're talking about re: Unicode problems with SSH.

Latency and bad network is just taken care of by not using guaranteed TCP with retransmits and backoff and such, instead just UDP back and forth until the objects on both sides are consistent (at 1/50s intervals).
posted by zengargoyle at 9:06 AM on April 12, 2012 [11 favorites]


To be honest I like the idea that I will need to re-enter my password/pasphrase if I connect to a different network before I wake up my sleeping laptop. I also enter a local password to wake the damn thing from sleep or screen-saver. Every little bit helps, especially when I am logged in with `sudo` permissions higher than most other users get.
posted by wenestvedt at 9:15 AM on April 12, 2012


autossh is very useful
posted by beukeboom at 9:21 AM on April 12, 2012


wenestvedt: "To be honest I like the idea that I will need to re-enter my password/pasphrase if I connect to a different network before I wake up my sleeping laptop."

Really, in that scenario, you should be using your own private key to log in. It's a bit safer. As long as you lock your laptop when it's asleep, and encrypt its hard drive, this is way more secure than a normal password.
posted by schmod at 9:22 AM on April 12, 2012 [1 favorite]


Unicode problems this solves are illustrated here: under "tricky Unicode".
posted by fightorflight at 9:38 AM on April 12, 2012


zengargoyle: "and I have no clue what they're talking about re: Unicode problems with SSH."

They explain that on their page in the Technical Info section.
posted by namewithoutwords at 9:38 AM on April 12, 2012


(and Screen is next to useless if you have a braindead router that drops SSH connections after 20s of idle. Mosh keeps on trucking nicely, I was delighted to discover)
posted by fightorflight at 9:39 AM on April 12, 2012


They explain that on their page in the Technical Info section.

That's interesting, and underscores how frustrating character set issues can be to troubleshoot: Three different behaviors on three different terminals.

One of my external developers had some nutty problems running svn updates over ssh when files in the repo had certain characters in their names. We ended up solving it by explicitly setting LANG in his bashrc, but he was the only person in our group who needed that workaround. We couldn't reproduce the problem on anyone else's machine.

That's the only time I've had a problem like that where ssh was even in the neighborhood when the problems occurred. Ruby 1.8 -> 1.9 has caused me many more problems with Unicode.
posted by mph at 9:51 AM on April 12, 2012


It appears that mosh's udp tricks should simply be folded into ssh itself, maybe via a command line option, certainly "mosh does not support X forwarding or the non-interactive uses of SSH, including port forwarding or sshfs" does not sound too promising.
posted by jeffburdges at 9:52 AM on April 12, 2012


Interesting. Doesn't immediately connect to localhost. Probably still requires port-forwarding gymnastics for my most common setup. But promising.

tmux is interesting too, thanks for that derail.
posted by fantabulous timewaster at 9:52 AM on April 12, 2012


Thanks for pointing out the Unicode problem. It's not a bug in ssh, it's a bug in rendering combining characters in the xterm and Gnome terminal emulators. Mosh has its own terminal emulator that does not have that specific Unicode bug. Yay!

Totally love the idea of a remote network program that works well on flaky networks. In the long long ago I used to use a program called "uw" (stood for "Unix Windows") for my login sessions. It was a similar design; the server process kept a lot of state and the client could reconnect and you picked up your sessions where you were last logged in. But like Mosh and unlike Screen, a lot more happened on the client side. For instance, the local character echo thing. It worked pretty well for dialup, doubly so because the client worked over a straight serial connection without needing TCP/IP (this in the days before Winsock, when SLIP was difficult and PPP some exotic new thing).

And to re-connect to Unicode, the uw vt100 emulator was garbage; the amd52 emulation was much better so you used that. At some point the uw authors even invented a sort of virtual termcap system so you could create a sane, simple set of terminal codes in the uw client and then teach termcap about them in the Unix system. Good times.

I can't find uw in a quick Google; it's not an easy string to search for. It may have died out before the Web became common. Here's a bunch of Usenet posts about it.
posted by Nelson at 9:58 AM on April 12, 2012


Q: Has your secure datagram protocol been audited by experts?

No. Mosh is actively used and has been read over by security-minded crypto nerds who think its design is reasonable, but any novel datagram protocol is going to have to prove itself, and SSP is no exception. We use the reference implementations of AES-128 and OCB, and we welcome your eyes on the code. We think the radical simplicity of the design is an advantage, but of course others have thought that and have been wrong. We don't doubt it will (properly!) take time for the security community to get comfortable with mosh.


I guess I'll hold off sending the launch codes with this for a while.
posted by double block and bleed at 9:58 AM on April 12, 2012 [1 favorite]


jeffburdges: "It appears that mosh's udp tricks should simply be folded into ssh itself, maybe via a command line option"

Horses for courses. Most users of vi have ed installed. Most users of emacs have hoth vi and ed installed. ssh does much more than interactive secure remote connections, and mosh uses ssh for part of its own functionality, this is pretty much how things have been done on unix systems forever.
posted by idiopath at 10:02 AM on April 12, 2012


This post bills mosh as a successor to SSH, but yet it uses SSH to handle authentication and login? I don't understand.

Nor do I, and I don't understand why the Mosh devs bill it as 'a replacement'. It is a replacement in that you type mosh at the command line where you might have typed ssh before, but really it's more like a companion to SSH (for the time being, at least).

Whatever, it's pretty cool - it let me get stuff done on the train recently using the incredibly flakey onboard WiFi which would've been a right hassle with SSH alone.

One tip: if you need to connect to SSH via a port other than the standard 22 you have to use a Host directive in your ~/.ssh/config. Mildly annoying, but they're fixing it.

The GNU screen problem is also long solved. See tmux for more details.

For whatever reason, I always found screen incredibly confusing, but tmux 'just worked' for me when I first tried it. It's super easy to configure, too. Defo recommended if you've had trouble with screen in the past (though I'm not sure whether it's worth switching if you're happy?).
posted by jack_mo at 10:03 AM on April 12, 2012


Hey, since there seem to be a bunch of gurus around, does anyone know an easy way to download something from within an ssh shell to my local machine without doing a separate sftp login? Coz that would be swell and make my life better.
posted by kaibutsu at 10:09 AM on April 12, 2012 [1 favorite]


Yeah. It looks like they're using SSH to do some of the heavy lifting, since SSH won't be going anywhere anytime soon, and there's no need to reinvent the wheel. This approach also has a few other perks such as the ability to run a Mosh server without root access.

Little features like these will undoubtedly speed up its adoption. It's slightly better than SSH, easy to install, and doesn't break compatibility with any existing services or applications. In the free software world, that's pretty much a slam-dunk for guaranteed eventual widespread adoption, as long as they don't fork or botch any future updates.
posted by schmod at 10:10 AM on April 12, 2012


kaibutsu: "Hey, since there seem to be a bunch of gurus around, does anyone know an easy way to download something from within an ssh shell to my local machine without doing a separate sftp login? Coz that would be swell and make my life better."

SCP?

As I alluded to above, it's a really good idea to set up private keys and an ssh agent if you're frequently SSHing between two machines. As an added bonus, you get to log in without a password.
posted by schmod at 10:13 AM on April 12, 2012 [1 favorite]


If you have rock-solid internet connectivity with low latency to the host you're connecting to (and you can connect to it directly), there's absolutely nothing wrong with SSH.

If you have to manage, say, a firewall that is in Zimbabwe, which you can only connect to via a gateway server in Germany, then SSH sessions are mouse-throwingly annoying.

We can't run Mosh though, so we have to jump through hoops of running everything screened using gateways at each location so that our RW sessions don't get locked when one disconnects. For even more of a nightmare scenario, the stuff that requires "fat clients" running on Windows is enough to make me constantly wish that I was in another line of work.

does anyone know an easy way to download something from within an ssh shell to my local machine without doing a separate sftp login? Coz that would be swell and make my life better.

There is FXP, which should theoretically work over a secure session, but there are so many bad things about this that you will probably never be able to use it due to it always being disabled. What you are asking for is essentially a MITM, with is highly frowned upon.
posted by Threeway Handshake at 10:15 AM on April 12, 2012 [1 favorite]


(and Screen is next to useless if you have a braindead router that drops SSH connections after 20s of idle.

For machines where you don't have access to the ssh config, you want to look into ServerAliveInterval otherwise you're looking for ClientAliveInterval.

With screen, you want something like this in your .screenrc, so it will always be updating:
caption always '%0c:%s %d/%m'
posted by madajb at 10:17 AM on April 12, 2012 [2 favorites]


SCP?

I think what he's asking for is something equivalent to the old "sz" and "rz" commands from when modems existed and dinosaurs roamed the earth.

I, too, would like to be able to just say, "retrieve X" in an SSH session, and have that file show up locally.
posted by mhoye at 10:18 AM on April 12, 2012


does anyone know an easy way to download something from within an ssh shell to my local machine without doing a separate sftp login? Coz that would be swell and make my life better.

OH, I totally misread that, so ignore my FXP comment.
You can't really do that. Any command you run to copy it will start a new session of SCP/SFTP and will be subject to authentication.
posted by Threeway Handshake at 10:20 AM on April 12, 2012


It's not that SSH is obsolete, it's that wireless sucks. And SSH over terrible wireless sucks quite a bit. High-latency connections also suck. (Slow connections suck too, but that goes without saying.) This is a fix for parts of that.

This is honestly news to me, and I use SSH a lot. I've actually been fairly surprised by the resilience, and had been meaning to ask a question about it.

There are many times when I connect to a server, put my laptop to sleep without disconnecting, and wake it up hours later and I can still type into the SSH window. It's actually a puzzle to me.
posted by odinsdream at 10:21 AM on April 12, 2012


I, too, would like to be able to just say, "retrieve X" in an SSH session, and have that file show up locally.

I'll need to dig out my SSH book at home, but I'm fairly sure there's a way to do this.
posted by odinsdream at 10:24 AM on April 12, 2012


I think what he's asking for is something equivalent to the old "sz" and "rz" commands from when modems existed and dinosaurs roamed the earth.

Ah, that's more a function of the terminal program than ssh. Windows ssh client and terminal emulator SecureCRT still supports ZModem. PuTTY might support it as well, or maybe Kermit. I don't think XTerm or the MacOS terminal do.
posted by Nelson at 10:25 AM on April 12, 2012


i want zmodem plus, so I can play tetris against the server while it downloads

(did i make that up? i can't find a reference to that on the googles, but I swear there was some _modem implementation that allowed for this)
posted by MysticMCJ at 10:30 AM on April 12, 2012 [1 favorite]


Downloading via existing SSH connection... it's easier if you remember to do a remote port forward when you start your session, it's a damned good idea to do both a -L and -R forwarding so you have pipelines sharing your connection. But you can use the ~ escapes to start one up on the fly.
[remote ~]$ ~C [ssh command line]                                               
ssh> -R8081:localhost:8082
Forwarding port.

[remote ~]$ 
[remote ~]$ ~^Z [suspend ssh]

[5]+  Stopped                 ssh remote
local:~/Downloads$ nc -l 8082 > foobar &
[6] 32071
local:~/Downloads$ fg %5
ssh remote

[remote ~]$ ls | nc localhost 8081
[remote ~]$ ~^Z [suspend ssh]

[5]+  Stopped                 ssh remote
[6]   Done                    nc -l 8082 > foobar
local:~/Downloads$ cat foobar | head
API.pm
check.sh
...

posted by zengargoyle at 10:44 AM on April 12, 2012 [12 favorites]


I have been using screen+ssh for years. The limitations of this aren't too severe, even when the ssh client is on my phone. However, mosh+screen is superior on the desktop, and I'm particularly excited about it on the phone (recently there's been activity on the 'mosh should be ported to android' bug, and it looks like it's this close to usability for someone who doesn't mind understanding what an "Android NDK" is)

There is at least one actual ssh limitation relating to utf-8 that I saw in the tech notes: the IUTF8 stty setting. I was not aware of IUTF8 before, so I had to take his word for it.

I'm curious how mosh works around these terminal limitations. Mosh is not itself a terminal program; it still relies on your local terminal emulator for the ultimate rendering. Are there workarounds for each of these identified problems hiding in there somewhere? If so...yuck!
posted by jepler at 10:51 AM on April 12, 2012


download something from within an ssh shell to my local machine without doing a separate sftp login

Opportunistic ControlMaster is your friend. SSH will use the same machinery that multiplexes forwarded ports onto your ssh connection to multiplex additional logins or sftp connections or whatever you like. You still have to invoke sftp, but there's no authentication delay or prompt since it uses the existing channel.
posted by hattifattener at 11:09 AM on April 12, 2012 [1 favorite]


As for Unicode, it's hairy and I'm not convinced mosh is correct, but would have to seek authority.

uniprops 'COMBINING CIRCUMFLEX ACCENT'
U+0302 ‹◌̂› \N{COMBINING CIRCUMFLEX ACCENT}
    \w \pM \p{Mn}
    All Any Assigned InCombiningDiacriticalMarks Case_Ignorable CI
       Combining_Diacritical_Marks Dia Diacritic M Mn Gr_Ext Grapheme_Extend
       Graph GrExt ID_Continue IDC Inherited Zinh Mark Nonspacing_Mark Print
       Qaai Word XID_Continue XIDC X_POSIX_Graph X_POSIX_Print X_POSIX_Word
uniprops 'LINE FEED'
U+000A ‹U+000A› \N{LINE FEED (LF)}
    \s \v \R \pC \p{Cc}
    All Any ASCII Assigned Basic_Latin C Other Cc Cntrl Common Zyyy Control
       Pat_WS Pattern_White_Space PatWS PerlSpace POSIX_Cntrl POSIX_Space Space
       SpacePerl XPerlSpace VertSpace White_Space WSpace X_POSIX_Cntrl
       X_POSIX_Space
uniprops a
U+0061 ‹a› \N{LATIN SMALL LETTER A}
    \w \pL \p{LC} \p{L_} \p{L&} \p{Ll}
    AHex ASCII_Hex_Digit All Any Alnum Alpha Alphabetic ASCII Assigned
       Basic_Latin Cased Cased_Letter LC Changes_When_Casemapped CWCM
       Changes_When_Titlecased CWT Changes_When_Uppercased CWU Ll L Gr_Base
       Grapheme_Base Graph GrBase Hex XDigit Hex_Digit ID_Continue IDC ID_Start
       IDS Letter L_ Latin Latn Lowercase_Letter Lower Lowercase PerlWord
       POSIX_Alnum POSIX_Alpha POSIX_Graph POSIX_Lower POSIX_Print POSIX_Word
       POSIX_XDigit Print Word XID_Continue XIDC XID_Start XIDS X_POSIX_Alnum
       X_POSIX_Alpha X_POSIX_Graph X_POSIX_Lower X_POSIX_Print X_POSIX_Word
       X_POSIX_XDigit
A Grapheme_Extended should follow a Grapheme_Base not a White_Space, and it's Letter + Combining Character*. What exactly is correct in the case of a combining character following a non-printing, non-letter (in the case it's chunked "\n\N{COMBINING CIRCUMFLEX ACCENT}" or the bare "\N{COMBINING CIRCUMFLEX ACCENT}", neither makes sense.

My xterm displays it as a '^' that's *under* the 'c' which seems to be like "display the LINE FEED (not as CR or CRLF but just LF) and then combine with the previous GRAPHlike (space/whitespace) vs stripping it out since it has no printable letter to combine with vs processing the \n normally, not as a LF+combining, and then displaying a combining character that has no base. Not sure what the Unicode gods prescribe.
\
posted by zengargoyle at 11:18 AM on April 12, 2012


There are -t options to ssh-agent and ssh-add which provide the sensible version of the functionality you desire, kaibutsu, i.e. if you've typed you public key password recently, then you'll get password-less logins. You can change this setting wherever your system starts ssh-agnet, which happens to be /System/Library/LaunchAgents/org.openbsd.ssh-agent.plist under Mac OS X.
posted by jeffburdges at 11:34 AM on April 12, 2012


Yeah, Unicode 6.0 Chapter 3 (pdf).
D50 Graphic character: A character with the General Category of Letter (L), Combining
Mark (M), Number (N), Punctuation (P), Symbol (S), or Space Separator (Zs).

Graphic characters specifically exclude the line and paragraph separators (Zl, Zp), as well as the characters with the General Category of Other (Cn, Cs, Cc, Cf).

D51 Base character: Any graphic character except for those with the General Category of Combining Mark (M).

• Most Unicode characters are base characters. In terms of General Category val-
ues, a base character is any code point that has one of the following categories:
Letter (L), Number (N), Punctuation (P), Symbol (S), or Space Separator (Zs).

• Base characters do not include control characters or format controls.

D52 Combining character: A character with the General Category of Combining Mark
(M).

D56 Combining character sequence: A maximal character sequence consisting of either a
base character followed by a sequence of one or more characters where each is a combining character, zero width joiner, or zero width non-joiner; or a sequence of one or more characters where each is a combining character, zero width joiner, or zero width non-joiner.

D57 Defective combining character sequence: A combining character sequence that does
not start with a base character.

• Defective combining character sequences occur when a sequence of combining characters appears at the start of a string or follows a control or format charac-
ter. Such sequences are defective from the point of view of handling of combin-
ing marks, but are not ill-formed. (See D84.)

So the mosh example `echo -e "abc\n\0314\0202\ndef"` is not ill-formed but is defective. I'm not sure there is a right way to display defective.
posted by zengargoyle at 12:30 PM on April 12, 2012


zengargoyle's downloading-from-inside-ssh magic just blew my mind
posted by rossmeissl at 12:49 PM on April 12, 2012


Hm. That may be...I'm honestly not all that familiar with the nitty gritty of Unicode.

However, if Mosh can prevent a misbehaving application that outputs a bit of bad Unicode from borking an entire SSH/terminal session, I'd honestly be happy if that was the only improvement they made.

Things should "just work." Unix applications and tools should be able to handle many kinds of inputs, and also be able to detect and compensate for obviously bad inputs. Sadly, Unix rarely does this.

I was kind of hoping that the open-source community was starting to drift away from the mentality of claiming that its solutions were 'good enough,' provided that some arbitrary workaround was applied, or offering suggestions like 'use screen' or 'the terminal's fine; patch your app so it puts out proper Unicode.'

There are times when I wonder if the GNU folks are actually worse at admitting flaws in their products than Microsoft or Apple are. Yeah, I get the hacker mentality behind the fact that screen is used (and works well) for 1,001 things that it was never intended to do, but for chrissakes they also need to fix the underlying problems that cause people to resort to those hacks in the first place. If the most common use of a terminal multiplexer is to maintain a stateless terminal, maybe we should just implement an actual stateless terminal, instead using the mathematicians' cop-out by proclaiming "Yes! The problem can be solved!" without actually devising an elegant way to solve that problem.

Kudos to the Mosh team for actually solving the problem, rather than bolting on another hack.

Unix needs to adapt to reflect the way that people do computing in 2012, especially so that new users do not need to figure out the Unix terminal's quirks by first assuming that their laptop is actually a teletype machine (TTY; in case you ever wondered what that stood for) and chronologically working their way up from there.
posted by schmod at 12:53 PM on April 12, 2012 [5 favorites]


>I, too, would like to be able to just say, "retrieve X" in an SSH session, and have that file show up locally.

This. The netcat trick looks cute, but its a hassle. I'd love to just type "localget $filename" and "localput $filename" be done with it. Now I go through ftp which is a protocol from 1971.

I imagine this can be solved on the client side without doing anything server side. The client can open an sftp connection at the same time and have some proprietary junk in there to accept the custom transfer command. RDP works a bit like this now.
posted by damn dirty ape at 12:56 PM on April 12, 2012


does anyone know an easy way to download something from within an ssh shell to my local machine without doing a separate sftp login? Coz that would be swell and make my life better.
The port forwarding magic by zengargoyle is pretty awesome. If you're going to drop back to a local terminal, though, it's easier just to use multiplexed sessions:
fu $ ssh bar -MfN
fantabulous@bar's password:
fu $ ssh bar
bar $ touch /tmp/whatever # n.b. no re-authentication was necessary

In another terminal somewhere:

fu $ scp bar:/tmp/whatever . # re-use the multiplexed session again
whatever 100% 0 0.0KB/s 00:00
On the other hand, if you're willing to set up .ssh/config files everywhere, you can do set up an alias for the machine you prefer to work from. I have
### in fu:~/.ssh/config
# Let myself connect back in:
RemoteForward 42747 localhost:22

### in ~/.ssh/config on many other machines:
Host fu
HostKeyAlias fu
HostName localhost
Port 42747
User fantabulous
which allows you to simply say
fu $ ssh bar
bar $ scp /tmp/whatever fu:
even when fu is hidden behind some outward-only firewall or coffeeshop NAT. (For a while I had tunnels like this enabled from a half-dozen different machines, so coming up with distinct random port numbers and distributing them to .ssh/configs was kind of annoying. But lately I'm doing almost everything from my one laptop, so it's not too bad.)
posted by fantabulous timewaster at 1:32 PM on April 12, 2012 [1 favorite]


I'd love to just type "localget $filename" and "localput $filename" be done with it.

Transmit plugs into the Finder to accomplish the same thing with SSH servers, as sshfs/FUSE used to.
posted by Blazecock Pileon at 1:34 PM on April 12, 2012


There is an underlying authentication issue that a Win SCP clone like Transmit cannot solve, Blazecock Pileon, the correct solution is either ssh-agent -t or restricted access ssh keys.

There are ways to implement these hypothetical localget and localput commands using agent forwarding, damn dirty ape.   Add your own public key to your local .ssh/authorized_keys file. Create a shell function that runs the desired ssh command using the SSH_CLIENT environment variable.

export L=${SSH_CLIENT%% *}
function localget { scp $L:$1 . ;}
function localput { scp $1 $L:. ;}

If you use bash-completion, then typing ssh $L:[tab][tab] would list files inside your home directory on your local machine because bash-completeion explores ssh connections.

There is an underlying security risk here that anyone who can access your authentication agent can access your machine, but perhaps you could employ a restricted key that only ran specific applications, like perhaps pastelocal or copyremote commands that only access your local cut & paste buffer.

You might also try using sshfs more because that exposes no security risks, but frequently does everything you need.
posted by jeffburdges at 2:33 PM on April 12, 2012


Create a shell function that runs the desired ssh command using the SSH_CLIENT environment variable.
Doesn't this require you to be ssh-ing from a publicly-visible machine?
posted by fantabulous timewaster at 3:13 PM on April 12, 2012


OH MY GOD HOW HAVE I LIVED WITHOUT SCREEN ALL THIS TIME
posted by koeselitz at 6:26 PM on April 12, 2012 [4 favorites]


Just use port forwarding if your machine isn't usually visible, changing the ssh commands to exploit it naturally.
posted by jeffburdges at 7:02 PM on April 12, 2012


OH MY GOD HOW HAVE I LIVED WITHOUT SCREEN ALL THIS TIME

I know, right? A simple .screenrc file changed my life, four years ago.
posted by Blazecock Pileon at 7:59 PM on April 12, 2012 [1 favorite]


I've just switched over to mosh as an experiment for my IRC use (I use a ssh + screen + irssi setup). Initial impressions are pretty positive. The local echo features are really really nice for this use case - lag when typing on IRC was a pain in the ass.
posted by DRMacIver at 1:25 AM on April 13, 2012


zmodem + ssh = zssh
posted by Rhomboid at 2:24 AM on April 13, 2012 [1 favorite]


I thought that ssh -MfN required setting the ControlMaster and ControlPath options, yes?

There are a bunch more amusing ssh tricks here listed in blog entries on urfix.com and itworld.
posted by jeffburdges at 4:03 AM on April 13, 2012


Another file-copying trick using ssh, which I like to call the "throwing and catching" method:

# ssh user@host 'cd /path/to/directory; tar cf - .' | tar xf -

copies a whole directory tree from remote with one command. (Doesn't work from your logged-in session, of course, but the concept is similar and while we're talking about these things I thought I'd share.)

Also works the other way around:

# tar cf - . | ssh user@host 'cd /path/to/directory; tar xf -'
posted by illongruci at 5:28 AM on April 13, 2012


koeselitz: "OH MY GOD HOW HAVE I LIVED WITHOUT SCREEN ALL THIS TIME"

for people just discovering terminal multiplexing for the first time, you really, really want tmux instead of screen.

For those interested, in the hopes they will prove useful - my rather featureful .screenrc (html | plain) and .tmux.conf (html | plain)
posted by namewithoutwords at 6:41 AM on April 13, 2012 [12 favorites]


namewithoutwords: " (html | plain)"

Derail: What'd you use to generate the pretty syntax formatting? It's very nice.

posted by schmod at 7:02 AM on April 13, 2012


OK, tmux converts:

In screen, I will often split vertically (I've had the patch forever) and display a couple screens side-by-side.

In tmux, however, a split creates multiple panes within the window. I don't like this. I want different top-level windows within the split. Is this possible? Am I missing something?
posted by These Premises Are Alarmed at 7:34 AM on April 13, 2012


There are several hits when searching for vertical in tmux's man page. I'd use terminal tabs by attaching multiple times via screen -x -r .. myself, apparently this is tmux attach-session -t ..
posted by jeffburdges at 7:51 AM on April 13, 2012


schmod: "Derail: What'd you use to generate the pretty syntax formatting? It's very nice."

The pretty syntax highlighted html was generated automagically by vim (with a sidetrip through tidy to validate it). I found a nice script on the internet for this process a while ago, not sure where.
You can grab it here: (html | plain)
posted by namewithoutwords at 7:59 AM on April 13, 2012 [2 favorites]


I thought that ssh -MfN required setting the ControlMaster and ControlPath options, yes?
Whoops, I forgot about that. But only ControlPath must be set, and only on the client where ssh runs, and there's a sensible global value. A one-line ~/.ssh/config.
$ ssh -F /dev/null -o'ControlPath ~/.ssh/master-%r@%h:%p' ft@remote  -MfN
ft@remote's password: 
$ ssh -F /dev/null -o'ControlPath ~/.ssh/master-%r@%h:%p' ft@remote  -O check
Master running (pid=16174)
$ ssh -F /dev/null -o'ControlPath ~/.ssh/master-%r@%h:%p' ft@remote  hostname
remote
posted by fantabulous timewaster at 8:16 AM on April 13, 2012


namewithoutwords: “for people just discovering terminal multiplexing for the first time, you really, really want tmux instead of screen.”

Yeah, tmux looks even better actually – and hopefully the key bindings don't conflict much with emacs. I'll have to download and compile that some time later today when I get a chance, to see how it works.
posted by koeselitz at 8:25 AM on April 13, 2012


You can fix the screen / emacs clash by rebinding the escape key from Ctrl-A to something else. I use `, have had escape `` in my .screenrc for 10+ years. (Press ` twice to insert a literal `).
posted by Nelson at 9:17 AM on April 13, 2012 [1 favorite]


tmux defaults to using control-B, FWIW. (being a longtime screen user, and not an emacs person, I actually rebind control-B to control-A in tmux).
posted by namewithoutwords at 10:01 AM on April 13, 2012


In tmux, however, a split creates multiple panes within the window. I don't like this. I want different top-level windows within the split. Is this possible? Am I missing something?

This may just be a terminological/conceptual issue. I didn't use screen for long enough to really fully grok its window/pane/buffer model, but the model in tmux is different, and this is one of the differences.

As I understand it, "panes" in tmux basically are what screen would call "windows."

Is there some specific thing you want to be able to do in tmux with two side-by-side panes that you're not able to do? Or is it more just that tmux uses a conceptual model that you find foreign? (If it's the latter, I sympathize — I find screen's conceptual model to be totally foreign and disorienting, even though AFAICT you can get the same shit done in either program.)
posted by nebulawindphone at 10:04 AM on April 13, 2012


(oops, I got distracted; thanks for all the ideas, everyone! I'll be trying them out...)
posted by kaibutsu at 2:25 PM on April 22, 2012


nebulawindphone: "This may just be a terminological/conceptual issue. I didn't use screen for long enough to really fully grok its window/pane/buffer model, but the model in tmux is different, and this is one of the differences."

In the real world a window is divided into grids or muntins, and each of these holds a pane.

emacs did windowing first (it has had "windows" as it currently uses them since 1975): here a frame holds one or more windows. Each window can view one buffer. When you split a frame, you can hold more windows (what in the real world would call window panes). This is clearly a bit out of sync with the real world referent of the metaphor. But just like with a real window you can put the same pane (buffer) in various defined places (and since this is the digital realm, we can put the same buffer/pane in more than one place at once).

As usual, emacs did it first, and then, once other software had the feature, the new standard was what everybody else agreed on: call the top level objects windows, each one or more frames inside it. The frames hold views. emacs stayed with the old terminology and behavior, since they had been doing it that way for a decade already. What is lost with the standard newer model is the separation of the holding object and the contents - each "window" here holds one view, and the idea of switching views inside one window, or sharing the same view in two windows is eliminated (or obscured - the exception here being MDI where one top level window has multiple views, but even in an MDI you wouldn't usually expect to be able to put the same contents in different containers simultaneously - the only way to do this is duplicate the containers).

screen uses the emacs model where the contents and display are abstracted from one another, you can show the same contents in multiple displays in one screen session. tmux uses the more modern / "natural" (and IMHO much less usable) MDI model, where each view is in only ever one place, and closing a window also closes all it's panes, etc.
posted by idiopath at 1:04 PM on April 23, 2012


I just got corrected by an online friend, tmux can put one pane in two locations with the "link-window" and "join-panes" operations.
posted by idiopath at 1:28 PM on April 23, 2012 [1 favorite]


Ha! I was puzzling over the phrase "in the real world" — you know, trying to figure out what OS/editor/multiplexer/whatever you were presenting as the Real One that Real Users use. It took me a minute to realize that you were actually referring to, you know, the real world.

Apparently I need to get out more?

Anyway, it's good to know that my inability to grok screen and my inability to grok split frames in emacs are really just the same problem.
posted by nebulawindphone at 1:56 PM on April 23, 2012 [1 favorite]


« Older Pie a la Apple - a recipe for completely edible, s...  |   Sure, the follies of art-spea... Newer »


This thread has been archived and is closed to new comments