>_
October 2, 2012 4:39 PM   Subscribe

To understand the command line you must first understand Unix...
posted by Artw (145 comments total) 69 users marked this as a favorite

 
"Basics of the Unix Philosophy"
posted by Egg Shen at 4:46 PM on October 2, 2012 [5 favorites]


It's a Unix system. I know this.
posted by mochapickle at 4:55 PM on October 2, 2012 [23 favorites]


Isn't it more or less just flying around over a bunch of file cabinets until you find the one you want and clicking on it, so that it does whatever you want?
posted by ShutterBun at 4:56 PM on October 2, 2012 [6 favorites]


Spoiler warning: Fourth paragraph reveals the ending of the Lord of the Rings. Seventh paragraph reveals the model and operating system of the author's phone.
posted by wam at 5:01 PM on October 2, 2012 [2 favorites]


This is great. Thank you for this post. Finally I understand why grep is called grep. I didn't even know that was bothering me, but actually, it kind of was, it being a proper verb now and everything. After that the stuff about the hjkl keys and so on was just a bonus..
posted by motty at 5:07 PM on October 2, 2012 [2 favorites]


And as observed before, if somebody else makes "grep" better, your "vig" command will improve as well!

An equally-if-not-more common scenario: If somebody else makes grep "better", your vig command will "improve" as well!
posted by Wolfdog at 5:07 PM on October 2, 2012 [22 favorites]


There's not that much about unix in the article. It's mainly about programs that ran on unix like the Bourne shell.
posted by Obscure Reference at 5:08 PM on October 2, 2012 [3 favorites]


Also, fgrep is not fast grep, but egrep is, although that wasn't it's original goal, which was to be the opposite of fgrep, unless you're on a GNU system, when it's just grep again, with more options.

And, yeah, this seems more about the Unix command line ecosystem rather than Unix itself. (And which Unix? These days, even the SRV/BSD split is pretty much meaningless.)
posted by Slap*Happy at 5:15 PM on October 2, 2012 [1 favorite]


Yeah, why read a lightweight article like that when you can read the really entertaining articles about UNIX.
posted by benzenedream at 5:15 PM on October 2, 2012 [7 favorites]


Knowing how the utilities in unix got their names doesn't really tell you anything about unix.
posted by empath at 5:15 PM on October 2, 2012 [8 favorites]


I went to see The Bourne Identity when it was in the theatres. I had avoided all of the publicity for the movie; all the possible spoilers, all the interviews, junkets, photo spreads in glossy magazines. I wanted to be totally untainted when I sat in front of that screen, a blank slate ready for the story of /bin/sh.

I sad with a small bag of popcorn to my side, on the edge of my seat. The house lights lowered. Fishermen find a strange man with gunshot wounds. I am puzzled, but stick with it. Perhaps this is an analogy for pipelines.

A body is planted in a morgue. I am very confused by this point. If this is a metaphor, it is a very obtuse one. I glance down at my watch, then my ticket, wondering if I went in the wrong door.

No-one ran id once.
posted by subbes at 5:20 PM on October 2, 2012 [38 favorites]


Knowing how the utilities in unix got their names doesn't really tell you anything about unix.

I don't agree. I think this article tells you nearly every thing you need to know about the Unix philosophy: smug, arcane, obtuse, up its own arse with bizarre, invented terminology. The best bit is about how 'less' got its name:

But 'more' was limited. If you realised you wanted to go back to a part of the file that had already scrolled past, you had to re-run the command. An upgrade was needed, and a better pager was created: One that could scroll up and down; and handle useful things like searches. And the name? Well, it was like 'more' but it did more.. and as everyone knows, less is more. So 'less' was born.

A juvenile pun; a bad joke. Only in the backwards, user-hostile world of Unix would such a convention not just survive, but thrive.
posted by kjh at 5:43 PM on October 2, 2012 [5 favorites]


Neal Stephenson did a fascinating (short) book on this topic, 'In the Beginning was the Command Line'. You can get it on his website here.
posted by Sebmojo at 5:45 PM on October 2, 2012 [8 favorites]


Fun is backwards and user hostile.
posted by helicomatic at 5:45 PM on October 2, 2012 [6 favorites]


See also yacc (Yet Another Compiler Compiler) which begat bison.

Or elm -> pine

There are probably other examples.
posted by Pruitt-Igoe at 5:49 PM on October 2, 2012


A juvenile pun; a bad joke. Only in the backwards, user-hostile world of Unix would such a convention not just survive, but thrive.

This is completely, utterly false. Languages have grown and extended that way since people spoke words. ALL sequences of letters are arbitrary. The people who named Unix tools used associations that made sense to their audience. We are not that audience, true, but we aren't anyone special, there will be audiences after us who will be just as confused over the things we find obvious as we are over Unix naming conventions.
posted by JHarris at 5:54 PM on October 2, 2012 [23 favorites]


A juvenile pun; a bad joke. Only in the backwards, user-hostile world of Unix would such a convention not just survive, but thrive.

I guess we shouldn't mention most.

Why settle for less?
posted by RonButNotStupid at 5:54 PM on October 2, 2012 [2 favorites]


It's mainly about programs that ran on unix like the Bourne shell.

I wonder how soon after using the past tense wrt to unix you used your unix-powered phone.
posted by DU at 5:55 PM on October 2, 2012 [8 favorites]


Only in the backwards, user-hostile world of Unix would such a convention not just survive, but thrive.

Only in the backwards, user-hostile world of non-Unix would the name of a program be used to judge an operating system.
posted by DU at 5:56 PM on October 2, 2012 [13 favorites]


Alright, time for the anti-command line people to come out and tell us how the people who keep the critical things that computers do happening are all assholes for using the command line

oh too late
posted by sonic meat machine at 5:56 PM on October 2, 2012 [19 favorites]


> UNIX video from 1982

"There is a crying need for useful software, we just do not have enough people to write all that software."
posted by four panels at 6:15 PM on October 2, 2012 [5 favorites]


I think the real magic lies with STDIN and STDOUT. The command line itself only provides a quick, easy and efficient way of linking one program's simple interface with another, but it was the constraints of a text-based environment that forced the evolution and use of these simple interfaces.
posted by RonButNotStupid at 6:15 PM on October 2, 2012


Rule of Repair: When you must fail, fail noisily and as soon as possible.

I was nodding my head vigorously through the whole catb link but this one in particular. Even my very-unixy boss doesn't get this one. He wants to "just put in some reasonable values and move on". NO NO NO NONONONONONONO! Fail, early and often!

I bet web browsers would be like 100 lines of code if they didn't have to handle all the junk they "helpfully" allow web developers to get away with.
posted by DU at 6:22 PM on October 2, 2012 [4 favorites]


The first real unix-like OS I ever used was actually Xenix, on an Altos86 in our school's computer lab in the early 80s. The login shell for students was a text based menu from which you could choose a BASIC interpeter, crude spreadsheet app and a few other things.

Entering "!" would drop you to /bin/sh. I felt like I was in TRON, except without the shiny bikes. I read man pages, played games and fooled around with the shell utilities. It felt as close to alchemy as I'll probably ever get, and it was awesome. Sometime before I graduated, it was discovered that I was lacking a few Computer-related credits, so I was put into an Programming Intro class. I checked the syllabus, knocked out the entire quarter's worth of assignments in about 2 weeks, and spent the rest of the time screwing around. Good times. It's been a lifetime love affair ever since.

Doesn't matter how many shiny gadgets enter this house, or how pretty their UIs are, or how well they're sealed against easy tampering or modding...there will always be a box, somewhere on this network, running BSD or Linux.

(goes outside to yell at cloud)
posted by jquinby at 6:22 PM on October 2, 2012 [10 favorites]


To attain true Unix wisdom, one must understand what an inode is, and why inodes are important.
posted by Chocolate Pickle at 6:25 PM on October 2, 2012


In fact, this appears later:

The original HTML documents recommended “be generous in what you accept”, and it has bedeviled us ever since because each browser accepts a different superset of the specifications. It is the specifications that should be generous, not their interpretation.

Yep.

I read man pages, played games and fooled around with the shell utilities.

I remember the first time I (accidentally!) typed cd .. while in my home directory on the school's Unix system. "Whoa, what are all these secret, hidden directories for?? /usr/bin/games?!"
posted by DU at 6:25 PM on October 2, 2012 [2 favorites]


A juvenile pun; a bad joke.

A mnemonic that makes it easy to remember.

Want to output a file from back to front? The program for that is called "tac". Why? Well, because it's "cat" backwards, obviously.

Might seem silly, but the fact of it is this: now that I've said that, you will never forget it, and that's worth something.
posted by mhoye at 6:27 PM on October 2, 2012 [13 favorites]


Chocolate Pickle : To attain true Unix wisdom, one must understand what an inode is, and why inodes are important.

Bah! You kids and your inodes. Who needs to know metadata about a file? The card says it all.

/ That said... Number your cards! Number them, or you'll live to learn why you should have numbered them! Say, if we could store the number in some sort of metadata...
posted by pla at 6:28 PM on October 2, 2012 [2 favorites]


Might seem silly, but the fact of it is this: now that I've said that, you will never forget it.


Need to produce tremendous amounts of affirmative data? Use the "yes" command, which prints the character "y" ad infinitum. Also useful for load testing, btw.
posted by jquinby at 6:31 PM on October 2, 2012 [1 favorite]


More to the point, however dumb the command names are, you only have to learn them ONCE. Unlike with many popular GUI-based programs where they "improve" the "look and feel" with every release. I'd much rather have a sharp, one-time learning curve than a knowledge-obsolescence treadmill, however gently inclined.
posted by DU at 6:34 PM on October 2, 2012 [12 favorites]


My favorite take on the command line is from Neal Stephenson's long essay which Sebmojo linked northward, where he compares operating systems to power drills and concludes that Unix is the Hole Hawg of operating systems -- it has unlimited power but will kill you if you aren't careful with it.
posted by localroger at 6:41 PM on October 2, 2012 [3 favorites]


At first glance, I thought the link text was "To misunderstand the command line...", which seemed reasonable.
posted by isnotchicago at 6:42 PM on October 2, 2012 [1 favorite]


A juvenile pun; a bad joke. Only in the backwards, user-hostile world of Unix would such a convention not just survive, but thrive.

Oh, I don't know. The juvenile puns "Apple Macintosh" and "Excel" seem to have thrived reasonably despite existing outside the backwards, user-hostile world of Unix.

(Although admittedly the Apple Macs have been built atop that user-hostile world for the past decade or so.)
posted by pont at 6:43 PM on October 2, 2012 [1 favorite]


Alright, time for the anti-command line people to come out and tell us how the people who keep the critical things that computers do happening are all assholes for using the command line.

Oh, horsehockey. You need to know how to navigate to /etc, how to run a tail, how to run a grep, and five or six basic vi commands and you, too, can be one of the people who keep the critical things computers do happening. Most Unix command line tools are quaint pains in the butt: You're just going to copy the config file from the terminal window into TextWrangler and then back again, and you know it. You don't even need "man" anymore, just google it, someone's already got a tutorial on their blog somewhere that explains it better.

(I'm full of oats because I just solved an issue with ulimit on Solaris 8 as a freebie for a family member who does web development on the Worst Server in the World. Feels good to get back to working with something reasonably insane instead of batshit stupid, which is the command line interface on modern security appliances and network gear.)
posted by Slap*Happy at 6:45 PM on October 2, 2012 [2 favorites]


Slap*Happy: "You're just going to copy the config file from the terminal window into TextWrangler and then back again, and you know it."

Why would I add extra steps? Vim works fine, thanks.
posted by wierdo at 6:53 PM on October 2, 2012 [6 favorites]


Eh, I write software that runs on Linux for a living. I use a lot of commands pretty regularly. Some less willingly than others. (Damn you, awk...)
posted by sonic meat machine at 6:55 PM on October 2, 2012


I feel like I read this article as a teenager in the nineties, but I guess not. I don't know what it is about Unix people that they think the principles of abstraction and linking and data-agnosticism and interopterability are only ever in evidence in this year's sexy new ksh derivative, but it's a thing.

More to the point, however dumb the command names are, you only have to learn them ONCE. Unlike with many popular GUI-based programs where they "improve" the "look and feel" with every release. I'd much rather have a sharp, one-time learning curve than a knowledge-obsolescence treadmill, however gently inclined.

Apple used to be amazing about not doing this, and keeping the UX consistent across different programs and OS versions and program versions and making all their third-party liscensees do the same, and then they stopped. :(
posted by The Master and Margarita Mix at 6:55 PM on October 2, 2012 [3 favorites]


I feel like this is the right place to put this:

Unix has always lurked provocatively in the background of the operating system wars, like the Russian Army. Most people know it only by reputation, and its reputation, as the Dilbert cartoon suggests, is mixed. But everyone seems to agree that if it could only get its act together and stop surrendering vast tracts of rich agricultural land and hundreds of thousands of prisoners of war to the onrushing invaders, it could stomp them (and all other opposition) flat.

Unix: the hole hawg of operating systems
posted by Freen at 6:56 PM on October 2, 2012 [1 favorite]


More to the point, however dumb the command names are, you only have to learn them ONCE. Unlike with many popular GUI-based programs where they "improve" the "look and feel" with every release. I'd much rather have a sharp, one-time learning curve than a knowledge-obsolescence treadmill, however gently inclined.

If you don't like the command names, you can always use aliases. They don't mean anything and they aren't important. Wasting so many words on them seems silly.
posted by empath at 6:58 PM on October 2, 2012 [1 favorite]


Long ago, as the design of the Unix file system was being worked out, the entries . and .. appeared, to make navigation easier. (...) Second, and much worse, the idea of a "hidden" or "dot" file was created. As a consequence, more lazy programmers started dropping files into everyone's home directory.

Hmm... ~/ ls -a

.CFUserTextEncoding
.DS_Store
.Extensis
.SFCore
.TemporaryItems
.Trash
.adobe
.bash_history
.cups
.dropbox
.dvdcss
.hsoftdata
.irssi
.java
.lesshst
.ssh
.tgpskey
.viminfo
.wapi

Whoa. I know what maybe half that stuff is doing there. Let's see.. .CFUserTextEncoding - no idea. .DS_Store - Apple's window state info. .Extensis - no idea what version of Suitcase put that there, or if I need it. .SFCore - no idea -- Suitcase Fusion? .TemporaryItems - OS X hides stuff from me in there. .Trash - Oh, I got this one! .adobe - how manygoddam directories do those people need? Looks like they keep user authentication info in there. Why there? .bash_history - seems self-explanatory. .cups - Common Unix Printing System - I knew this! - it was a big deal when Apple added cups in 10.2 -- we could print things! .dropbox - looks like a lot of stuff that should be in ~/Library/Preferences or ~/Library/Application Support .dvdcss - no idea. .hsoftdada - no idea. .iriis - no idea. .java - I bet this has something to do with Java. I'm smart that way. Why here, though? .lesshst - no idea. .ssh - authentication keys for remote servers are stored in here. No idea why it should be hidden. .tgpskey - no idea. .viminfo - not a vim user, but looks self-explanatory. .wapi? Wapi?
posted by Devils Rancher at 6:59 PM on October 2, 2012 [2 favorites]


But everyone seems to agree that if it could only get its act together and stop surrendering vast tracts of rich agricultural land and hundreds of thousands of prisoners of war to the onrushing invaders, it could stomp them (and all other opposition) flat.

Well, now that unix underpins OSX and iOS, that seems to be what's happening.
posted by mhoye at 7:17 PM on October 2, 2012 [1 favorite]


The Computer Chronicles: UNIX (1985)
posted by helicomatic at 7:17 PM on October 2, 2012


Yeah, why read a lightweight article like that when you can read the really entertaining articles about UNIX.


Really really entertaining articles about UNIX.

posted by Runes at 7:21 PM on October 2, 2012 [2 favorites]


.lesshst - no idea

Internationalization for sales receipts in Canada. Not sure how it got on your Mac.
posted by Lemurrhea at 7:25 PM on October 2, 2012


See, the basic idea of Unix is that each command does one thing, and does it well.

Providing you can remember which flags to set.
posted by ChurchHatesTucker at 7:26 PM on October 2, 2012 [3 favorites]


Internationalization for sales receipts in Canada. Not sure how it got on your Mac.

Or why it needed to be hidden in the home directory - sort of the author's point. Plus now, we have flags yeshidden and nohidden that can hide directories as well. Not sure of the history on that -- just learned about it a year or so ago when ~/Library disappeared upon updating to 10.7.

I really don't know much unix, or have a need to, but I also appreciate his point about the elegance of | for tying things together.
posted by Devils Rancher at 7:31 PM on October 2, 2012


I bet web browsers would be like 100 lines of code if they didn't have to handle all the junk they "helpfully" allow web developers to get away with.

Yeah, and the web would be as successful as Gopher.

Alright, time for the anti-command line people to come out and tell us how the people who keep the critical things that computers do happening are all assholes for using the command line

I don't mind the command line, it's just that Unixy implementations of it are simply so hostile that proponents sound like Stockholm Syndromed battered spouses.

All text-based OS interfaces are not equal. It feels to me like there's a spectrum from, say, JCL to MPW, and the *sh family are way further to the left than the right.
posted by fightorflight at 7:32 PM on October 2, 2012


Plus now, we have flags yeshidden and nohidden that can hide directories as well.

You can dot-hide directories too - just give it a name starting with a full stop.
posted by Dysk at 7:42 PM on October 2, 2012


And when hostility and minimal browsers join together, you get stuff like SURFRAW - Shell Users' Revolutionary Front Rage Against the Web. I think it's great. I would never use it, but I do find the hostility charming somehow.
posted by Lorin at 7:43 PM on October 2, 2012 [3 favorites]


As one who basically started with Apple/DOS/Amiga pre-GUI era, then Unix, and totally skipped as much as geeky humanly possible everything Windows and Mac GUI era... all your crazy assed GUI shell stuff is so complicated, limiting and just plain frustrating to use. George Jetson button clicking finger cramping rat in a maze looking for the right thing to click to get something done insanity. I pity you.
posted by zengargoyle at 7:51 PM on October 2, 2012 [3 favorites]


I'm most interested in the weird convergence of search autocomplete and command lines, currently best exemplified by Github's command bar.
posted by feckless at 7:52 PM on October 2, 2012 [1 favorite]


You can dot-hide directories too - just give it a name starting with a full stop.

Seriously, I learned this the hard way when I upgraded a G4 from OS 9 to OS X. I used to use underscores, tildes and periods to float directories in the alphabetic sorting order, & when I first booted into OS X I was all "BUH? Were are my important files?!?"
posted by Devils Rancher at 7:58 PM on October 2, 2012


Biff's name has a goofier backstory: http://en.wikipedia.org/wiki/Biff#Origin_and_name
posted by wenestvedt at 7:59 PM on October 2, 2012


I don't mind the command line, it's just that Unixy implementations of it are simply so hostile that proponents sound like Stockholm Syndromed battered spouses.

I have a good many gripes with the experience of a thing like Bash, and I would dearly love for the CLI to evolve towards fewer sharp edges and more modern capacities, but this still strikes me as unfair. The fact is that there's not a single general-purpose computer interface in wide use with the particular combination of expressive power, concision, and cultural longevity embodied by the varied descendants of the Unix shell and the ecosystem that has grown up there over decades. Die-hards aren't always particularly willing to acknowledge its considerable flaws, but there is literally no alternative half as viable.

I'm most interested in the weird convergence of search autocomplete and command lines, currently best exemplified by Github's command bar.

I think this is less of a weird convergence than an inevitability. What's Google, or for that matter the URL bar in a web browser, if not a domain-specific command line? The command line has never gone away for long, and will (I suspect) reassert itself in one form or another for as long as the human mind is profoundly oriented towards language.
posted by brennen at 8:06 PM on October 2, 2012 [7 favorites]


Here is how I explain the command line:

You know how at certain restaurants you get a picture of every menu item, or some people with impediments to language usage will have a chart with icons representing common concepts one must communicate? The "normal" way to use a computer is like this - very little learning needed, and very little depth is possible, and it quickly gets very very awkward for anything interesting. On the other hand, the command line is like natural language - it may take literally thousands of times as long to learn, but there is a payoff in expressiveness if your need to communicate ever moves beyond what you can convey by pointing at picture.
posted by idiopath at 8:32 PM on October 2, 2012 [9 favorites]


You can dot-hide directories too - just give it a name starting with a full stop.

There are at least four different ways to hide files on a Mac, and some of them are horrible old hacks from the bad old days of non-plus HFS.

Don't confuse HFS+ with Unix; it's a terrible janky filesystem, the worst thing about OSX in my opinion.
posted by mhoye at 8:39 PM on October 2, 2012 [8 favorites]


I'm most interested in the weird convergence of search autocomplete and command lines, currently best exemplified by Github's command bar.

The vimperator and pentadactyl addons for Firefox are also in that ballpark. The vi-esque shortcuts and command line have ruined me for other browsers.
posted by Lorin at 8:44 PM on October 2, 2012 [1 favorite]


DU: "More to the point, however dumb the command names are, you only have to learn them ONCE. Unlike with many popular GUI-based programs where they "improve" the "look and feel" with every release. I'd much rather have a sharp, one-time learning curve than a knowledge-obsolescence treadmill, however gently inclined."

There are still meaningful differences between GNU and BSD, and the toolchain is actually still evolving. Even utilities like Grep have added features over the past few years, which can be bewildering if you sit down in front of an older system, and things don't work like you expect them to.

Also, I'd strongly disagree with the notion that the Unix learning curve is a "one time" ordeal. I've been using Unix on-and-off for 10 years, and I'm still learning (the curve is sharp though, which can impede further learning once you've gotten comfortable). The GNU coreutils package alone contains 94 different commands (each of which has about a dozen switches/parameters).

Don't thumb your nose at GUIs. They have their uses, just like the command line does.
posted by schmod at 8:45 PM on October 2, 2012 [2 favorites]


As one who basically started with Apple/DOS/Amiga pre-GUI era

The Amiga had a pre-GUI era?
posted by dumbland at 8:59 PM on October 2, 2012


I worked on linux systems for a while, and that servitude was rewarded by one of the saddest phrases I have ever read in a technical manual.

In a sub-text for the command wait is the caution:
Not all the processes of a pipeline with three or more stages are children of the shell, and thus cannot be waited for.

If read without context, you might weep in mourning for the poor mites...
posted by arzakh at 9:08 PM on October 2, 2012 [4 favorites]


I've talked about this numerous times now, but this is one of those topics where I'm always happy to repeat myself.

The thing that I believe is most important about Unix is that there's no bright line between 'users' and 'programmers'. In Windows and Mac OS, you have to explicitly Decide To Be A Programmer. You obtain programming tools, often very, very expensive tools, install those, and then start a process that is completely, utterly divorced from the normal process of using the machine. The complexity jumps by at least a hundred times, maybe a thousand, and writing a functioning program is a long and painful process.

At the command line, it's not really like that. Users become programmers, almost by accident. As soon as you start piping one program into another, you're doing a very simple form of programming. As you work with the command line, accomplishing whatever it is you need to do, you will naturally come up with ways of making it faster and more efficient. You can change your operating environment freely to suit your specific needs and idiosyncrasies. As you figure out sequences of commands that do things you need, you can simply record them into a text file, call it something memorable, slap an 'executable bit' on it with chmod, and put it somewhere in your path. Voila, a new word in your environment, that works just like the old ones did.

You can then get into scripting languages, like Perl and Python and Ruby, which are powerful and friendly, and can be developed completely with the basic tools you've already learned. Writing a Python program is basically the same overall process as writing a shell script, just using different language syntax. And, in many ways, these languages are much easier than shell scripting, so things that would baffle you on the command line suddenly become easy. It takes a fair bit of study to understand the languages, but once you get them, you can immediately integrate that power right back into your environment, with almost no friction. The programs you write become part of your environment, just as much as anything that came with the OS originally.

If you get as far as programming GUI applications, that's just about as difficult as it is on Windows or the Mac. It's probably harder, actually. Programming GUIs seems to be completely separate, and far more difficult, than the process of using them. But that's most emphatically not true at the command prompt, and that's part of why I think kids should learn that stuff. Any user past his or her first baby steps automatically becomes a programmer, and his or her control over the system is limited only by brainpower and available time. And, on an open source system, everything is open, all the way down to the bare metal. It's ridiculously complex compared to the 8-bit era, but every line of code that's supporting you, when you're running on open source, is available to you to inspect and improve. It's a natural outgrowth of the command line -- there's no glass ceiling, keeping you out of some corporation's hair. You can have as much power as you can handle. Everything, all the way down, all the way up, everything is freely changeable by anyone.

That's why I find OS X distressing... it didn't bother me at first, but as Apple has steadily moved toward closing things down, screwing up the App Store for the Mac so that the apps can't be powerful anymore, I perceive this as a direct attack against my ability to think. Our ability to take control of a Mac, and then to share that control, is being steadily circumscribed, and now it looks like Microsoft is headed that same direction, with Windows 8.

It doesn't have to be that way.
posted by Malor at 9:26 PM on October 2, 2012 [27 favorites]


dumbland: The Amiga had a pre-GUI era?

All Amigas had a GUI in ROM, but they had a powerful CLI language. It didn't sing like the rest of the system did, but it was pretty functional. Stronger than DOS, not as powerful as Unix. It was perfectly possible to use the Amiga without ever loading the Workbench, and in my later years on the platform, that's what I generally did.

However, even then, it was still mostly just launching programs that used graphics and drew their own GUIs, as opposed to running truly command-line utilities. Even launching from the command line, it was a very visually-oriented system.
posted by Malor at 9:36 PM on October 2, 2012 [1 favorite]


It takes hundreds of hours to become fluent in the command-line. That makes no sense for the great majority of users. If you really want to feel the power, really get to know the innards of your machine, and are willing to invest that many hours, learn a programming language. You'll wind up with something you can show to people that they can understand and appreciate. You can contribute to the FOSS movement and gain the appreciation of thousands.

Unless you know why you need to, the rewards for CLI-FU are vanishingly small.
posted by Twang at 9:45 PM on October 2, 2012


After watching that video that four panels linked, I was reminded of this document I was reading the other day... an article by Dennis Ritchie on the origins of the C language (and seeing some of the evolution between B, BCPL and C)

Also? Fuck man, I <3 Programmer Beards.

Speaking of shells, I remember when MS came out with Monad/Powershell and thought that was like the epitome of shells, being able to pipe non-textual data between programs. It's a shame it didn't really take off that much.
posted by symbioid at 9:45 PM on October 2, 2012


Pipes are beautiful, but man—those godawful fucking shells. Two feet outside of the door of UNIX's good design, the weeds grow tall.
posted by fleacircus at 9:47 PM on October 2, 2012 [1 favorite]


I was thinking a day or two ago about how the two main smartphone platforms (iOS, Android) are UNIX or Linux-based. Nobody ever thought we'd be carrying around little touchscreen *nix systems, eh?
posted by mrbill at 9:55 PM on October 2, 2012


Yeah, the fact that the shell program expands wildcards is an endless source of problems. The shell should just pass through the commands verbatim, and the programs should call some kind of system function to resolve wildcards, if they choose. With the shell getting its sticky fingers all over everything, trying to figure out the various levels of escapes required to send a wildcarded command line to a remote machine is mind-numbing tedium.

Basically, you can't always (or at least easily) tell what arguments a command is actually getting. I don't know why they thought that was a good idea. I really, really don't.
posted by Malor at 9:59 PM on October 2, 2012 [2 favorites]


All Amigas had a GUI in ROM, but they had a powerful CLI language.

That was my understanding, yeah. Even the 1000 had Workbench.

I just recently picked up the Amiga Forever pack and disturbed myself a bit with just how much AmigaDOS I still remember.

And that beautiful speech synthesiser.
posted by dumbland at 10:04 PM on October 2, 2012


It takes hundreds of hours to become fluent in the command-line. That makes no sense for the great majority of users. If you really want to feel the power, really get to know the innards of your machine, and are willing to invest that many hours, learn a programming language.

One of the most reliable ways I've yet found to weed out programmers is to discover whether they know their way around a command line. The ones who have never bothered to try are generally deficient and incurious, and the ones actually incapable of learning are just flat-out hopeless.

At any rate, you're drawing a distinction without a difference. Any shell worth using is likely a programming language, albeit usually one optimized for interactive use and programming in the small.
posted by brennen at 10:17 PM on October 2, 2012 [3 favorites]


On "the rewards of CLI-FU are vanishingly small": they don't have to be. Most of what you do on a machine is manipulating data. A system of massaging data "by hand" in certain codified ways to get a desired result is so common an operation that it has a name: "worfkflow." There is little reason that has to be done in GUIs with files moving across programs taking a half hour at a time instead of automatically by your machine; it's gruntwork, moving the cursor to the same options, opening the same dialog boxes, clicking the same checkboxes, entering the same values and clicking the same buttons, over and over and over again. You could say that most people don't need to do that, sure, but most people only use their machines as content consumption devices anyway, an attitude that's been encouraged, perhaps, by the limited nature of GUI software.

On piping non-text data between programs: you don't need Powershell to do that. You use the tools to create files, and you refer to those files by name. If you can assign a name to it, you can handle it, a paradigm so powerful that it leaks into our thinking so that often we treat names as if they were the things named.

But what would be nice, now? More visual reminders to help you keep in working memory the details of what you're working on. Spreadsheets now, while you're entering formulas, will highlight the cells they're dealing with. I'd like it if there was an option (an option , mind you) to have those kinds of popup reminders of what commands you're working with. Lists of available tools? A window with a man page? A quick list of command-line parameters? A view into the file system? Syntax coloring? I think that would be useful to be, although I am loathe to suggest everyone would want it.
posted by JHarris at 10:21 PM on October 2, 2012 [5 favorites]


Malor: I'm of the opposite opinion. I love that the shell does glob expansion, and that the escaping rules are simple. It can get tricky with tools like ssh where you might have to double-escape, but I'll take that over the inconsistent brokenness in Windows.
posted by Pruitt-Igoe at 10:26 PM on October 2, 2012 [4 favorites]


But what would be nice, now? More visual reminders to help you keep in working memory the details of what you're working on. Spreadsheets now, while you're entering formulas, will highlight the cells they're dealing with. I'd like it if there was an option (an option , mind you) to have those kinds of popup reminders of what commands you're working with. Lists of available tools? A window with a man page? A quick list of command-line parameters? A view into the file system? Syntax coloring? I think that would be useful to be, although I am loathe to suggest everyone would want it.

Yeah, the ability to manage context is everything. This is partly why robust version control systems like git are becoming so ubiquitous (and why, I suspect, that sort of functionality will eventually migrate into the larger environment, the filesystem, etc.). I've got some stuff in my zsh config to make me aware of git context, directory history, etc., but I badly want more of these kinds of things.

I would kill for a shell/terminal environment with a set of plugins to natively display images and HTML, graphically format structured files, navigate directory hierarchies with previews, etc. Something with the best attributes of zsh, [pick your GUI file manager], and [pick your favorite web browser] rolled into one thing.
posted by brennen at 10:35 PM on October 2, 2012 [3 favorites]


It takes hundreds of hours to become fluent in the command-line.

It takes hundreds of hours to get even reasonably good at ping pong. A few hundred hours is nothing, in the grand scheme of things. It's been said that most disciplines take about ten thousand hours of study before reaching mastery.

I don't think very many people realize just how insanely complex a modern CPU is. They are, very probably, the most intricate creations by mankind. Ever. They have billions of components. It's engineering of the caliber that launched the Space Shuttle, but focused into a chip sitting under your desk.

How could it not take thousands and thousands of hours to get really good with a machine like that? I mean, sure, these machines are now so powerful that you can ignore how powerful they are, and just twiddle some bits on the screen and call yourself a 'computer user', but that's like playing with a seatback entertainment system in a 747, and calling yourself a pilot.
posted by Malor at 12:15 AM on October 3, 2012 [6 favorites]


To understand the command line you must first understand Unix...

...now you have two problems.
posted by comealongpole at 1:14 AM on October 3, 2012 [7 favorites]


It takes hundreds of hours to become fluent in the command-line.

Pish and tosh! Tosh.0 even! (OH HOW I HATE THAT SHOW)

It takes hundreds of hours to become fluent in real languages. You can write and understand command lines in less than an hour. Their implications take a very long time to master, but that's true of most languages, and you don't need to know much to construct useful command lines. Most native English speakers aren't particularly masterful in their use of their language.
posted by JHarris at 1:16 AM on October 3, 2012


I would kill for a shell/terminal environment with a set of plugins to natively display images and HTML, graphically format structured files, navigate directory hierarchies with previews, etc. Something with the best attributes of zsh, [pick your GUI file manager], and [pick your favorite web browser] rolled into one thing.

I wrote a post on a closely related project not too long ago.

My work is in one shell or another, and I'm often having to navigate between a variety of file formats and cryptic interfaces to them. Semantic data and better presentation that something like TermKit does would enhance the shell and make it even more powerful for scientists like myself, to get things done more efficiently.
posted by Blazecock Pileon at 1:21 AM on October 3, 2012 [1 favorite]


Knowing how the utilities in unix got their names doesn't really tell you anything about unix.

I don't agree. Unix has always struck me as very similar to Drosophila genetics; it has a nomenclature that is above all terse, but which also reflects an accretive history. Every name has a story; sometimes, knowing those stories can be helpful to a student by making it easier to infer what something does; at other times, not so much.

But in both cases, this jargon can seem to an outsider to be hostile; the arcane in-jokes of an insular priesthood:

A juvenile pun; a bad joke. Only in the backwards, user-hostile world of Unix would such a convention not just survive, but thrive. — change a word and this sentence is as likely to be heard in the biology department as in computer science.

In both cases, an important part of any introduction to the subject is an explanation of how the language got to be so weird, a reassurance that it doesn't always make sense and not to worry. It's important to learn the culture as context for the technical stuff.
posted by nowonmai at 1:55 AM on October 3, 2012 [4 favorites]


The UNIX way was/is the last computer system that expects the user to have the command reference manual handy. If you have one nearby, you are fine. A computer was a machine to process data and get stuff done.

Newer operating systems don't really expect that. Computing were turned into something where new users were expected to learn by discovery. You buy one and then figure out what kind of cool stuff it can do. Completely misunderstanding that only a very small percentage of people actually enjoy that sort of unstructured learning process when they are trying to get something done.
posted by gjc at 2:39 AM on October 3, 2012 [2 favorites]


"When in doubt, use brute force."

Man, I guess I really do have a lot of doubt in my life, huh?
posted by cthuljew at 2:55 AM on October 3, 2012


When you first encounter Unix, it seems confusing and stupid. After a steep learning curve, you gain an appreciation for its power - it can do everything. At this point you may feel inclined to write a blog post extolling its virtues. And that's great.

However, after you use it for decades, you start to see chinks in the armor. Problems with the fundamental design philosophy that are impossible to fix, because you would need to change everything. At this point, if you are an optimist, you write a new OS fixing these issues, like Plan 9 or HURD. And then you see your dreams dashed, because Unix may be flawed, but it is good enough, and you can't convince the entire industry to switch over when none of their software runs on it.

If you are more of a realist, you resign yourself to your fate. But you're not going to go down quietly. So you write a beautifully illustrated, 360 page rant, and call it The UNIX-HATERS Handbook.
posted by foobaz at 3:05 AM on October 3, 2012 [3 favorites]


It takes hundreds of hours to get even reasonably good at ping pong. A few hundred hours is nothing, in the grand scheme of things. It's been said that most disciplines take about ten thousand hours of study before reaching mastery.

Seriously. If you use a computer as a job, that's about 2 months of 40-hour work weeks. Meanwhile, the time saved by not having to spend your time and attention on clicking buttons and "pointing at picture menus" is enormous.

...a beautifully illustrated, 360 page rant, and call it The UNIX-HATERS Handbook.

I haven't been able to read the book myself, but a coworker has told me more than once that this should have been called The C Shell Haters Handbook.
posted by DU at 4:04 AM on October 3, 2012


Also:

"Take the Traders' method of timekeeping. The frame corrections were incredibly complex — and down at the very bottom of it was a little program that ran a counter. Second by second, the Queng Ho counted from the instant that a human had first set foot on Old Earth's moon. But if you looked at it still more closely ... the starting instant was actually about fifteen million seconds later, the 0-second of one of Humankind's first computer operating systems."

-A Deepness in the Sky by Vernor Vinge
posted by cthuljew at 4:31 AM on October 3, 2012 [7 favorites]


> UNIX video from 1982

"There is a crying need for useful software, we just do not have enough people to write all that software."


Oh man, that was brilliant. People will point out how stupid I am, but I finally figured out why it's called the shell: because it surrounds the kernel. The penny drops.

Also, I liked that guy in the blue shirt with the keyboard on his lap. He explained things exceedingly well; he spoke clearly and showed examples. I hope he went far. What was his name again? Brian Kernighan?

Oh... wait.

To top it off, he looked good too. Is there no end to his brilliance?
posted by milkb0at at 4:43 AM on October 3, 2012 [4 favorites]


In Windows and Mac OS, you have to explicitly Decide To Be A Programmer. You obtain programming tools, often very, very expensive tools, install those, and then start a process that is completely, utterly divorced from the normal process of using the machine.

Automator and Applescript comes with every copy of the Mac. No crunchy-nutty CLI windows or hairy-scary mega-IDE's required.

This bugs me, actually. People who want to learn how to program will be able to do more stuff in less time learning Automator and Applescript than they will using Python or Ruby or Javascript... and the skills they pick up will absolutely translate into "traditional" programming, should they pursue it.
posted by Slap*Happy at 6:07 AM on October 3, 2012 [2 favorites]


Artw: "To understand the command line you must first understand Unix..."

It's Neckbeard Carl Sagan!
posted by schmod at 7:04 AM on October 3, 2012


The fact is that there's not a single general-purpose computer interface in wide use with the particular combination of expressive power, concision, and cultural longevity embodied by the varied descendants of the Unix shell and the ecosystem that has grown up there over decades.

This is equally true of Perl+CPAN though, but it doesn't mean there shouldn't be strenuous efforts to come up with something better. I think half the problem is the people who have spent months learning the unix shell and will now argue that carrying its many chains help them to be fitter and stronger.

There are so many ways it could be better in the modern era. Off the top of my head, stopping pretending that the user is interacting primarily via a 1970s teletype machine would be one advance. But even suggesting that requires you break down a forest of inherited assumptions, starting with "but that would mean I couldn't ssh into any box any where", which is a problem PowerShell (among others) solved much more elegantly (the commands run locally, the effects take place on the server).

On the other hand, the command line is like natural language
Ah, the patronising analogy. Except it's nothing like true. In the restaurant example, the command line is actually like the snooty French waiter who refuses to even attempt to understand your "botai de low" until you pronounce "l'eau" with pitch-perfect precision. Sure, it's quicker after a few hours when your pronounciation is correct, but then you want bread as well and you're back at square one. Meanwhile the menu-pointers have eaten and left.
posted by fightorflight at 7:24 AM on October 3, 2012 [2 favorites]


(goes outside to yell at cloud)

cat onion > belt
posted by Mr. Bad Example at 7:30 AM on October 3, 2012 [1 favorite]


No, because then your onion becomes the belt, and an onion-belt was not the fashion of the time and of dubious benefit to trouser securement.

What you want is to append an onion to an existing belt:

cat onion >> belt
posted by Slap*Happy at 7:44 AM on October 3, 2012 [13 favorites]


Sure, it's quicker after a few hours when your pronounciation is correct, but then you want bread as well and you're back at square one. Meanwhile the menu-pointers have eaten and left.

Yes, I think that was exactly the analogy. With a little investment, you can get exactly what you want whereas without it you can only choose from the menu. If all you want is on the menu, I guess that's great. I'm not sure I'd call it "computing" but words change meaning and if "consuming via a computer" is the new meaning, so be it.
posted by DU at 7:58 AM on October 3, 2012 [1 favorite]



However, after you use it for decades, you start to see chinks in the armor. Problems with the fundamental design philosophy that are impossible to fix, because you would need to change everything. At this point, if you are an optimist, you write a new OS fixing these issues, like Plan 9 or HURD. And then you see your dreams dashed, because Unix may be flawed, but it is good enough, and you can't convince the entire industry to switch over when none of their software runs on it.


Wake me up when HURD or Plan 9 is fully caught up on Posix, and plays nicely with enough video cards to have X11, and somebody wrapped a packaging system around it (RPM/DPKG/ whatever). And I'll switch.
posted by ocschwar at 7:58 AM on October 3, 2012


fightorflight: "Ah, the patronising analogy. Except it's nothing like true. In the restaurant example, the command line is actually like the snooty French waiter who refuses to even attempt to understand your "botai de low" until you pronounce "l'eau" with pitch-perfect precision. Sure, it's quicker after a few hours when your pronounciation is correct, but then you want bread as well and you're back at square one. Meanwhile the menu-pointers have eaten and left."

"is there anything vegetarian available?"

"I want to eat half of this here and have half wrapped for later"

"Does this have nuts? I could die if I ate a single nut. How about shellfish?"

Also, the command line is pedantic about syntax, because the consequences of an unintended or ambiguous interpretation can have consequences not understood by any one command (this command may be harmless, but if you pipe its output to another that removes or changes a file...). That pedantry is caution, the command line makes a big deal about syntax the same way police make a big deal about drivers licenses and not drinking and wearing a seatbelt. But of course if you prefer to ride the bus none of those rules are needed (I personally don't drive a car because I don't want to worry about taking my life in my own hands, and I walk or take the bus, I am in no way looking down on the simpler option if it suffices).
posted by idiopath at 8:21 AM on October 3, 2012 [2 favorites]


fightorflight: "On the other hand, the command line is like natural language"
Ah, the patronising analogy. Except it's nothing like true. In the restaurant example, the command line is actually like the snooty French waiter who refuses to even attempt to understand your "botai de low" until you pronounce "l'eau" with pitch-perfect precision. Sure, it's quicker after a few hours when your pronounciation is correct, but then you want bread as well and you're back at square one. Meanwhile the menu-pointers have eaten and left.


Once you've learned to speak French, though, you'll find you can discuss with the waiter to have the chef prepare you items that are not on the menu, or modified menu items. Trying to learn French purely by speaking to waiters might not be all that easy though.
posted by Dysk at 8:26 AM on October 3, 2012 [2 favorites]


Yes, I think that was exactly the analogy. With a little investment, you can get exactly what you want whereas without it you can only choose from the menu.

It's possible the analogy accidentally exposes the real problem with unix, then. Because it's not a little investment, it's an outlandish amount for the benefit gained, and not nearly enough of it is transferable. It's a collection of brittle flags and switches and toggles and inflections that has to be memorised.

Getting French right is a picnic by comparison, frankly.
posted by fightorflight at 8:30 AM on October 3, 2012


A lot of this would be fixed if man pages weren't written for people who already know how to use the program.
posted by cthuljew at 8:34 AM on October 3, 2012 [4 favorites]


Speaking of shells, I remember when MS came out with Monad/Powershell and thought that was like the epitome of shells, being able to pipe non-textual data between programs. It's a shame it didn't really take off that much.

You may be interested in checking out Xiki; it's textual, but output is structured and live in a way that the stdout of a UNIX command line (which is stone dead and a flat stream of unstructured ASCII by the time it hits the tty) isn't.
posted by acb at 8:43 AM on October 3, 2012 [2 favorites]


I wouldn't say that. However you do need to know the name of the command (I've never had man -k or apropos work very well), and man pages are strongest as a reference explaining individual command line options, not a thorough examination of how/where/why you might use a command.
posted by Pruitt-Igoe at 8:50 AM on October 3, 2012


To leave analogies and enter the realm of use case:

Right now I am using two computers. One has a huge amount of RAM and CPU power but does not strictly belong to me. It is running an OS that needs to be there but I am loathe to interact with directly. I call it "beauty" (that is literally the host name - so called because the interface is shiny but I find it unusable). The other has an interface I have full control over and I know like the back of my hand. I call it beast (so called because the interface is ugly and inconsistent but I can get things done with it very efficiently).

I need to run processes that would bog down beast. Because the command line is so simple (not in interaction but component design), I can use command line tools that have been off the shelf standard in usage since the early '90s to connect to beauty and start up the heavy programs I need to run. Using these command line programs I can also edit files while sitting at beast's keyboard - but those files are actually sitting on beauty and control the behavior of those processes I need beauty to be running. Also, this is all password protected and encrypted. Also, those programs on beauty open up local-only network ports to interact with them, but I can simply connect the ports on one machine to the ports on the other machine so that I can use them directly from beast. I can take one of the computers or the other home with me, or even put one or both to sleep, without interrupting those processes or editing sessions, and seamlessly resume and reconnect without any interruption when back in the office.

There may be GUIs that would give me this functionality, but they are not so stable that they have been making mostly bugfixes / algorithmic improvements for the last decade, they are not open source and free, and they provide no assurance that I won't need to learn a whole new way of doing things in another decade because of some forced upgrade.
posted by idiopath at 8:50 AM on October 3, 2012 [1 favorite]


Because it's not a little investment, it's an outlandish amount for the benefit gained, and not nearly enough of it is transferable.

Complete control over every non-Windows computer on Earth is a) of low benefit and b) not transferable?
posted by DU at 9:09 AM on October 3, 2012 [2 favorites]


This is one of the topics that perpetually inspire hostility on MeFi and I find it one of the strangest.

I like the command-line. I have a textual bias and it fits the way I think better than GUIs. I have an old RSI and sticking to the keyboard is more comfortable than pointing and clicking -- my body has given me a lot of explicit negative reinforcement about that. Having decent fluency in the CLI, I find it much easier to do a large number of things than it would be with a GUI. I don't even know how you'd begin to do some of the things with a GUI that I routinely do with shell pipelines that include Perl one-liners. But, of course, other people's needs are different, and the cost-benefit ratio of learning CLI fluency could look very different. And some tasks will always be ridiculous with a CLI and are better suited to a GUI.

I know my preferences are in a minority and that lots of people love GUIs. I'm glad that's working out for them.

Can't we all just get along?
posted by Zed at 9:09 AM on October 3, 2012 [3 favorites]


To leave analogies and enter the realm of use case:

Umm. I copy and paste stuff into an RDP session all the time - spreadsheet datasets, images, text. I do this most often with our ticketing system, which is Windows 7 only and has an... interesting... text editing system. I do the writeup on the Mac, and it pastes into the ticketing software. No muss, no fuss. I went so far as to automate the process - command-shift-F9 and it will copy and paste for me.
posted by Slap*Happy at 9:10 AM on October 3, 2012


Can't we all just get along?

Well yes, we can. CLI users hate GUIs but they aren't out to destroy GUIs. The reverse is not the case, however. There's constant pressure to provide fewer features ("cluttered UI"), remove "ugly" CLIs, "make it intuitive", etc at every level of computing from users up.
posted by DU at 9:16 AM on October 3, 2012 [3 favorites]


There is no copying or pasting - I am literally editing the file, and all I have to do is save like normal. Inside the same editor I am using to edit all my other files. This is fully integrated into the UI - I don't have a special window that represents a program on another machine, which closes down when the other machine is not connected. This all runs without my even appearing to be "logged in" to the remote machine - none of the standard UI is running, just the network interface, the ssh daemon, the login screen, the shell sshd spawns and whatever programs I explicitly start (which is a benefit when using extremely resource intensive programs and trying to juice every last bit of performance out of a machine).
posted by idiopath at 9:18 AM on October 3, 2012


By the way, anyone who likes the original article should check out Life with UNIX, which is mostly a social history of UNIX, with lots of origin stories like in the article, not a how-to. It's available used shipped for $4 -- a really great value.

(And I spotted one error in the article -- cat is from catenate, not concatenate.)

It blew my mind when I learned that su didn't stand for superuser.
posted by Zed at 9:24 AM on October 3, 2012 [1 favorite]


There is no copying or pasting - I am literally editing the file, and all I have to do is save like normal.

Move the goalposts much? I am not talking about replicating your exact setup, I'm talking about doing the things you claim are impossible - sharing data between machines via network to take advantage of remote computational resources. Achievable in a GUI, and I can even automate large parts of it with Automator or Applescript.
posted by Slap*Happy at 9:27 AM on October 3, 2012


Move the goalposts much? I am not talking about replicating your exact setup, I'm talking about doing the things you claim are impossible...

The point is that idiopath can do these additional things without learning anything extra, merely recombining existing knowledge, whereas you have to learn (and perform) a whole new set of activities to solve this particular problem.

Combinatorial explosion of a spanning set of operations is always going to be more powerful.
posted by DU at 9:31 AM on October 3, 2012 [2 favorites]


This all runs without my even appearing to be "logged in" to the remote machine - none of the standard UI is running, just the network interface, the ssh daemon, the login screen, the shell sshd spawns and whatever programs I explicitly start (which is a benefit when using extremely resource intensive programs and trying to juice every last bit of performance out of a machine).

Through some sshfs and autofs configuration, I have things set up so I can just refer to /mnt/remotemachinename/path/to/file and the ssh connection will be established if it didn't already exist and the filesystem mounted so the right thing happens. For someone who routinely deals with 20 different machines, this has been a godsend. sftp and friends have become a thing of the past.
posted by Zed at 9:31 AM on October 3, 2012


(I just used tramp mode in Emacs for the first time the other day and it was amazing. Current editing session on local machine opens file on remote machine via $MAGIC_I_DIDNT_HAVE_TO_CONFIGURE and lets me operate as normal.)
posted by DU at 9:37 AM on October 3, 2012 [2 favorites]


Slap*Happy: "Move the goalposts much? I am not talking about replicating your exact setup, I'm talking about doing the things you claim are impossible - sharing data between machines via network to take advantage of remote computational resources"

... while using the same workflow I used 10 years ago and knowing I can use the exact same workflow 10 years from now to accomplish this task.
posted by idiopath at 9:46 AM on October 3, 2012


whereas you have to learn (and perform) a whole new set of activities to solve this particular problem.

wat?

No I don't. Pointy-clicky, draggy-droppy, fill out a combo box or two. Progressive disclosure - I don't need to read as much documentation to do it, either.
posted by Slap*Happy at 9:47 AM on October 3, 2012


No I don't. Pointy-clicky, draggy-droppy...

Exactly. To edit files HERE you edit files. To edit files THERE you do the above. You aren't recombining elements, you are coming up with a new workflow to accommodate a minor variable change.

When I edit files THERE I don't need to read any documentation at all. I just point my editor at the file and continue as normal.
posted by DU at 9:51 AM on October 3, 2012


I gave up on this article after spotting one mistake too many.

Because, trust me on this, it's full of 'em. Well, if not full, then sufficiently burdened with them to be untrustworthy. (Not to mention smug and non-rigorous.)
posted by cstross at 9:56 AM on October 3, 2012 [1 favorite]


DU: The point is that idiopath can do these additional things without learning anything extra,

Further, idiopath can do that between any two machines that run Unix; with the magic of secure shell, you can tie remote machines into your local workflow so completely that it's hard to tell where your local machine ends and the remote machine begins.

So, while Slap*Happy's setup is perfectly functional, it's probably specific to those two operating systems, connected in that exact order; Mac client, Windows server. But if he wanted to go the other way around, and use a Mac server and a Windows client, his entire setup would instantly break. And he'll probably never be able to tie into a Solaris machine, or a million-CPU Linux supercomputer, which idiopath can do trivially.
posted by Malor at 9:59 AM on October 3, 2012 [1 favorite]


cstross: Because, trust me on this, it's full of 'em. Well, if not full, then sufficiently burdened with them to be untrustworthy. (Not to mention smug and non-rigorous.)

Unless you point out what's wrong, then 'smug and non-rigorous' applies to your comment as much as it does to the article.
posted by Malor at 10:00 AM on October 3, 2012


Well yes, we can. CLI users hate GUIs but they aren't out to destroy GUIs. The reverse is not the case, however. There's constant pressure to provide fewer features ("cluttered UI"), remove "ugly" CLIs, "make it intuitive", etc at every level of computing from users up.

I don't disagree that there's a lot of unfortunate pressure to obscure the command line from users, and a tendency to build certain tools as shiny clicky GUI widgets that have no place relying on being shiny and clickable (Ubuntu, I'm looking at you).

That said, I'm pretty much mystified by this supposed dichotomy.

I mean, ok, I live and die by the command line. At present count, on this machine alone, I have 8 instances of xterm, which is about 5 fewer than usual. I have spent entire years with my primary machine running nothing that wouldn't work on a Linux console. I also really love graphical environments. I spend most of my time that's not in a text editor or zsh in Firefox or Chrome. I use a couple of Android devices more or less constantly. The single programming environment I feel the most real nostalgia for is probably HyperCard. I build a web application for a living. I use a fancy window manager to organize all my piles of xterms.

Why would I hate GUIs again?
posted by brennen at 10:07 AM on October 3, 2012 [1 favorite]


Complete control over every non-Windows computer on Earth is a) of low benefit and b) not transferable?

So much of why this claim is absurd is detailed in the Unix hater's handbook. It's really not just a moan about the C-shell, and I recommend it.

My point is not that CLIs are bad. My point is that the world of unix CLIs are bad, or at least deeply sub-optimal, outdated and brain-dead in many places. But people say "unix is everywhere even my phone amazing" as if that were somehow evidence of quality (when they'd reject a similar move applied to Windows).

People will willingly accept that some CLIs are junk -- hello, JCL. Why must the gamut of unix nonsense be defended to the hilt as if it were the best of all possible interfaces? You can have piping without necessarily having a mess of twisty option flags, all dissimilar.

Technology grows and adapts along with innovation. There's certainly a value in stability, but holding your interface close and boasting about how it's so great because it hasn't had any improvements beyond bugfixes in 20 years suggests to me that inertia or some other failure is hindering progress.

And it sure as hell isn't because they had perfect foresight in the 1970s.
posted by fightorflight at 10:38 AM on October 3, 2012


Why must the gamut of unix nonsense be defended to the hilt as if it were the best of all possible interfaces?

It's easy to win an argument against a strawman.
posted by DU at 10:46 AM on October 3, 2012 [1 favorite]


It's easy to win an argument against a strawman.
Yes, tell us more about how GUI users want the death of the CLI
posted by fightorflight at 10:48 AM on October 3, 2012 [2 favorites]


so...in claiming I set up a strawman, you are actually setting up another of your own?
posted by DU at 10:55 AM on October 3, 2012


Malor:

1. UNIX was not invented in the 1960s. MULTICS was, but UNIX is not MULTICS; UNIX development began in the 1970s (on Ken Thompson's spare PDP-8).

2. cat is not short for concatenate; it's short for catenate. Because it's a filter for catenation.

3. less was not a canonical component of the UNIX userland and a replacement for more; it was a separately developed program donated to the FSF's GNU userland in the 1980s. In other words, not part of BSD, SVR3.x/4.x, POSIX, and so on.

4. "grep fubar . -r" - recursively grep for files containing 'fubar' Not on POSIX systems it doesn't. GNU grep has the -r flag, but plenty of other UNIX systems don't supply this. Oh, and the -r flag is in the wrong place. The canonical form for grep is grep [flags] [pattern] [file args ... ].

5. Spoilers for the Lord of the Rings. (Not that I can watch the movies -- jerkycam and retinal problems don't mix well.)

6. Blatant appeal to authority: I'm not Rob Pike or Ken Thompson, but I did work in the documentation of a UNIX OEM for several years, writing man pages and marinating in this stuff. Seeing un-fact-checked errors annoys me.
posted by cstross at 11:07 AM on October 3, 2012 [8 favorites]


Heh. Mr. Stross showing he still has neckbeard cred after all his iDevice dalliances.
posted by Artw at 11:16 AM on October 3, 2012 [2 favorites]


cat is not short for concatenate; it's short for catenate. Because it's a filter for catenation.

I'll be damned. I always heard "concatenate" from people, but this seems to be true.
posted by brennen at 11:20 AM on October 3, 2012


fightorflight: "There's certainly a value in stability, but holding your interface close and boasting about how it's so great because it hasn't had any improvements beyond bugfixes in 20 years suggests to me that inertia or some other failure is hindering progress."

I like new interfaces, I try them often (plan9 had some great ideas that never caught on). What I don't like is the forced obsolescence of interfaces, and I have not seen a single GUI tool of any sort that I could name that is modular enough to stay useful and feature complete as part of a system of GUI tools in the long term.

I love alternatives, I think there should be huge multitudes of them, but GUI tools are so persnickety about how the interoperate and so tightly coupled to one another and their host system that integrating them into your workflow only ever reduces options rather than increasing them. Relying on functionality of any graphic tool is signing up on a waiting list for betrayal.

I make compromises as we all must - I use my editor in windowed mode, I use a modern web browser. But I don't rely on either of their specifics as windowed applications for anything crucial to my job or my projects, because I cannot trust in the long term that they remain available and usable on the computer I use in the future. So I have a workflow that integrates a browser to help look up docs or an editor that I can drag and drop to, but I also know how to work without those things, if and when I am forced to use a different graphic layer.
posted by idiopath at 11:22 AM on October 3, 2012


The software that would be named Unics in 1970 was running in simulations on the Multics machine (Virtualization before virtualization was cool!) before then, so technically, 1969 is the birth year of Unix, the same way 1991 is the birth year of Linux - wouldn't want to do any real work with 'em at that point, as they're well short of an functional system, but they're in the world. This is why Jan 1, 1970 is the start of Unix time - there was a proto-Unix around to start keeping track of it.
posted by Slap*Happy at 11:24 AM on October 3, 2012


I like new interfaces, I try them often (plan9 had some great ideas that never caught on). What I don't like is the forced obsolescence of interfaces, and I have not seen a single GUI tool of any sort that I could name that is modular enough to stay useful and feature complete as part of a system of GUI tools in the long term.

There's a lot of truth to this. Where I think GUIs have succeeded pretty well, especially things like browsers and window management systems, is in creating environments for tool usage. It's good that I can throw a lot of stuff around the screen, and extraordinarily good that the web, with a lingua franca for documents and application display (and an obsession with backwards compatibility) exists, and it doesn't really matter a whole lot that the specific window manager and browser periodically change, because they continue giving me the same capabilities as a user and the interaction I have to relearn is minimal.

What's missing in my immediate computing experience - or rather, what would be missing without the command line - is that these graphical environments don't have a consistent, low-overhead, dirty-but-reliable way of expressing connections between reusable tools, or concisely declaring different behavior for a specific tool. There's copy-and-paste, there's drag and drop, and there's "save it as this kind of file and open it with this other thing". None of that really offers to the user a replacement for pipes, redirection, STDIN/STDOUT, plain old text, and a vocabulary like the standard tools with their various flags and options.

There have been a million attempts to solve this, and every longlived graphical user environment on the planet appears to be littered with their wreckage, but if anything I sometimes suspect we're losing ground.
posted by brennen at 12:32 PM on October 3, 2012


Well, yeah, of course there's a lot of inertia. A successor to a UNIXish shell command-line would have to be not only better than what we have now, but so much better that it'd worth the learning-curve to become as expert in it as I am the current command-line, dealing with whatever hassles it'd create in inter-operating with the *nix boxes that'll continue to be in my life, and all of that while still needing to keep of on the *nix environment because that's going to continue to be part of my livelihood for the foreseeable future.

I also still write programs big and small in Perl. It's not that I don't know that Perl has flaws -- those who use it know best what they are. But there's nothing so much better for the uses to which I'm putting it that it'd be worth it for me to become fluent in something else (and learn what its flaws are and how to compensate for them.) Though it'd probably be easy enough to learn the Ruby command-line switches that'd make Ruby a suitable candidate for one-liners in pipelines, and I might do that at some point.

The worse is better issue plays out over and over at every level. Something else might be recognizably better in some ways. But would switching to it really improve my life overall within the month? Within the year? Within five years? Will anyone care about this new thing in five years? Will it change the way I think about programming or interacting with a computer so much that it'd be worth learning for its own sake even if it never catches on in any big way? The answers to those are usually not so obvious in advance.
posted by Zed at 1:56 PM on October 3, 2012 [1 favorite]


JHarris: "...But what would be nice, now? More visual reminders to help you keep in working memory the details of what you're working on. Spreadsheets now, while you're entering formulas, will highlight the cells they're dealing with. I'd like it if there was an option (an option , mind you) to have those kinds of popup reminders of what commands you're working with. Lists of available tools? A window with a man page? A quick list of command-line parameters? A view into the file system? Syntax coloring? I think that would be useful to be, although I am loathe to suggest everyone would want it."

LOSETHos!!!!!
posted by symbioid at 2:35 PM on October 3, 2012 [2 favorites]


After thinking about it a little, cstross, I believe you're being a little unfair, picking on nits that actually aren't that important. (Thanks for taking the time to be specific, though.)

re:60s versus 70s, from the article, this is the original claim:
Unix was invented in the 60s.
This is what Wikipedia says:
Unix (officially trademarked as UNIX, sometimes also written as Unix) is a multitasking, multi-user computer operating system originally developed in 1969 by a group of AT&T employees at Bell Labs,
And they go on to say that it was substantially rewritten in C, by 1973. So, if he's wrong, which is certainly possible, then Wikipedia is wrong, too. Far from unthinkable, but it's not an obvious mistake.

re: catenate.... that's true, but considering that one of the primary uses of cat is actually to concatenate files, it's easy to see how the folklore could be a little confused, in being passed down. Even if new students learn his slightly-wrong version, it's so close to the original idea that I don't see it as much of a problem. Honestly, it's easier to remember that way -- who uses catenate as part of their working vocabulary, even computer professionals?

re: less... I don't think it's so much wrong, as elision. Less was added later, but an awful lot of (most?) Unixy OSes have it now. He is right about why it's called 'less', but sort of implies that it was part of the original design, rather than coming along later. "man less" shows that the version on my Linux box was first copyrighted in 1984, which is almost thirty years ago. Condensing a decade or two of development into a few sentences doesn't strike me as a major sin, nor an especially important one. When you're dealing with this much history, you have to leave a lot of it out, and it seems to me that the part he left out there is of no real consequence to someone trying to understand why the pager is often called 'less'.

He probably should have mentioned that many distros do still come with 'more', though, and that it's still often the default, even (at least) 28 years after less was created.

re: grep... yep, he got that syntax wrong. Note that it DOES work on most machines that most students would be using, but you're right, he's got his arguments in the wrong order, and GNU grep happens to be smart enough to figure it out and do the right thing anyway. Not all greps are that smart.

But picking out an example command, and skipping over the wonderful section on how grep got its name? Sheesh. I had no idea why it was called grep, and I've been working with Linux professionally since 1998 sometime, and my first toy install was sometime in 1993, almost twenty years ago. Yet, I would probably never have known why grep is called grep, if I hadn't read that article.

So I can forgive an example that would fail on weird, arcane Unices that only professionals typically have access to. (ooh, and maybe the Mac. Does the Mac have GNU grep? I'm not near my laptop to check. )

These things really strike me as nitpicks. The fundamentals are all true, but there are a few surface blemishes that could use improvement. I think, with that background as a doc writer, you're maybe a touch over-sensitive to minor inaccuracy.
posted by Malor at 3:04 PM on October 3, 2012 [1 favorite]


All I want to know who concatenated cate and nate?
posted by symbioid at 3:10 PM on October 3, 2012 [1 favorite]


The birdsandbees, symbioid. The birdsandbees.
posted by Malor at 3:11 PM on October 3, 2012 [2 favorites]


gjc: "The UNIX way was/is the last computer system that expects the user to have the command reference manual handy."

Huh, I wonder what all these .chm files are on my Windows 7 machine? Seems rather analogous to man.

fightorflight: "It's a collection of brittle flags and switches and toggles and inflections that has to be memorised."

No, you memorize a few recipes and the names of a few commands. Beyond that, you have apropos and man (or the help flag) to help you. And help if you're running bash and can't figure out some syntax. Here's the thing: At the command line, I can spend 2 or 3 minutes (what can I say, I'm slow) coming up with a string of piped commands that will do some complex renaming or organizing of thousands of files or whatever. Alternatively, I can spend the better part of a day clicking and retyping in a GUI. I would rather spend my days doing pleasant things rather than tedious things, so I let the computer do the tedious things whenever possible.
posted by wierdo at 4:39 PM on October 3, 2012 [1 favorite]


What killed the Linux Desktop?
posted by Artw at 4:43 PM on October 3, 2012


A sentence from Artw's link which appears a bit bogus:

When OSX was launched it was by no means a very sophisticated Unix system. It had an old kernel, an old userland, poor compatibility with modern Unix, primitive development tools and a very pretty UI.

The kernel is a Mach microkernel, which is more modern and sophisticated than monolithic UNIX-style kernels such as Linux. I'm guessing "an old userland" means that it uses the BSD utilities and not the GNU ones, which is fair enough. "Poor compatibility with modern UNIX" strikes me as a bit odd, given that it's POSIX compliant, unless by "modern UNIX" they meant a graphical Linux distribution (and Apple provided an X server for that). As for "primitive development tools", it had the same GNU toolchain used on Linux and the free BSDs.
posted by acb at 6:55 PM on October 3, 2012


As an example of how useful scripting can be, when I was ripping my CD collection, I wanted to rip CD images, ones good enough to let me recreate the originals if I lost them. This meant I wanted to use CUE files, which are a method of describing the precise layout of a CD, including tracks, gaps, and chapter marks. Originally, CUE only worked with WAV, but I also wanted to compress the WAVs with FLAC, because it would save a ton of space without losing any data. Slimserver, my music player of choice, supported CUE/FLAC just fine, but in 2006, there was no real way to generate these files with a GUI. I could extract CUE/WAV files with EAC, but that's as far as it went.

So, since that's as close as I could get in Windows, I ripped all my CDs to CUE/WAV. And then I got busy with a script on my Linux server. That script would:
  1. Parse through a hierarchy of CUE/WAV files, recreating the directory structure in a new, target location
  2. Copy all the CUEs, editing them in passing so that they referred to FLAC files instead of WAVs
  3. Compress the WAVs to FLACs in the new location
  4. Embed the CUE sheet into the FLAC, as well.
Result: my entire library converted to FLAC, with corrected CUE files, which Slimserver could then add to my music library painlessly.

Nowadays, I can now do much of this with GUI tools; six years later, Foobar2000 in particular has turned into an absolute Swiss army knife of format conversions. But I have to figure it out, each and every time, and there's no easy way to record what I do, so on subsequent attempts, I may not be perfectly consistent with my earlier jobs. And, while it will handle the FLACs fine, I don't think Foobar would fix my CUE files, nor would it have the ability to embed them, which is a feature of FLAC that's not well-known.

My script does all that, and it still works just as well as when I wrote it. Even now, I can rip a CD to CUE/WAV, fire off that script, and it will be converted properly, exactly the same way that all my other files have been. And I fully expect that same script will still work in exactly the same way in 2030 or 2050, where Foobar, and maybe even Windows, will have long since passed out of common usage.

In other words, I invested quite a bit of upfront time writing that script, probably about six hours, much of which was learning how to correctly deal with spaces and wildcards in filenames in Unix; the first version was actually a script that made scripts, and getting everything working just right through two levels of indirection was pretty painful. (I eventually condensed it back into one script again, once I had it working). But, compared to the time involved in ripping all those CDs, it was nothing. In exchange, that script will almost certainly work for the remainder of my life, and perhaps quite a bit longer. I have a tool I can count on, permanently.

And then, of course, I adapted the ideas in the first script to do something else.... I wrote a second script to scan through my music folders, and in any directory where there is a FLAC file, but no PAR2 files, it generates some, covering any CUE and any FLAC files in that directory. So when I rip new CDs, whether a singleton or a batch, I compress them with a script, manually move them to their final homes (which I could probably automate, but never bothered with), and then automatically generate some PAR2 files to protect myself against bitrot. I hated doing all that CD ripping. I don't ever want to have to do it again, and spending an extra 10% in space, to potentially save me a hundred or more hours of tedium, is so worth it.

I could probably have written a Windows program to do all this for me, but it would have been probably two orders of magnitude harder, and I would likely have had to find and integrate source code for FLAC. And I'd already be starting to worry about how my program would work in Win8 and later.

I'm not worried about my scripts. They work, they've worked for six years, and they will keep working.
posted by Malor at 7:15 PM on October 3, 2012 [4 favorites]


What killed the Linux Desktop?

Unity.
posted by JHarris at 7:24 PM on October 3, 2012 [3 favorites]


The kernel is a Mach microkernel, which is more modern and sophisticated than monolithic UNIX-style kernels such as Linux.

That's not at all true. Well, 'microkernel' and 'monolithic' are true, but the other stuff is value judgements that do not match the real world. I've never seen Linux lose a benchmark against Mach, except in the case of regressions, which were quickly fixed. Mach is slow, and inefficient, and it's not even particularly stable as Unixes go, which is supposed to be the big reason to go microkernel in the first place.

Basically, with that comment, you're sort of recreating the famous Tanenbaum-Torvalds argument. Most folks believe Tanenbaum lost that argument decisively.

"Poor compatibility with modern UNIX"

It usually required a fair bit of work to port something to OS X, as the POSIX layer was missing a fair bit of stuff that's in more sophisticated environments. Clearly, almost all of it does port with some effort, and I believe they've added some of the missing things back in, but it was hardly ever just a drop in and recompile.

Mach isn't Unix; OS X has a POSIX personality that runs on Mach, but there's a definite impedance mismatch there, and overall system performance suffers as a result. I suspect this is much of why Apple mostly abandoned the server business; their software just didn't stack up even to Windows, much less Linux.
posted by Malor at 7:29 PM on October 3, 2012 [1 favorite]


What killed the Linux Desktop?
Unity.

If it doesn't succeed, it won't be for lack of trying.
posted by brennen at 9:39 PM on October 3, 2012


I suspect this is much of why Apple mostly abandoned the server business; their software just didn't stack up even to Windows, much less Linux.

What do Apple host their online software on? Part of it seems to be rented AWS instances (which, I presume, are running Linux or Windows), though I wonder whether they built custom OSX/Darwin servers for their own data centres or just threw up their hands and ran something based on Linux, with characteristically tight Apple security to ensure that there isn't a repeat of the "Hotmail doesn't even run on Windows, MS are lame" bad publicity.
posted by acb at 4:35 AM on October 4, 2012


At the command line, I can spend 2 or 3 minutes (what can I say, I'm slow) coming up with a string of piped commands that will do some complex renaming or organizing of thousands of files or whatever. Alternatively, I can spend the better part of a day clicking and retyping in a GUI.

Again, my point is not that CLIs are bad, it's that Unix's CLIs are bad. The alternative I'm suggesting we try and get is not "the GUI", it's a better CLI. Like one that would let you come up with your complex renaming or organizing without 3 minutes each time plus the painful initial learning/memorisation.

And yeah, worse is better. But the funny thing is that despite Unix being a superb example of worse is better, people write these plauditory blog posts using the same sort of tone reserved for Lisp and similar better is betters
posted by fightorflight at 5:48 AM on October 4, 2012


iCloud runs on Azure, or at least did when it started.
posted by Artw at 7:12 AM on October 4, 2012


From What killed the Linux Desktop?:
Why bother setting up the audio?

It will likely break again and will force me to go on a hunting expedition to find out more than I ever wanted to know about the new audio system and the drivers technology we are using.
ALSA has been around since the late 90s, and in the Linux kernel since version 2.6.0 in 2003. I'm not sure what driver issues he's referring to.

Your linux desktop of choice might require or not-so-gently recommend some kind of PulseAudio/Esound/aRts/GStreamer layer on top of ALSA, but you don't need anything beyond a base distribution of Linux to make noise. It Just Works (tm).
posted by helicomatic at 8:04 AM on October 4, 2012


Helico, sound in Linux is a fustercluck. If you've got bog-standard Realtek motherboard hardware, and you just want to play stereo sound from desktop applications, it usually works okay. Usually. But if you change almost any variable, it's likely to blow up badly.
posted by Malor at 8:14 PM on October 4, 2012


Malor: "sound in Linux is a fustercluck"

The primary cause of this is that sound cards are the last component on a modern computer that expects to be able to use a proprietary custom driver that doesn't follow a pre-existing spec. If a sound card follows the firewire, USB1.1 or USB2.0 standards for audio i/o devices they will work perfectly in a plug and play manner. But most high end sound cards do not follow any of these standards, so each individual card requires a commitment of developer time, and the set of developers who know how to write a linux sound card driver and own FooBlasterMegaBass2013 is vanishingly small. Then add the fact that they are probably not being paid for their time, and it is a wonder any drivers at all get written.
posted by idiopath at 8:53 AM on October 5, 2012


Neither of the netbooks I've stuck Ubuntu on have had sound work. Admitedly, I wanted them for using OpenOffice and didn't care about sound, and so didn't try very hard, but as a representative sample that's not great.
posted by Artw at 9:14 AM on October 5, 2012


(if the Surface doesn't prove to be to my liking I'll probably just get the smaller Mac Air next. I know, gasp, shock, etc...)
posted by Artw at 9:15 AM on October 5, 2012


Thanks for the clarification on sound issues.

Ubuntu in particular, in its desktop incarnations at least, is one of the bigger offenders in terms of extra sound stuff on top of ALSA being required rather then optional from the beginning.
posted by helicomatic at 9:29 AM on October 5, 2012


« Older The Fall of the Dinosaurs (in Dominoes)...  |  Live-Giffing the 2012 Debates... Newer »


This thread has been archived and is closed to new comments