TermKit
May 19, 2011 4:12 AM   Subscribe

Steven Wittens uses WebKit to rethink the UNIX terminal [via]
posted by Blazecock Pileon (208 comments total) 33 users marked this as a favorite

 
As a result, you can cat a PNG and have it just work.

If by "just work" you mean "do what the computer thinks you want". If I want to view a PNG, I'll launch gqview or something. If I say cat what I want is to see some binary.

If I wanted a computer that thought it was smarter than me, I would just use OSX to begin with.

That said, I'll check it out because some of the other stuff looks potentially interesting.
posted by DU at 4:26 AM on May 19, 2011 [12 favorites]


Can't imagine it's going to be that useful once I've ssh'd into another box.

Also, like DU, if you want to view, then by far the easiest thing to do on an OSX box is just use 'open'. Reworking cat seems pointless.
posted by edd at 4:33 AM on May 19, 2011


This is just super-cool, but it also strikes me as a little overambitious given that what he seems to be proposing amounts to a more less total rewrite of the unix command line vocabulary.

On the other hand, maybe it's time for something like that.
posted by pts at 4:42 AM on May 19, 2011 [1 favorite]


Oh that's just horrible. Keep your design degree hands off my xterm; it's my calm constant center in computing world that just can't leave well enough alone.
posted by octothorpe at 4:44 AM on May 19, 2011 [26 favorites]


I agree with DU and edd. This misses the point. It's a wonky hybrid between command line and GUI that does neither well.

Most modern UNIX implementations already have GUI configuration control panels if you want to use them. But the command line has the advantage of speed and flexibility for those that learn how to use it. This just seems like the worst of both worlds.
posted by j03 at 4:47 AM on May 19, 2011 [3 favorites]


My bearded UNIX geek side says it's time to bring back witch burnings, and to start with this fellow.
posted by ConstantineXVI at 4:48 AM on May 19, 2011 [30 favorites]


He's not reworking cat, he's rethinking the shell. And while I'm sure that there is a lot to disagree with in this particular implementation, I do think that the direction he is going for is certainly an interesting one.

I see a reworking of the *nix toolchain that is more flexible, allows you to work inside an environment that keeps the power of the way shell things work without being stuck in plain text land.

The biggest roadblock I see is that he' rethinking the way stdio is working to allow for human-workable and machine-workable data flows. I believe this is a great idea (and I'm sure there will still be a way to give you the raw data whenever you need it), but tocatch on it will require at least some rewriting in every tool that's supposed to work like that. Then again, I can't see why "classic" tools could not be handled as they always have been.

I have to say that I don't expect this particular project to end up as the thing we will be using X years from now, but I think it's at the very least a nudge to get us thinking.
posted by _Lasar at 4:48 AM on May 19, 2011 [11 favorites]


Why would I want to view tabular data shoehorned into an ASCII display of some sort? Why not use that processor in my machine and have it draw up an easier to read view? We're in the land of arcance configuration files here. If a concept like this catches on, then you will be able to customize the hell out of it to suit whatever your needs and wants are. And you will be able to have it give you an ASCII table if you want to.
posted by _Lasar at 4:51 AM on May 19, 2011


I'm convinced that desktop quicklaunchers, such-like-as Quicksilver and Kupfer, are the way the command line will be reinvented. They specifically aren't reimplementations of the Unix command line; they can pass commands there, but they're mainly designed for controlling desktop applications. Not just "launching"; apps that know DBUS can receive commands from these things, and Kupfer can actually construct macros on the fly, simply by entering several commands separated by commas.

If you try to reinvent the Unix terminal, you may succeed. But while the result may in fact make the classic Unix terminal friendlier, that will only make a difference to people who want to use the classic Unix terminal. If you really want to make people more, um, intimate with their computers, that's not your target market; you've already got them interested, you can probably get them to read a couple of O'Reilly books. You want to target the desktop user.
posted by LogicalDash at 4:51 AM on May 19, 2011 [14 favorites]


Just deleted my comment on preview, as LogicalDash summed it up - you want to make things that aren't mutant terminals but something new that shares some of their important features.

_Lasar: If I want to view tabular data in a file via the Terminal I can already do that - I use the open command and it opens my spreadsheet file in a spreadsheet, or a plist file in a plist editor or whatever. If I want to use some other specific viewer I use the application I want, not some innate feature of the shell I happen to use.

If I want to see the files in a folder with icons etc I type 'open .', and if I have a view of files in a folder in a Finder window and I want to use a terminal command on it I use the 'cdto' button I installed the other day.
posted by edd at 4:57 AM on May 19, 2011 [2 favorites]


Actually rethinking this, I've changed my mind to a large extent. All I'd ask for is a key combo to turn off the additional parsing it does.
posted by edd at 5:08 AM on May 19, 2011


Let's not forget the Unix world came around when graphics were squirted out on oscilloscopes and plotters.

Reminds me a lot of how Field thinks, that text/code and visuals and richer contexts should work together.
posted by davemee at 5:08 AM on May 19, 2011


The command line isn't arcane out of willfulness or lack of thought. The command line is arcane because powerful tools require knowledge to use correctly. This is as true in a machine shop or a surgical theater as on the command line.

Not every tool is as simple as a hammer. There are inevitably going to be buttons, switches and knobs that n00bs don't understand or misuse. An apprentice in any workshop, computer or otherwise, needs to start small and work in a sandbox or be supervised, gradually expanding their repertoire.

That's not to say that tools can't be improved. Obviously they almost always can. But adding a layer of indirection isn't going to be a popular way of doing that. When I examine a file, say, I want to examine the hell out of that file. I don't want layers of MIME types and invisible Content headers potentially confusing me.

And when someone has an itch, they want to be able to easily scratch it by writing a quick little utility. They don't want to have to learn how to talk to DBUS or include a bunch of GNOME libraries or use a WebKit SDK or whatever. The beauty of the plain text interface (between programs, but also between humans and programs) is that it is just enough technology to solve the problem. Using more tech only invites failures and confusion.
posted by DU at 5:09 AM on May 19, 2011 [23 favorites]


I don't see why it's too much to ask to do both. The Unix toolset currently stinks of inconsistent and archaic commands, and frankly doesn't even handle plain-text horribly well. Yes, if you've read through that stack of O'Reilly books (and I have), it begins to make sense. However, very little of it is consistent or intuitive.

Of course, I have very little faith that any significant progress will be made on this front. Remember GoboLinux? It *completely* discards the traditional Unix filesystem, and replaces it with something that looks vaguely reminiscent of the MacOS filesystem, and included a nifty compatibility layer so that traditional Unix applications still worked.

Naturally, there was a hysterical backlash against it, and the project is now essentially dead. Proposals to port the new filesystem layout to other distributions are also almost immediately shot down, despite there being no real reason to continue using the current egregiously-outdated Unix layout. (Can you explain the practical difference between /usr and /opt, or /bin and /sbin, or why /var exists at all?) We still refer to terminals as TTY, despite the fact that teletype was long dead decades before I was born.

Unix is great, and I'd love to see these problems fixed, precisely because I love it. Unfortunately, I also think that any fixes risk turning into bloated object-oriented kerfuffles. Attempts to make "modern" terminals have almost universally failed. Windows PowerShell should be good, but is completely unusable in practice.
posted by schmod at 5:10 AM on May 19, 2011 [7 favorites]


Given the surplus of computing resources available on today's common desktop computers, why make these changes at the operating system level? Why not make a new scripting language to encapsulate all the old Unix utilities into something more congenial?

Come to think of it, I guess Python pretty much does that.
posted by LogicalDash at 5:17 AM on May 19, 2011 [1 favorite]


Not every tool is as simple as a hammer. There are inevitably going to be buttons, switches and knobs that n00bs don't understand or misuse.

Look, dude. I've read through the stack of O'Reilly books, I'm a ninja with regular expressions, and know the unix command line inside out. There are probably at least a dozen times each day that I mumble to myself "I can't believe that we still do things this way."

Can you really defend the fact that there's no easy way to pipe a text file full of URLs into cURL, or that BSD (Max) and GNU are drifting further and further apart each year?

I understand that Unix is very much like working in a machine shop. The tools are powerful, and there's a learning curve for many of them. However, in a machine shop, the purpose of a tool is generally obvious, does not differ greatly from model to model, and remains constant over time. When necessary, specialized tools are developed that don't cause the original tools to become bloated, or require the machinist to use several dozen tools to perform a simple task.

We haven't seen that sort of evolution in the Unix world. It's like we're trying to etch microprocessors with a heavily-modified Bridgeport Mill.

There's a learning curve, and that's fine -- I honestly see very few ways to make Regular Expressions digestible for "lay-users." However, internal inconsistency, extreme bloat, and a general unwillingness to support any file format that isn't plaintext is killing the practicality of Unix in the modern environment. It's not 1970 anymore, and it's about time that we started acting like it.
posted by schmod at 5:22 AM on May 19, 2011 [7 favorites]


a general unwillingness to support any file format that isn't plaintext

Is that really much of a handicap? XML and JSON are both plaintext formats.

If you're talking about office documents or images or videos or whatever, no, command line utilities for those exist: unoconv, imagemagick, ffmpeg.
posted by LogicalDash at 5:29 AM on May 19, 2011


Can you really defend the fact that there's no easy way to pipe a text file full of URLs into cURL...
for url in `< urls`; do curl $url; done

posted by DU at 5:32 AM on May 19, 2011 [15 favorites]


Oh and your comments about machine shops are also wrong. Even the basic machine, a lathe, has many non-obvious modes, applications and options.

However, in a machine shop, the purpose of a tool is generally obvious, does not differ greatly from model to model, and remains constant over time

And if "usage remains constant over time" is a plus in a machine shop, why not on a command line?
posted by DU at 5:34 AM on May 19, 2011


I understand that Unix is very much like working in a machine shop.

I don't think you do. If you did, then you'd know that the first thing you learn how to do in a machine shop is to make your own tools, which is also the first thing you should learn how to do in the Unix world.

I've got a /bin directory that's full of utilities I've written over the years. Any time I'm presented with a situation where an existing utility doesn't work, then I write a utility that works for the problem at hand. If you're relying on other people to provide tools for your job, you're doing it wrong.

(Most of those utilities are just chains of existing Unix commands piped together somehow).
posted by smcdow at 5:36 AM on May 19, 2011 [5 favorites]


Can you really defend the fact that there's no easy way to pipe a text file full of URLs into cURL...

I was thinking more along the lines of
xargs curl < urls

posted by smcdow at 5:42 AM on May 19, 2011 [9 favorites]


I'm convinced that desktop quicklaunchers, such-like-as Quicksilver and Kupfer,

Is there anything like quicksilver for windows 7 yet?
posted by empath at 5:43 AM on May 19, 2011


Isn't this kind of like powershell on windows7, btw?
posted by empath at 5:44 AM on May 19, 2011 [1 favorite]


Can you really defend the fact that there's no easy way to pipe a text file full of URLs into cURL...

DU's solution works, but I prefer smcdow's solution. It allows you to parallelize the task with the xargs command line options (-N and -P specifically)
posted by papercrane at 5:45 AM on May 19, 2011 [3 favorites]


empath, try launchy
posted by LogicalDash at 5:45 AM on May 19, 2011 [1 favorite]


I thought of xargs but I'm embarassed to admit I don't know how to use it well. :(
posted by DU at 5:46 AM on May 19, 2011


I think he just invented windows 3.1.
posted by gjc at 5:52 AM on May 19, 2011 [25 favorites]


I'm amused that his directory output, thanks to his formatting, displays less than a third of the information in the same space. And note, the output hasn't even made it halfway through A. Way to destroy the ability to display information in a space. There's nothing that graphical output tells me that the text version couldn't -- and I'd see three times as many files in the window.

But that's not the biggest reason to run screaming away from this: he completely and utterly does not understand basic unix commands. "cat" concatenates file. Period. If you don't give it a second file, it concatenates the file with STDOUT.

It is not a file display utility. It can blast test to STDOUT, which means that you have the sometimes useful side effect of a primitive file display utility, but if I type "cat foo.pdf bar.jpg bax.txt", then it had better not display a pdf or jpeg, because the output will not be either of those. It had better dump a binary display of the three files concatenated.

You may ask why would anybody do this? I don't know. But I do know that assuming that side effects are the intended functionality is what make things broken when you try to use the intended functionality.

Basically, the entire posit of his system is based on (BING!) a Useless Use of Cat!
posted by eriko at 5:52 AM on May 19, 2011 [16 favorites]


Next, Bob uses Rock Band 3 to rethink Rachmaninoff's 2nd.
posted by kmz at 5:53 AM on May 19, 2011 [4 favorites]


eriko's got it! I don't care how clever things are underneath (and quickly scanning the article made me think "Just port PowerShell."), if you use Web2.0 graphics and only show 15 lines of text per screen, I'm immediately turned off.

If you're using Unix, your a "power user". You deal with complexity and large volumes of information. 'Simplifying' your display means you only get to see a smaller portion of the bigger picture at a time. Not wanted.
posted by benito.strauss at 5:57 AM on May 19, 2011 [2 favorites]


I think some of the ideas are useful. There are times that I wish I could have just done something like "cat image.png | pngview" and have that just show the image inline. I'd be very happy if that just worked while I was logged in over ssh.
posted by papercrane at 5:59 AM on May 19, 2011 [1 favorite]


There are some interesting suggestions he makes at the end of the article that could be implemented without upsetting the current command line paradigm:

"I want to see the relevant part of a man page in a tooltip when I'm typing argument switches. I'd love for dangerous flags to be highlighted in red. I'd love to see regexp hints of possible patterns inline."

I think the first two would be especially useful. I think the third would be best done not as a list of possible patterns but rather more like the first idea. So as you're typing a regex a tool tip could pop up each time you type a special character or escape character.

Implementing these ideas would require modification to the shell but they wouldn't require a fundamental reworking of the concept of the command line.
posted by jedicus at 6:02 AM on May 19, 2011 [2 favorites]


Doesn't zsh already do some of that?
posted by smcdow at 6:04 AM on May 19, 2011


Although this presents some interesting ideas, I'm more apt to believe text should be text and graphics should be graphics and never the two shall meet. That said, what I'd really like is an updated terminal that allows for 24-bit color rather than 8 color + 8 half-color palette.
posted by pashdown at 6:11 AM on May 19, 2011


There are times that I wish I could have just done something like "cat image.png | pngview" and have that just show the image inline. I'd be very happy if that just worked while I was logged in over ssh.

You don't forward your X session when you ssh?
posted by kmz at 6:14 AM on May 19, 2011


Rethink? Did he already think it?
posted by Eideteker at 6:15 AM on May 19, 2011 [2 favorites]


That said, what I'd really like is an updated terminal that allows for 24-bit color rather than 8 color + 8 half-color palette.

I've had some spotty luck with 8 bit color by setting my TERM to xterm-256color.
posted by DU at 6:15 AM on May 19, 2011


My girlfriend works with a young child with a terminal neurological disorder that frustrates his control of his body. As a result of his condition, he cannot talk, and has to use visual cue sheets to describe his needs to his family and caregivers. Anyone who has experience with this communication interface knows that it can be immensely frustrating. Often the idea to be expressed is just out of reach of the symbols, and interpreting this mismatch requires personal experience and expectations on the part of the other party.

Visual communication is attractive, but limited in flexibility. When it comes to sharing our thoughts with others, coordinating action, and building relationships, language is our first instinct. Using pre-designed symbols for communication is better than nothing, but it does not provide the flexibility and creativity that language does. It requires us to wait for the designer to generate the symbol, and to hope that the symbol which is designed is at least clear enough that we can grow to understand its meaning.

We choose language to interact with each other because it provides us the ability to synthesize new symbols as rapidly as we think. This effect is just as true in the realm of computer-mediated human interaction, and it should not be forgotten.

I think we use Unix because it is more human, not less, than a visual interface. Text streams are archaic, but not more so than any human language.
posted by melatonic at 6:16 AM on May 19, 2011 [66 favorites]


I've mashed the Favorite button on melatonic's comment as hard as I could, but it only incremented by 1.
posted by DU at 6:18 AM on May 19, 2011 [1 favorite]


melatonic: I have met people with similar disabilities and had the exact same analogy occur to me. Very well put.
posted by idiopath at 6:19 AM on May 19, 2011


I appreciate that people are thinking about this, but ultimately I don't think that separating programs' outputs into 'view' and 'data' will play well with the core unix philosophy of having small programs that do simple tasks which become awesome when piped together.

As a concrete example, I frequently du -x | tee /dev/tty | sort -nr | less. du recursively computes space consumed by directories, tee provides a crude progress display (copies the unsorted output of du directly to the display), sort sorts the output so that directories with more contents appear before directories with less content, and finally less displays it in scrollable format.

How do you transport this concept into TermKit? I guess by anticipating every possible view that users would construct with pipes and allowing it to be constructed either by arguments to the disk-usage program or by user interaction after the gui has been opened.

Of course, we have this now in any number of graphical disk usage programs. Heck, I've written graphical disk usage programs myself—though they invariably have the ability to read 'du' output instead of performing the disk scrounging themselves. This has the curious property of letting me display other hierarchical data in the same way--for instance, you can write a program that converts httpd access logs into the proper format, and visualize site usage by URL, for instance.

Git, a source code management system, does something curiously simple that radically improves its usability in the terminal: by default it pipes almost everything to a pager when the output was going to the display instead of to a pipe or file. With a little configuration, it colorizes almost everything when it does this. And this, I think, is a pointer in the right direction.

The right direction, if there is one, is to retain the identity between "display out" and "stdout" that has been indispensable since the unix pipe was invented but give the shell the ability to implicitly invoke appropriate an appropriate visualization application on the ultimate output of a pipeline.

Once someone hits on the right way to do this (and for reasons of remote access, hopefully this can be done without firmly welding the terminal to the shell), the next step is some very good viewers for arbitrary data in formats like xml and json. After that, new programs intended for use in unix pipelines will be free to choose these structured formats instead of textual formats as their primary (or only) I/O format (and we can go back and add this to unix favorites like ls and du, if only we hadn't used up all the option letters already).
posted by jepler at 6:19 AM on May 19, 2011 [4 favorites]


Burn the heathen!

They can have my terminal session when they pry it from my cold dead hands.
posted by hardcode at 6:20 AM on May 19, 2011


You don't forward your X session when you ssh?

Unfortunately many of the machines I work on have X forwarding turned off, or I have to access in ways the preclude using it (ssh to server A, request token to access B, then use token to access B from A, odd stuff like that.)
posted by papercrane at 6:20 AM on May 19, 2011


Also, does this guy know about screen? Shells other than bash?
posted by kmz at 6:24 AM on May 19, 2011 [1 favorite]


jepler: "Git, a source code management system, does something curiously simple that radically improves its usability in the terminal: by default it pipes almost everything to a pager when the output was going to the display instead of to a pipe or file. With a little configuration, it colorizes almost everything when it does this. And this, I think, is a pointer in the right direction."

GODDAMNIT I HATE IT WHEN GIT DOES THAT IT PISSES ME OFF

If I want a pager I will ask for one. My terminal can scroll up, I can run a command a second time, I can read a log file if the output is one time only but very important. This second guessing is exactly what I DON'T want out of my tools.
posted by idiopath at 6:26 AM on May 19, 2011 [5 favorites]


It would make a lot more sense to add image-and-stuff viewing power to less and not cat.
posted by LogicalDash at 6:27 AM on May 19, 2011 [5 favorites]


This discussion reminded me of the zen of python. I think much (all?) of it applies to how I want my shell to behave.
    Beautiful is better than ugly.
    Explicit is better than implicit.
    Simple is better than complex.
    Complex is better than complicated.
    Flat is better than nested.
    Sparse is better than dense.
    Readability counts.
    Special cases aren't special enough to break the rules.
    Although practicality beats purity.
    Errors should never pass silently.
    Unless explicitly silenced.
    In the face of ambiguity, refuse the temptation to guess.
    There should be one-- and preferably only one --obvious way to do it.
    Although that way may not be obvious at first unless you're Dutch.
    Now is better than never.
    Although never is often better than *right* now.
    If the implementation is hard to explain, it's a bad idea.
    If the implementation is easy to explain, it may be a good idea.
    Namespaces are one honking great idea -- let's do more of those!

posted by papercrane at 6:30 AM on May 19, 2011 [2 favorites]


Can you really defend the fact that there's no easy way to pipe a text file full of URLs into cURL...

Yes. Though I think 'curl -O < urls' will in fact work. Note that you need something like -O to handle the fact that you'll be getting multiple files back -- curl by default expect you to tell it what to name the output file. "cat urls | curl -O" might work, but this is (BING) a Useless Use of Cat!

If you give an entire block of data to one instance of curl, then, by previous design decisions, the correct answer is for curl to give you the all the output at once, or fail with error. To do what we probably want (download the files that we can, report errors and continue on what we can't) we need a mechanism to grovel over that file and parse each URL separately.

But then we run into Unix design, which is "don't duplicate functionality." There are tools to grovel over a list, use them.

The design of curl is "give me a URL and I'll go get the information there." It expect a URL.

If you need to present a series of URLs, then...actually, there are a couple of traps. Forex, you might think this would do it:
#!/bin/bash
for i in `cat urls`
 do curl -O $i
done
..but this fails if a URL has a space. You can play games with the IFS variable and dance around this, but it's not really the way. There is a tool to take a collection of lines and offer them as a set of eXtended ARGumentS. It's called cat...if you spell cat "xargs"

xargs is *important* if your going to do this sort of thing often, because if you are, you're going to run into the fact that what a for or while loop may look like this:
for files in `ls files`...
while `ls files`....
what they really are, when evaluated by the shell, is this:
for files in (file1 file2 file3 file4 file5.......fileN)
while files in (file1 file2 file3 file4 file5.......fileN)
and if N is large enough, you run into the maximum length of a line restriction of a shell, and it fails. This is the source of the 'how do I delete all these files, rm * isn't working" posts that litter the net. The answer is xargs, which will recast
for i in `ls $files`
do something $files
done
...to...
something $file1
something $file2
something $file3
....
something $fileN
...all of which fit, even if N is very large. So, the best, and simplest, and as so often true of the Unix shells, the *shortest* way to take a list of URls and feed them to curl is

xargs curl -O <urls

And, if your using a modern version of xargs, you'll find the -P switch will speed things up if you have the bandwidth to run multiple downloads at once.
posted by eriko at 6:30 AM on May 19, 2011 [5 favorites]


The Haskell boys are all crazy over the idea of 'composability'. If you create a software object but you can't compose it with other objects, then it is, essentially, dead. And that's why visual output is dead -- you can't use it as input to something else. It's pretty, sure, but dead. Kind of like hair.
posted by benito.strauss at 6:30 AM on May 19, 2011 [4 favorites]


So like most (all?) people, when I was a toddler and learning my first human language I would attempt to invent words and syntaxes and such because I thought I could put something together that was better than this language I was learning.

I think it is very common to have the same process occur when learning how to really use a computer in the domain of language rather than just pointing and gesturing. If toddlers were articulate in some other language while learning speech, their ideas and complaints would look much like the one in this post.

For example, to quote a commenter on the FPP: "You cite "r-xr-xr-x" (you probably meant "-r-xr-xr-x" or "dr-xr-xr-x" actually) as an example of not telling you useful facts and feel that replacing that string with "you can't touch this" would be useful.".

He is attempting to redesign things he does not understand yet. That is normal, natural, an important part of the process of learning. If he ever actually figures Unix out, he will be embarrassed that he posted this publicly though.
posted by idiopath at 6:34 AM on May 19, 2011 [5 favorites]


The Haskell boys

That's a bit sexist, don't you think?

And that's why visual output is dead -- you can't use it as input to something else

That's not actually an argument against the proposal in the FPP. The underlying textual data is still there, the terminal just renders it differently. So, for example, you could do the following in TermKit:

ls

But you can also do:

ls | grep foo

The textual data is not destroyed by the fact that, at each step in the series of commands, graphical output is possible. Also, there's no reason that TermKit couldn't have a command, switch, or environment variable that turns off all the graphical doo-dads.

And anyway, visual output is not dead, though it may be hard to work with using a text-oriented paradigm (e.g. compare ImageMagick to Photoshop).
posted by jedicus at 6:39 AM on May 19, 2011


Honestly, this kind of reminds of me of Emacs. The implementation here is terrible, but I like the idea of a richer terminal. For example, in my Lisp REPL in Emacs, if I run something and it returns an object, that object is live -- I can move the cursor to its printed representation and pull it up in the inspector, or I can use it as an argument to another function. Why can't I look at the output of ls and pull up metadata on one of the files?

The other huge limitation of the shell is that the tools for working with modern formats are so poor. Look at all the things sed lets you do with columnar data! But none of that is useful for very much in 2011 except for dealing with the output of other basic shell programs. I would love to be able to do something like:

for url in `xpath http://example.com/url-list.xml < /some/path[@link=unexpired]/to/urls`; do curl $url; done

Why can't I work with XML, or with JSON, or with HTML, or with any of the other ubiquitous modern plain-text formats in the shell as easily as I can pull data out of the columnar lsof output? There's no good reason this stuff doesn't exist.
posted by enn at 6:40 AM on May 19, 2011 [2 favorites]


I think anybody may write anything they want to write but I most strongly object to re-using established names. You want to write a utility that does something radically different from what cat does? Super! But don't call it cat, the name cat is taken.

If I shout "Hey Shorty, hand me up the pipe wrench" I don't want Shorty handing me up a flower print umbrella and saying "Wrenches, they been re-thought."
posted by jfuller at 6:41 AM on May 19, 2011 [1 favorite]


There are times that I wish I could have just done something like "cat image.png | pngview"

(BING!) UUoC.

Never type "cat foo | bar". "bar < foo" does the exact same thing without sending the data through another process, and indeed, almost all utilities will accept "bar foo" -- that is, the first argument will be the filename to work on, and if there is no first argument, it'll work on STDIN. This is, after all, why "bar < foo", or even "cat foo | bar" works.

You do not need cat to put a file onto the standard input. Just as > will redirect the output from the screen to a file, < redirects the input to come from a file.

in this case, "pngview image.png" is probably all you need, and if that doesn't work, stick a < between them and it will. Leave the cat alone.
posted by eriko at 6:41 AM on May 19, 2011 [2 favorites]


Isn't this discussion really about applying the MVC pattern to the unix suite of utilities, and whether that's a good idea?

Unix uses a pipes pattern; this goes to that, which goes to another thing, which ends up getting written to a file (which may actually be a tty).

This seems to be adding a view and model layer as a wrapper around that standard suite of tools, and treating the existing set of tools as controllers.

The thing is that you need to design controllers to be controllers, and you need to design models to be models. He's rewriting tools to output JSON just so he can have a reliable way to serialize models between controllers.

This sounds like it could be really interesting, but the work involved seems pretty enormous, and perhaps underestimated.
posted by jenkinsEar at 6:41 AM on May 19, 2011 [2 favorites]


If you consider his data vs. view dichotomy for even moment, then you'll realize he's spewing bullshit. The whole advantage of the Unix philosophy is people learn how one interacts with the data by experiencing the view.

You might propose replacing text by xml, replacing regular expressions by xslt, and building standard xml interpretation into the terminal, but that's got major issues too.

Also, some commands exhibit different asymptotic running times depending upon the information requested, meaning display and processing are not always orthogonal like he requires. You'd need at minimum a framework for lazily instantiated xml documents or some equally horrid object request broker crap.
posted by jeffburdges at 6:41 AM on May 19, 2011


There are times that I wish I could have just done something like "cat image.png | pngview"

(BING!) UUoC.

I'm sorry. I didn't realize we were experiencing a shortage of processes. I'll try to be more careful in the future.
posted by papercrane at 6:45 AM on May 19, 2011 [5 favorites]


For example, to quote a commenter on the FPP: "You cite "r-xr-xr-x" (you probably meant "-r-xr-xr-x" or "dr-xr-xr-x" actually) as an example of not telling you useful facts and feel that replacing that string with "you can't touch this" would be useful.".

So I agree that his suggested solution is a bit silly. But might the terminal not benefit from a 'new user' mode that provided various helpful contextual hints? So ls -l might print a summary line explaining what the various fields mean. Importantly, there would need to be a mechanism to only print this information to the terminal, not pass it on to other programs. Or what if you could mouse over or right-click the "r-xr-xr-x" portion and get a tooltip explanation of what means?

I think there's room to add useful information to the terminal without interfering with the underlying command line paradigm.
posted by jedicus at 6:48 AM on May 19, 2011


Some people, when confronted with a problem, think: "I know, I'll use XML." Now they have <?xml version="1.0"?><root ><quantity>a ton</quantity></root> of problems.
posted by kmz at 6:53 AM on May 19, 2011 [22 favorites]


idiopath, you can have what you want too.
To disable pagination for all commands, set core.pager or GIT_PAGER to cat.
posted by jepler at 6:54 AM on May 19, 2011


This thing seems more like a toy then any kind of revolutionary design. But one thing I think might be a useful add-on to the console system is the ability to format HTML data in the output from programs (as well as including images). So for example you could type a program name and it would give you a report in HTML that you could read right on your console. HTML obviously gives you a lot more options for representing complex data. So you could have tables and graphs and so on.

This thing seems more like a toy then anything really all that useful. Being able to cat png files as images seems kind of pointless. That said, there's no reason this thing couldn't work over SSH
for url in `< urls`; do curl $url; done
I was going to say cat urls.txt | xargs curl, although apparently cat is unnecessary here. If you want to filter the URLs with grep or something you can do cat | grep | xargs, which is nice.
We choose language to interact with each other because it provides us the ability to synthesize new symbols as rapidly as we think. This effect is just as true in the realm of computer-mediated human interaction, and it should not be forgotten.

I think we use Unix because it is more human, not less, than a visual interface. Text streams are archaic, but not more so than any human language.
I know when my mom first got email I taught her how to check it using a command line. She would actually modem in (rather then use telnet or something) and check her mail that way and it was pretty easy to teach her. She just had to memorize a few commands and that was it.

With GUIs she was totally lost. Part of the problem was that she didn't really have any interest in learning which obviously made it more difficult. Now she's a bit more proficient at it. But it's definitely true that for people who didn't grow up using computers the command line is much easier for them to use.

I think with GUIs though, when you have someone who wants to learn by exploring instead of learn by memorizing, such as a kid who's just sat down at a computer the GUI works better. I think it would be helpful if there was a way to show people what commands are available to do what they want. Sort of like auto complete in IDEs maybe. Organizing commands by package name so you could explore them could be one thing.
And that's why visual output is dead -- you can't use it as input to something else. It's pretty, sure, but dead. Kind of like hair.
Why not?
posted by delmoi at 7:02 AM on May 19, 2011 [2 favorites]


The Haskell boys are all crazy over the idea of 'composability'. If you create a software object but you can't compose it with other objects, then it is, essentially, dead. And that's why visual output is dead -- you can't use it as input to something else.

It's not just Haskell (or boys). Functional programming is all about composability. Putting the uselessness of GUIs in terms of functional programming isn't something I'd thought of before, so thanks for that.

Never type "cat foo | bar". "bar < foo" does the exact same thing without sending the data through another process...

Yes, but bar < foo | baz looks super-weird while cat foo | bar | baz has clarity.

The other huge limitation of the shellmodern formats is that the toolsmodern formats for working with modern formatsthe shell tools are so poorly.

Although, to be fair, this is true of HTML. It's an actually useful format that shell tools aren't too great at. XML, OTOH, can piss up a rope.

posted by DU at 7:02 AM on May 19, 2011


I'm more interested in my immediate violently negative reaction to this proposal than the proposal itself, which could have some merits. But I think I and most UNIX users value the density of display information in the terminal. This UI doesn't hit that, yet.

You know what? I've had the pleasure of working with some serious UNIX mavens in the past and currently. The guy over my cube wall can do more in awk than anyone I've ever met. If, on the other hand, I was clobbering my way through the terminal in a vacuum, without these resources, I might come up with something very much like this.

In other words, a lot of UNIX power usage is passed down as folklore.
posted by These Premises Are Alarmed at 7:04 AM on May 19, 2011 [2 favorites]


I think with GUIs though, when you have someone who wants to learn by exploring instead of learn by memorizing, such as a kid who's just sat down at a computer the GUI works better.

Optimizing for learning is a bad idea, though. Learning is a one-time event and then everyone has to pay for that over the lifetime of using that knowledge. Not only that, but you will tend exclude hard-to-learn features.

If instead you take a slightly harder road and learn the command line, you'll be faster and more powerful in the long run, even after you amortize that learning time.
posted by DU at 7:06 AM on May 19, 2011 [3 favorites]


As a - *cough* - windows guy who spends ALOT of time in the ancient MS-style command-prompt of old (still learning PowerShell), I have to agree with many people in the thread.

When I am in a shell, I want text, I want utilities/interactions that work the way they were designed to. I don't want to wait while graphical views are rendered.

If I want a graphical view, I will use a GUI program of some sort.
posted by jkaczor at 7:07 AM on May 19, 2011


I love the command line. In fact, I love it so much that Drush, a command-line Drupal utility, is now an essential part of my work. The fact is that once I know how to do something, I just want it done, I don't want to fiddle with a UI. Drush allows me to do work without ever visiting the Drupal site itself.
posted by melissam at 7:13 AM on May 19, 2011 [1 favorite]


Can you explain the practical difference between /usr and /opt, or /bin and /sbin, or why /var exists at all?

man hier
posted by Celsius1414 at 7:13 AM on May 19, 2011 [5 favorites]


jedicus: "there would need to be a mechanism to only print this information to the terminal, not pass it on to other programs"

It's called stderr.

jepler: "you can have what you want too."

My complaint is that I have to customize it. It's the command line. There are literally hundreds of commands I need to use. Most of them don't need configuration files. I shouldn't need to use a configuration file to get a standard behavior. People who wish more things showed up in their pager rather than spewing to the screen make aliases and shell functions and such, and that is the right solution to the problem. Making me opt out of a pager I didn't ask for is STUPID and BROKEN.
posted by idiopath at 7:13 AM on May 19, 2011 [1 favorite]


Making me opt out of a pager I didn't ask for is STUPID and BROKEN.

Specifically, it breaks the Principle of Least Surprise. Most (all?) other command line tools don't auto-page, with man being an important exception.
posted by DU at 7:21 AM on May 19, 2011


It's called stderr.

Yes, I know about stderr but the kind of information I'm talking about isn't an error or warning, so I'm not sure if that would be considered a kosher use of stderr.

Although, to be fair, this is true of HTML. It's an actually useful format that shell tools aren't too great at. XML, OTOH, can piss up a rope.

Well-formed modern HTML is XML in almost all but name.
posted by jedicus at 7:21 AM on May 19, 2011


Why can't you use graphical output as input? Well, strictly speaking, you can, but it's a whole lot harder. The case I'm thinking of is that old war horse, disk usage.

I use the wonderful WinDirStat for its amazing visualization, but I can't think of any easy way to have the next utility in my imagined pipeline pick out "that large file over there" and do something with it. While a "du | sort" is perfect for piping to "head -1".

We love visual displays because we have an amazing visual system. It's foolish to not take advantage of that in computers. But until computers start tracking our gaze I don't see how that gives us control. Composable, re-usable, inventive language is our greatest human technology for control. Through a thin wire I can tell you "take the ring finger on your left hand and touch the spot two inches over from the left side of your desk". I don't see how I can get that degree of control with a picture.
posted by benito.strauss at 7:21 AM on May 19, 2011 [1 favorite]


delmoi: "when you have someone who wants to learn by exploring instead of learn by memorizing, such as a kid who's just sat down at a computer the GUI works better. I think it would be helpful if there was a way to show people what commands are available to do what they want"

I am mostly self-taught in using Linux. My main teaching tool was as follows:
  1. run the script command
  2. hit tab twice, when bash asks "do you really want to see all 5,000 options" enter y
  3. exit from script
  4. scroll through that insane list of commands I just generated in my text editor
  5. find one that looks interesting and read the man page
  6. try it and ake notes on it
  7. if I broke anything while using it, figure out how to fix what I broke
  8. goto 4
posted by idiopath at 7:28 AM on May 19, 2011 [5 favorites]


Dear god, this guy just doesn't get it, does he? How can anyone who has spent any time administering anything claim that monospace fonts don't have a place in modern computer usage?

He might have said something worthwhile later on - that made me shake my head in disbelief so hard that I got a headache...
posted by sodium lights the horizon at 7:30 AM on May 19, 2011 [5 favorites]


BTW, I'm a fan of "cat image.png | pngview". It prevents me from overwriting files.

It's too easy for me to mis-type

pngview > image.png

instead of

pngview < image.png

through inattention or temporary minor dyslexia.

But the fragment "cat image.png |" creates a read-only data source (or a local const reference, pick your analogy) . I love it when the language makes it harder for me to make mistakes.
posted by benito.strauss at 7:30 AM on May 19, 2011 [2 favorites]


sudo what?
posted by blue_beetle at 7:34 AM on May 19, 2011


> The command line is arcane because powerful tools require knowledge to use correctly. This is as true in a machine shop or a surgical theater as on the command line.

This is also why you are not allowed to get your student driver's permit until you demonstrate to the authorities that you can field-strip and rebuild an Oldsmobile V8.

Computers operating as servers or other high-reliability situations are definitely best administered by expert users skilled enough to do things like run live kernel patches without downtime. But I can't believe you're seriously proposing that every desktop and phone may only host a terminal session for those worthy of the elite ranks.
posted by ardgedee at 7:34 AM on May 19, 2011


Well-formed modern HTML is XML in almost all but name.

Yes, but with a purpose. XML for the sake of XML is what can piss up a rope. The only feature I can see that it helps is simple object-serialization, usually from Java. Among its many other faults, supporting mindless laziness, OO and Java are not features I want to encourage.
posted by DU at 7:35 AM on May 19, 2011


But I can't believe you're seriously proposing that every desktop and phone may only host a terminal session for those worthy of the elite ranks.

There's a good reason you can't believe that. It has to do with the truth value of your statement.
posted by DU at 7:36 AM on May 19, 2011 [2 favorites]


idiopath: "My main teaching tool was as follows: "

also, I forgot to mention, much less interesting but at least as useful:

The apropos command. If I ever had a bit of curiosity, I trained myself to immediately use apropos and check out some of the options.

This is something to bugs me big time about Ubuntu: yeah they have more conventional modern usability, but they also don't have man pages for many of their programs (whereas the original Debian explicitly has a man page for everything, even if the man page is just the name of the program followed by "someone screwed up and didn't make a man page"). The failure of commands to have man pages makes the apropos command much less useful. The man page becomes a sort of index card in a card catalog you can browse by keyword with apropos - so every command and program, even those that are so simple that they don't need a manual, should at the very least have a page with the name and a short summary with some good keywords in it.
posted by idiopath at 7:36 AM on May 19, 2011 [1 favorite]


I'm somewhat surprised by the virulence in these comments. I'd be SUPER pissed if someone took my X terminal away and replaced it with something like this, but is that what is being proposed? I think not.

I don't want one tool to rule them all, I want 10,000 tools to rule them all, and this is potentially one of them. What I like about this is that it's effectively a hybrid command-line/GUI tool. That's something I'd have use for, as I jump back and forth between terminal and GUI so much I'd probably identify it as the largest productivity inefficiency I have in my desktop work environment.

There are certainly things I strongly disagree with in his philosophy, but I also think it's a damn cool experiment.
posted by mcstayinskool at 7:39 AM on May 19, 2011 [3 favorites]


The discussion about this article reminds me of something Alan Kay once said: "The great problem with Lisp is that it is just good enough to keep us from developing something really good."
posted by jasonhong at 7:41 AM on May 19, 2011 [3 favorites]


Like empath said up above, if you want to look at a modern shell paradigm you really need to start by looking at Windows PowerShell. It's got the same idea as TypeKit of typed data as the communication between processes instead of text. Microsoft's been working on it for awhile now, at least some 4 years.

Personally I find PowerShell confusing and unusable, but then I'm the type of nerd who has a fondness for Cygwin, as buggy and bloated as it is. If my brain weren't crippled by 22 years of Unix command line hacking I might find PowerShell more appealing.
posted by Nelson at 7:41 AM on May 19, 2011 [1 favorite]


Through a thin wire I can tell you "take the ring finger on your left hand and touch the spot two inches over from the left side of your desk". I don't see how I can get that degree of control with a picture.

Snarky ImageMagick example:

convert label:"take the ring finger on your left hand and touch the spot two inches over from the left side of your desk" command.png

But this flows two ways. Yes, it can be difficult to make a picture that's represents text in a non-trivial way, but it can be difficult to go the other direction, too. For example, a typical exploded-view diagram of a complex machine can be described in text, but it's extremely ungainly to do so (patents are full of this kind of thing).
posted by jedicus at 7:44 AM on May 19, 2011 [2 favorites]


I'd like somebody to come up with a better GUI metaphor instead, since that seems to be what everybody actually uses.

For the most part, I think that the command line is easy to learn and understand, once you get over the gut fear.

Any of the GUIs however, are hopelessly arcane. I think you'd get a lot more benefit from reworking those.
posted by Stagger Lee at 7:46 AM on May 19, 2011


"That's not actually an argument against the proposal in the FPP. The underlying textual data is still there, the terminal just renders it differently. So, for example, you could do the following in TermKit:

ls"

Now there's pretty much the root of what's left of my complaints. If a terminal application chooses to render the data passed to it in an interesting way then that's fine. It's pretty much what happens if 'ls --color' passes escape codes and my terminal turns them into colours. Just straight ls however isn't passing any information about the icons associated with files or the file types, so how is the terminal application here finding the icons?
If he had a terminal program that chose to interpret data in certain wrappers in a certain way that'd be fine. If he also had a variant of ls that passed the extra information for his terminal application to display, that'd be fine too. But changing ls to have some behind-the-scenes communication with the terminal app that would change its behaviour in ways I might not fully understand is going to be a problem.
Likewise, cat should be just spitting the raw data at the terminal. If the terminal chooses to render it differently in different circumstances that's arguably acceptable, but if you want to change what cat actually spits, then please give it a new name and leave the old cat alone.
posted by edd at 7:50 AM on May 19, 2011


Using cat instead of file redirection is sometimes useful when you're debugging a pipe chain along the lines of:

foo | bar | baz > report.txt

If I'm having problems at any point in the chain, I can just slip in a 'cat testfile' into the stream without changing the whole order of things.

But on the topic of TermKit, we've had dozens of file browsers with Norton Commander and emacs dired at one end, and Konqueror, Finder, and Windows Explorer on the other hand. These handle most of the basic data-management tasks. For building pipes and scripts, Automator is a good model for building linear marcos. Although I'm not convinced that there's a huge gap between graphical drag-and-drop and command-line scripts that needs to be filled.
posted by KirkJobSluder at 7:53 AM on May 19, 2011


"You can't touch this" is more useful knowledge than "r-xr-xr-x", to someone who just started using Unix a week ago. It would be great to have mouseover tooltips, hyperlinks, etc to provide such new users with an easier way to learn than RTFM... But it's insane to *replace* that terse, informative syntax. r-xr-xr-x has 9 obvious bits of information (and 3 more obscure bits) compared to 1 bit in "you can't touch this"... and you want the computer to assume that I don't care about the other 8? The computer would usually be wrong.

This thinking is similar to my suspected reason why, for years, many Linux distributions combined polished beautiful installation systems with half-assed upgrade and configuration tools: they were optimizing for good reviews from journalists who sit down with a brand new system for a few days, not from users and administrators who sit down with that system for months or years afterward.

While we're at it, why not rearrange everyone's keyboard to use alphabetical order instead of qwerty? That will be *less* disruptive to long-term users than most of these proposed changes, and it will be a fantastic improvement for new typists...
posted by roystgnr at 7:53 AM on May 19, 2011 [1 favorite]


As a result, you can cat a PNG and have it just work.

I look forward to seeing such a cat on my wyse50 amber terminal.

(Just what i want. Tools that are gonna run different content based on what they think the data stream is. This is how Microsoft (and others) gets into trouble with programs embedded in graphics. This "plan" is a security nightmare - but hey, won't it be cool?)
posted by rough ashlar at 7:55 AM on May 19, 2011


This kind of reckless desire to improve familiar things is how the world got Esperanto and Plan 9.
posted by Stagger Lee at 7:55 AM on May 19, 2011 [2 favorites]


Oh, good grief. It's the DOS vs. Mac arguments all over again in this thread, only the DOS kids have all learned a smattering of Unix shell commands, and think they're something.

For simple tasks that are doing, spatial memory is much better than verbal memory. It's why GUIs won. It's why there are GUI's everywhere, overlaying everything, including ivory-tower applications like advanced firewall configuration and log analysis. It collates complex concepts into tasks suited for our monkey-put-stick-in-hole instincts. Everyone loves it when done well - I sure as shit don't want to go back to munging pf or ipchains command files by hand.

Furthermore, GUIs allow you to view more information at once, in innumerably useful ways. The more information you're dealing with, the more unwieldy it is to control with strict command syntax.

All that said - this TermKit thing is going in the wrong way. The uses command lines are still put to - remote administration and automation tasks - don't require a lot of flashy display options. I think a purely programmatic interface, not reliant on "commands" as independent executable programs, is probably a better way to go. Most of the attempts in this direction have pretty much sucked.

Another interesting direction is Quicksilver and its like - command lines that work in tandem with GUIs for complex or repetitive tasks not suited for spatial metaphors.
posted by Slap*Happy at 7:58 AM on May 19, 2011 [2 favorites]


I am mostly self-taught in using Linux. My main teaching tool was as follows:
run the script command
hit tab twice, when bash asks "do you really want to see all 5,000 options" enter y
exit from script
scroll through that insane list of commands I just generated in my text editor
find one that looks interesting and read the man page
try it and ake notes on it
if I broke anything while using it, figure out how to fix what I broke
goto 4


Your problem is bolded.
posted by rough ashlar at 7:58 AM on May 19, 2011


Oh, good grief. It's the DOS vs. Mac arguments all over again in this thread, only the DOS kids have all learned a smattering of Unix shell commands, and think they're something.
If reading flame wars grieves you, it helps to avoid posting anything inflammatory.
posted by roystgnr at 8:04 AM on May 19, 2011 [4 favorites]


Yes, but bar < foo | baz looks super-weird while cat foo | bar | baz has clarity.

What's wrong with "bar data | foo" rather than "cat data | bar | baz"? "showjpg image.jpg" is clear enough, right, as is "posterize image.jpg | showjpg".

This is what I don't get. "bar < foo" makes as much sense to me as "bar foo". Because, in 99.95% of the cases today, *it is the same*. The reason to remember it is there's times where explicitly telling something that it's going to STDIN keeps things sane.

I'm sorry. I didn't realize we were experiencing a shortage of processes.

It costs you real time and resources. When you're groveling over 10 items, no big deal, right? The point is to build good habits, so that when you're doing something to a million items, it goes much faster.

Everyone thinks resources are free. They're cheap -- right up to the point where you need more of them and don't have them in the rack, in which case, they are effectively infinite cost. You cannot get more resources online in an instant. (No, not even with virtual technologies -- if you don't have CPU/Disk/IO in the rack, you can't instantly bring it online.)


It's too easy for me to mis-type

pngview > image.png


Right, so don't do that. There's no reason, here, for either cat or redirecting STDIN. Pass the file as an argument to the command.

pngview image.png

I do dearly love how people cite "cat data | program" as more readable than "program data". Really? Seriously? You're saying that the only way to put data into a program is a pipe?

The only defensible argument is benito.strauss's, where a typo could cost you data. The Unix shell is loaded with those. Hell, even cat is, "cat file1 file2 > file1" will nuke file1. This is the source of the button that says "Unix doesn't keep you from doing stupid things, because that would keep you from doing smart things." But he is correct that it is a typo trap, right up there with "rm -f $DIR/*"

But the answer isn't to use cat, it's to use the program's argument line. There are some truly ancient and crusty programs that won't take a file as an argument, and , to me, the "program < file" is a much clearer syntax, because it matches the normal "program file" convention.

And, of course, don't run as the superuser, so typos don't do as much damage. :-)


In case you're wondering, what happens if the $DIR variable isn't defined? Or it is, but you typed $DID?
posted by eriko at 8:05 AM on May 19, 2011


what happens if the $DIR variable isn't defined? Or it is, but you typed $DID?

The incomming VoIP line gets billed a whole lot and by accident it dialed 911 and now the SWAT team is on your doorstep because the call was a no-response?
posted by rough ashlar at 8:07 AM on May 19, 2011


So far I haven't seen a visually oriented programming language that is good and is general enough so I can't see why we could expect to see a visually oriented shell to be any good. For me a shell is just another programming language. If you think that you're going to run into a case where a visual representation is a massive improvement often enough, then write a program that builds the representation from text and plug it into the end of your shell command line. If you want, tooltips, colorizing, command option completion, hyperlinks to manual pages, then just hack on emacs shell mode a bit and you can get whatever you want provided you actually know what you want.

Or you could build a machine with an object oriented programming language focused on the MVC paradigm throughout the system but that would be repeating history, Smalltalk.
posted by rdr at 8:09 AM on May 19, 2011 [1 favorite]


> There's a good reason you can't believe that. It has to do with the truth value of your statement.

line 3705690: syntax error
posted by ardgedee at 8:10 AM on May 19, 2011


"That's a nice UNIX terminal you've got there, but it could really use some proprietary bloatware and difficult-to-troubleshoot and wholly unnecessary binaries/binary streams."
posted by fifthrider at 8:17 AM on May 19, 2011 [3 favorites]


I believe I ran into one of those crusty old programs that didn't handle a file as a command line argument, and hit on "cat file.ext |" as the universal solution. And the thing is, unless I'm just running a single command, I think of Unix as system for passing data through pipes. So "Put a file's contents into a pipe" is a useful primitive for me.

But I'm old. There are things I figured out how to do 20 years ago, and just keep doing them because they get the job done. I'm pretty sure one of the flags in my standard

ps -efa

is redundant, but it's not worth my time to figure it out.
posted by benito.strauss at 8:18 AM on May 19, 2011


It's always a good idea to think about life on the commandline, but I don't see what problems are solved by this approach and I hate MIME.

It is a shame that his effort is built around the UNIX heritage. The UNIX shell and the toolchain constitute a language. You cannot just rework it because it is as much culture as it is technology. /dev/null and rm -rf are idioms that can be found on t-shirts - I don't think it's ever going to change.

The MIME thing is a problem not just because MIME is ugly, grasping and largely useless, but also because it introduces structure where you don't want any. I don't want apps to refuse to process information just because the header says it's in a format they don't understand. I want my sawing machines to saw, regardless of what I feed them, even if I lose my fingers in the process.

UNIX has severe limits as a system and as a culture it is almost Skinnerian in its outlook. It rejects superfluous structure and stubbornly insists that "the view is the model". But these limits have been instrumental to its success, encouraging the production of simple applications that do simple things well and not too much bullshit.

It is certainly worthwhile to explore other avenues. Microsoft's PowerShell seems to be the most practical alternative right now. But working with PowerShell you notice immediately why UNIX is so great: PowerShell enables apps to do things and interact on levels that UNIX doesn't even acknowledge - but the effort that app builders need to put in to make these interactions actually happen and be meaningful is hard and so there is not a great deal of interest in it. AppleScript had a similar problem years ago. This TermKit seems to require even greater effort on behalf of app builders, with less reward.
posted by eeeeeez at 8:18 AM on May 19, 2011 [2 favorites]


eriko, I was being facetious. The cat command is overused in the shell. It is unnecessary filler, and I should stop using it, just as my little sister should stop saying 'like' so much.
posted by papercrane at 8:24 AM on May 19, 2011


I looked at this, and I first thought "Webmin already does it, and does it better".
posted by Old'n'Busted at 8:24 AM on May 19, 2011


I love the idea behind PowerScript (Pass objects through the pipeline!), but I don't know that the inventors gave enough attention to the whole culture behind UNIX that eeeeeeez noted.

I haven't investigated it, but if I don't have a home directory, and a .aliases file that gets sourced every time I open a terminal (and can source other people's .aliases), I'm not going to adopt it.
posted by benito.strauss at 8:24 AM on May 19, 2011


For simple tasks that are doing, spatial memory is much better than verbal memory. It's why GUIs won.

GUIs have not won. The reason they are everywhere is that they optimize for the new user. A lot of interfaces have "new users" all the time and it makes sense to make them GUI. New sales. Kiosks (like self-checkout, etc). Many people are perpetually "new" users because they don't use computers often enough to really learn them. Etc.

But as computers become even more ubiquitous and users become more familiar with them, the metaphor layer will start to erode, I think. At a certain point, the tasks you are trying to do are not well-represented by pictures on a screen. A column of numbers, yes. A table of numbers, yes. A 3D array of numbers? Not really. Higher dimensions of multiple kinds of data? Almost certainly not.

A picture is worth a thousand words. They say that like it's a good thing, but it really means that any nuance captured in less than a thousand words is lost in a GUI. GUIs are limited by the person that made them, the command line is limited by your own imagination.
posted by DU at 8:25 AM on May 19, 2011


The Haskell boys
That's a bit sexist, don't you think?

posted by jedicus



Goddamn it, whatta ya know, you're right. Noted and taken on-board.
posted by benito.strauss at 8:27 AM on May 19, 2011 [2 favorites]


And, from the article:
This seems like an okay choice, until you start to factor in the biggest lesson learned on the web: there is no such thing as plain text.
I don't know that this is the biggest lesson learned on the web at all. If anything, the web taught me that "chaos works and nothing matters, least of all character encodings".

I do love his blog banner image though.
posted by eeeeeez at 8:29 AM on May 19, 2011 [2 favorites]


GUIs have not won.

Most *nix users still seem to drop to the command line whenever they have to actually do something. And rightly so.
posted by Stagger Lee at 8:31 AM on May 19, 2011 [1 favorite]


The UNIX shell and the toolchain constitute a language.

This. I view the irregularity of thinks like -r vs -R and the options in find and dd as analogous to irregular verbs in human language; they've survived because they're conveying meaning. DD is seriously stuff, don't blurt it out without knowing what you're saying. This project looks like a useful tool for learning UNIX, but not for fluency.
posted by bendybendy at 8:31 AM on May 19, 2011


I can't help but think he was trolling, and given the response here (bites), I'm even more convinced he is trolling.
posted by k5.user at 8:31 AM on May 19, 2011


But as computers become even more ubiquitous and users become more familiar with them, the metaphor layer will start to erode

This isn't 1981. The gui model is 25 years old now and continues to evolve. Indeed, I'd argue that the UX world has been moving away from this vision, not towards it. Look at the Web, Android or iOs UIs. All very visual. The current "command line" is a search box: Firefox's awesomebar, Chrome's integrated search & url address.

The problem with a text-only command line is that it offers very few affordances (tooltips, integrated instant help, etc...). Low level control is important, but even machine tools have safety sheilds, labeled buttons, and clear markings on the tools.
posted by bonehead at 8:33 AM on May 19, 2011


GUIs have not won. The reason they are everywhere is that they optimize for the new user.

Most modern security appliances use GUI interfaces to configure and troubleshoot, and network traffic monitors are going the same way. "The new user" isn't going to be anywhere near that kind of equipment. It really is easier for professionals who know what they're doing to drag, drop, cut and paste and navigate with a mouse or gesture.
posted by Slap*Happy at 8:35 AM on May 19, 2011 [1 favorite]


MetaFilter: A Useless Use of Cat
posted by WalkingAround at 8:36 AM on May 19, 2011 [2 favorites]


Most *nix users still seem to drop to the command line whenever they have to actually do something. And rightly so.

Most OS X, Ubuntu, etc. users? I doubt that.

Wouldn't something like a CLI library for OS X's Automator be a better approach?
posted by ChurchHatesTucker at 8:37 AM on May 19, 2011


The problem with a text-only command line is that it offers very few affordances (tooltips, integrated instant help, etc...). Low level control is important, but even machine tools have safety sheilds, labeled buttons, and clear markings on the tools.

Well, there are man files, and a missing suffix or bad syntax gives you a helpful error message with suggestions. There can always be improvement in the text, but it's certainly possible and available in either format.
posted by Stagger Lee at 8:39 AM on May 19, 2011


So much neckbeard up in here we're going to need a gasoline powered weedwhacker.
posted by Horselover Phattie at 8:41 AM on May 19, 2011 [4 favorites]


So much neckbeard up in here we're going to need a gasoline powered weedwhacker.

Naw, the Blue is run on Windows so that cancels it out.
posted by rough ashlar at 8:46 AM on May 19, 2011


We should probably talk about something more important, like whether nethack is better in ascii or tiles.
posted by Stagger Lee at 8:47 AM on May 19, 2011 [1 favorite]


I don't want the computer equivalent of a screwdriver and a hammer, I want a tricorder and a laser saw.

Typical prissy Star Fleet yIH ngaghwI'. It's a wonder the Vulcans tolerate them.
posted by Celsius1414 at 8:51 AM on May 19, 2011


ideopath, all that git stuff is configurable.
git config --global core.pager cat

posted by zengargoyle at 8:52 AM on May 19, 2011


Look at the Web, Android or iOs UIs. All very visual.

And none of them are power user apps.

Most modern security appliances use GUI interfaces to configure and troubleshoot, and network traffic monitors are going the same way. "The new user" isn't going to be anywhere near that kind of equipment.

These applications are ordered by (in all senses of the term) the ultimate perpetual n00b: managers. GUIs look great on sales pamphlets and training videos.

It really is easier for professionals who know what they're doing to drag, drop, cut and paste and navigate with a mouse or gesture.

It really isn't. If they know what they are doing, they are going to write a script to automate the 90% drudge work.
posted by DU at 8:56 AM on May 19, 2011 [4 favorites]


Most modern security appliances use GUI interfaces to configure and troubleshoot, and network traffic monitors are going the same way. "The new user" isn't going to be anywhere near that kind of equipment.

Actually, no, very few people spend their entire day head-down in these sorts of things, so they're perpetually used by "new" users. And the people who do use them frequently, tend to prefer a command line.
posted by me & my monkey at 9:01 AM on May 19, 2011 [2 favorites]


Or maybe I should say: If they aren't writing scripts to automate the 90% drudge work, they don't know what they are doing and/or have been prevented from doing that buy someone buying a GUI based app.

(Like the backup app someone saddled our department with. I had a script that locate the files that mysteriously disappeared, but I had to manually click on each one to restore it. Stupid.)
posted by DU at 9:02 AM on May 19, 2011


> Actually, no, very few people spend their entire day head-down in these sorts of things, so they're perpetually used by "new" users. And the people who do use them frequently, tend to prefer a command line.

Nah, who wants to learn a bunch of new commands that only work on Checkpoint units? The GUI is fine. Just because and admin prefers a GUI to configure a firewall doesn't mean they're some kind of lightweight. Many smaller IT shops require a person to be conversant in many different systems so the GUI lets them perform tasks without having to memorize particular commands to that brand. They still need to understand the basics of networking, jut not the subjective internals of one brand.
posted by Horselover Phattie at 9:05 AM on May 19, 2011


Well, there are man files, and a missing suffix or bad syntax gives you a helpful error message with suggestions.

I will admit that sometimes a little knowledge can be a dangerous thing on the commandline. Back in the day I had root access to a math department webserver (which I really shouldn't have had, but I was the admin for the math camp homepage and was buddies with the profs that ran the place), which was running some version of Solaris. I was already used to Linux on my home box, and didn't consider the differences between SysV and BSD style syntax. I wanted to kill all instances of httpd for something, so I did killall httpd but that gave me an error. So I did what I usually did when that happened, and typed killall by itself hoping to get some guidance.

But in SysV, killall is a bare command that kills every process it can. Oops. Thankfully the box rebooted automatically so after a few minutes I could log back in and fix everything, but that was a pretty painful lesson on not making assumptions about what a command does. Especially when you're root.

(Also a bad idea? rm -rf .*)

(Interestingly, that math lab had a Solaris webserver, AIX file server, and an A/UX yp server. Along with about 20 other A/UX boxes. Apparently at that time it was the largest A/UX installation in the world.)
posted by kmz at 9:06 AM on May 19, 2011 [1 favorite]


These applications are ordered by (in all senses of the term) the ultimate perpetual n00b: managers. GUIs look great on sales pamphlets and training videos.
Quoted for truth. God kills a kitten every time a new appliance comes out with an awful web/java GUI. The next N+1 months are spend with the network/security gurus complaining to the vendor about their piece of crapware, complaining about lack of programmable APIs, making voodoo dolls and cursing every fucking time they have to do anything the vendor didn't anticipate and make a fucking button for. motherfuckers!
posted by zengargoyle at 9:06 AM on May 19, 2011 [2 favorites]


I love unix terminal but it has some very archaic limitations that need to be fixed. For example, if a listing or a cat turns out to be too long, there should be a shortcut to got to a clearly separated top of the listing, and page up/ page down should scroll page up / page down. It would be good to be able to go to a directory by clicking on it and so on. It would be good to have some graphical elements that format the view without losing space, divider bars, cat output indented in a border, icons next to file listing as long as they don't take up more space than a regular listing.

By the way, zsh can be set up with a lot of keyboard shortcuts that make shell use much handier (at least for me): ctrl-b to go one level up in directories, ctrl-l to ls, auto-ls after 'cd', ctrl-e for incremental search of commands (type and it shows you matching command, enter 2 times to run it), ctrl-o to insert previous word, etc. (actually now I realize I probably set up all of these shortcut keys myself, they're not set to anything by default in zsh).

It would be nice if shells had some built-in, nicer-looking midnight commander view because some times it's easier to select files visually, other times to use wildcards.

Shell should always remain 98% text-oriented. That doesn't mean that in the remaining 2%, a hell of a lot of useful stuff can't be added.
posted by rainy at 9:14 AM on May 19, 2011


zengargoyle: "ideopath, all that git stuff is configurable.
git config --global core.pager cat
"

Yeah I have done that.

As I said before, command line apps should not need a config file for normal behavior. Normal behavior is not spawning a pager I did not ask for.

What really rubbed this in for me was the fact that the terminal I was running git from was already embedded in a window in my text editor. If git had checked terminfo it would have noticed it was a dumb terminal and not attempted to run any commands that use curses terminal control commands, since they just show up as junk on my perfectly scrollable, editable, and usable dumb terminal screen (pretty much a pager already, but better).

Like I mentioned before, like any other command line user I use hundreds of command line tools. I don't like the trend of needing to make and configure a fucking .rc file for each one in order to get reasonable behavior.
posted by idiopath at 9:16 AM on May 19, 2011


@rainy I think you've answered your own list of shell limitations. You can install any number of "midnight commander" file viewers. You can easily add your own custom keyboard commands and aliases to your shell.
posted by Celsius1414 at 9:17 AM on May 19, 2011


with all due love to eriko and the others, I think the claimed extraneous use of cat is often misunderstood.

For a lot of people, myself included, 'cat' is a little preface to what might turn out to be a queue of commands, a little BIOS bootup saying "hey, I am now embarking on what could easily turn out to be a pipeline; please understand, dear brain and any future readers*, that chained execution of piped executables is intended to ensue. And further, the starting data element is the thing I immediately now name."

Hence:

cat spam.txt | x | y | z | a | b | c | d > spamout.txt

in brain-speak for many people, including myself, looks like:

we-are-pipelining-now SPAM.TXT *pipeline of cleanly delimited commands, all alike* > SPAMOUT.TXT

Whereas:

x < spam.txt | y | z | a | b | c | d > spamout.txt

in brain-speak involves 'breaking the chain' of cleanly delimited commands to change the order; now x is not located next to y and z, but rather occupies the first position, as if it were special and exciting (which sometimes it is, sometimes it isn't), and looks like:

X < spam.txt | y | z | a | b | c | d > SPAMOUT.TXT

Why is X more special? What are we doing -- oh yes, we're pipelining -- the symbol interpretation of < means that spam.txt is in the input even though it's in the third position, got it...

In summary when you believe that a mentally and visibly clear description of what you are doing, followed by a regular chain of pipelined commands, is worth the extra process overhead of a single extra execution of a super well polished 30 year old Unix utility, versus a slightly less visually and mentally obvious phrasing and an extra few milliseconds and kilobytes of memory saved, then the 'cat thing | x | y ..' syntax might well be a good idea for you.

Disclaimer: only been using Unix for 31 years.
posted by felix at 9:17 AM on May 19, 2011 [13 favorites]


Nothing needs to be changed about the unix shell itself. However - when I am King, the first against the wall will be the scoundrels who refuse to adhere to basic etiquette for command like switches, particularly common ones like -h, -v and -o. Your app is not a unique snowflake, you inconsistent bastards!
posted by vanar sena at 9:23 AM on May 19, 2011 [5 favorites]


command line. Up against the wall I go.
posted by vanar sena at 9:23 AM on May 19, 2011


I think he's attacking the wrong problem here. The problem -- or at least, an area I think worth revisiting -- is the idea that a terminal is basically a sheet of paper running through a printer. Even PowerShell follows this metaphor.

However, with worksheets like those in BBEdit or the much-missed MPW, you can treat all of the output as input. Type ls, hit enter, get the directory listing added to your sheet. Spot the file you want, put yr command name before it, move to end of line, hit enter, command runs. And that's just the start of it. MPW is great.

The other innovation it had that I miss was Commando -- hit opt-enter on a command and it would bring up a GUI with all the options. You could choose them from a form w/radio buttons and drop-downs etc, and as you did it would show you the command line for faster use next time.

That's how the command-line should be. Not so damn hoity-toity about being for pure power users and GUIs only for the new, but saying "good for the new? Hey, let's use that to help new people become power users, then".
posted by bonaldi at 9:23 AM on May 19, 2011 [5 favorites]


The guys on the Bespin / Skywriter project have been thinking along the same lines for a while. Good stuff.
posted by weston at 9:28 AM on May 19, 2011 [2 favorites]


felix, I use cat the same way. The initial may start as a 'cat example.clip' and later be replaced by a 'dump giant gzipped log from remote host through gzcat'. Plus when you go to add things at the end it's easier to up arrow and be at the last argument of the pipeline where you can tweak it or add another pipe vs having to navigate around a file redirection. 'cat foo | egrep -v "foo|bar"' is easier to add to than 'egrep -v "foo|bar" < foo'.
posted by zengargoyle at 9:29 AM on May 19, 2011 [1 favorite]


Celsius: not at all, my point was that a lot of stuff is being added to shell environment that makes it better, but graphical elements were not yet added because it'd be hard to do well. Shell is neither frozen in the 70s nor should be.

Incidentally, people knock guis as if they are inherently flawed. Guis are done badly in nearly all cases right now but there are some exceptions: photoshop is the one I know very well, it's extremely powerful and polished, and uses keyboard extensively and not in a standard, dumbed down gui way.

Another example is Autocad that has both the command line and powerful and flexible gui that mirrors command line functionality. Whenever you click on a button and provide arguments, the complete command line command is also shown on the bottom. It can be reused by going back in the history of commands. It's ironic that in the case of autocad, you are actually limited by the fact that often you need to click on the right location in case of many commands so there's much less you can do with just the command line compared to most other gui apps that could be completely run in parallel by either gui or scriptable command line.
posted by rainy at 9:30 AM on May 19, 2011 [1 favorite]


On the topic of TermKit, I am very excited by this despite being somewhat disdainful of this newbie 'vim' thing with all the training wheels that the hipster kids on my lawn appear to be using these days with the window splitting and the syntax highlighting.

TermKit by itself won't replace xterm/sh for most of the Unix audience and definitely not the metafilter Unix audience, but you have to give the guy immense credit for pointing at the scriptures and asking the hard questions. And some dumb, easily answered questions ("why don't terms render in Helvetica?"), but hey, he's new! Cut him some slack.

The big question I really like, which yeah arguably I think he didn't know he asked, is "why isn't there a concurrent stream of metadata?"

For example, I think we can all agree that the idea that core Unix utilities should not change their behavior when streaming stdout to a terminal vs. to any other file descriptor; 'cat blah.png' should forever make your terminal shit itself in a glory of rendered ascii escape sequence interpretations.

But, things like progress, status, non-failure informational messages, logging; that stuff has always been a hodgepodge, and frankly programs (e.g. curl) have always guessed at the output and/or played stdout/stderr trickery to try to get people what they want, which is a visual meta description of the action in addition to the proper unix everything-is-a-character-stream behavior.

STDIN/STDOUT/STDERR/STDMETA?
posted by felix at 9:30 AM on May 19, 2011 [5 favorites]


Yeah, the problem with TermKit is that all it does is make more problems by adding a layer of abstraction. You are literally missing the entire point of Unix to dream something like this up. It does not work, it does not make any task easier or information more easily parsed.

Terminals could certainly be improved. This isn't even close to the right approach.
posted by wrok at 9:32 AM on May 19, 2011


enn: "Why can't I work with XML, or with JSON, or with HTML, or with any of the other ubiquitous modern plain-text formats in the shell as easily as I can pull data out of the columnar lsof output?"

Is the hypothesis that there aren't grep equivalents for XML? Because I seem to have an xpath program. I also use a tool called xmlstarlet which aims to be a kind of coreutils for XML.
posted by pwnguin at 9:33 AM on May 19, 2011


And none of them are power user apps.

What the hell is a power user?

Actually, no, very few people spend their entire day head-down in these sorts of things, so they're perpetually used by "new" users.

Umm. No. We're familiar with the command line, as there will occasionally be situations where the GUI isn't playing nice with the hardware, but day-to-day tasks are all staged and scheduled in the GUI. As someone who used to spend all day neck-deep in text editors and terminals of various stripes, the new way is much, much, much better. (With one exception - I wish to crap they'd just standardize UI widgets instead of everyone hiring their own UX wizard to reinvent the wheel. And no more standalone clients, move to web GUIs already.)
posted by Slap*Happy at 9:34 AM on May 19, 2011


@rainy -- yes, I don't think we're in disagreement.

The real point is to use the best tool for the job. Sometimes that's the CLI. Sometimes (especially with visual data -- e.g. image or video manipulation) that's a GUI.
posted by Celsius1414 at 9:35 AM on May 19, 2011


STDIN/STDOUT/STDERR/STDMETA?

STDIN/STDOUT/STDERR/STDMETA/STDGUI.

Or should that be GUIIN/GUIOUT?
posted by fings at 9:41 AM on May 19, 2011


And none of them are power user apps.

I do almost all of my unix admin work these days through web forms now.
Admittedly, it's all pretty simple, but I can get down to the shell if I need to. I just don't need to most of the time now.
posted by bonehead at 9:41 AM on May 19, 2011 [1 favorite]


Yeah, but when I leave my laptop screen open for everyone in the coffeeshop to see, how will they know I'm better than them?
posted by four panels at 9:44 AM on May 19, 2011 [2 favorites]


Most modern security appliances use GUI interfaces to configure and troubleshoot, and network traffic monitors are going the same way

Most "GUI"s in that case aren't really GUIs but graphical configuration file generators.

And frankly, I love me some graphical config file generators. SSHing into a server and editing foo.conf to enable some option is nice and has a lot of power to it - but I've spent way too much time playing "Find the misplaced semicolon" and "is that a tab or space and does it matter ?" to be really happy with editing those files by hand.

By providing a GUI for doing it, responsibility for generating a correct config file falls on the developer and not me. Given that I drink a lot, that's always a good thing.
posted by Pogo_Fuzzybutt at 9:45 AM on May 19, 2011 [2 favorites]


Yeah, but when I leave my laptop screen open for everyone in the coffeeshop to see, how will they know I'm better than them?

The glowing Apple logo on the lid takes care of all that. ;)
posted by Celsius1414 at 9:49 AM on May 19, 2011 [1 favorite]


bonehead: I do almost all of my unix admin work these days through web forms now.

That and editing config files directly is fine if you think you will never have to do it again, but that's hardly ever true. I find myself almost exclusively using puppet (with vim and git) now, since life is too short to do the same shit again and again.
posted by vanar sena at 9:55 AM on May 19, 2011 [1 favorite]


A GUI or a webform to do admin? These are exactly the people I'm talking about who don't know what they are doing.

If your task can be captured the action of a button, it can be scripted by definition and you should almost never have to do it manually. For everything else, and there is a lot of "everything else", you have the flexibility of stuff that CANNOT be (or has not yet been) scripted.
posted by DU at 10:01 AM on May 19, 2011 [1 favorite]


It's not nearly as egregious, but the whole thing reminds me of when somebody tried to make a "better" text editor for LInux. A closed-source ("you'll steal all my innovations!"), suid ("I need bare metal access because abstractions are too slow!"), edit.exe ("DOS edit is the best editor ever!") clone.

It was... not well received.
posted by kmz at 10:02 AM on May 19, 2011 [1 favorite]


kmz - do you remember what it was called? It seems hilarious.
posted by eeeeeez at 10:04 AM on May 19, 2011


rainy: It seems that most of those features are already available, either built into the terminal emulator or via shell environments like Midnight Commander.
posted by KirkJobSluder at 10:04 AM on May 19, 2011


The best part of this thread is "I'm not alone" feeling.

Although actually, almost all the coworkers I actually work with are CL bigots too. But just means we have no one to argue with
posted by DU at 10:07 AM on May 19, 2011


DU: A GUI or a webform to do admin? These are exactly the people I'm talking about who don't know what they are doing.

To be fair, that's only true if the use-case scenario for their entire unix administration needs are not covered by the web tools. For admins of customized complex boxen, that never happens. For casual users of prebuilt single-purpose AMIs ...
posted by felix at 10:09 AM on May 19, 2011


eeeeeez: It was mostly a Usenet thing. Here's one of the threads on it.
posted by kmz at 10:10 AM on May 19, 2011 [1 favorite]


I have read
the comments
that are in
this post

and which
you were probably
hoping
will be favorited

Forgive me
they were skipped over
so incomprehensible
and so arcane
posted by clorox at 10:13 AM on May 19, 2011 [2 favorites]


KirkJobSluder: no graphical functionality is avalable in xterm or konsole or widely used emulators at all. MC is very flawed.. I don't remember the specifics as it was years ago, but you can't effectively switch back and forth between shell and MC. For example, I don't think it even switches the directory. Maybe I should hunt around in MC configuration..
posted by rainy at 10:14 AM on May 19, 2011


If your task can be captured the action of a button, it can be scripted by definition and you should almost never have to do it manually.

Your decision however, if not scriptable, will need to be communicated to the computer. So now you're either putting values into command-line arguments you forget, or putting them into a web form. It's probably 4 and 8 of a dozen.

Stop being such an arse, eh? I mean, you don't even understand xargs, ffs
posted by bonaldi at 10:17 AM on May 19, 2011 [1 favorite]


What having web configurations really means to me is not having to spend time checking man pages and --help options to make certain I've got the right option checked on a command I last used months ago. For those of us who are only part-time admins, that's been a great boon. And yeah, some of those buttons are scripts or just complicated one-liners which I've had to write in some cases. Still, it's great to just be able to use a web browser to fire a quick command to reboot a switch (say) rather than even having to tunnel in and run a reboot script.
posted by bonehead at 10:21 AM on May 19, 2011


One problem with --help listings and man pages is that they don't separate stuff that you nearly always need from stuff that only 1% of people need once in a few years. Maybe there should be a parallel help system for command line that's less of an obscure reference and more of a "damnit how do I..".
posted by rainy at 10:28 AM on May 19, 2011


rainy: "damnit how do I.."

There is apropos(1), but in practice I've rarely found it useful. It either returns too much irrelevant info, or absolutely nothing.
posted by vanar sena at 10:33 AM on May 19, 2011


rainy: xterm at least has a scroll buffer, and I think MC had a shell pane. While these kinds of utilities won't cover every need, they do expand what you can do with the shell.
posted by KirkJobSluder at 10:36 AM on May 19, 2011


kmz - rich, thanks.
posted by eeeeeez at 10:36 AM on May 19, 2011


Never type "cat foo | bar". "bar < foo" does the exact same thing

I prefer "bar < foo | cat"
posted by sfenders at 10:38 AM on May 19, 2011 [1 favorite]


kmz: You don't forward your X session when you ssh?

I'm probably either irrationally old-fashioned or paranoid, but I treat the presence of libX11 on a server as a sort of small personal failure.
posted by vanar sena at 10:40 AM on May 19, 2011 [6 favorites]


vanar: yes, it seems like one of those things that can be useful but so rarely that you won't think of it even when it would be helpful.

KirkJob: I wish xterm scrolled with single key strokes. holding shift as you scroll many pages.. that's just perverse.
posted by rainy at 10:45 AM on May 19, 2011


http://en.wikipedia.org/wiki/Big_Duck
posted by erniepan at 10:49 AM on May 19, 2011


@rainy Can't you just remap the Page Up/Page Down behavior if you don't want to use the shift key?
posted by Celsius1414 at 11:00 AM on May 19, 2011


Okay, just to make it clear where each person is coming from in the GUI vs. Config file war -- you have to state whether or not you use any sort of source/configuration control tool.

My guess is that the GUI folk this it's unnecessary and the config file folk this it's essential.
posted by benito.strauss at 11:08 AM on May 19, 2011


weston: "The guys on the Bespin / Skywriter project have been thinking along the same lines for a while. Good stuff."

Ooh. I like his concept a lot better than TermKit. A bit more Unix-y and more thoroughly thought through in general (while also being presented as a concept rather than a finished product, which, IMO, is kind of important).
posted by schmod at 11:11 AM on May 19, 2011


Celsius1414, possibly, and then it will break something else like some curses app or the pager or whatever. Shell has many little shortcomings and my regular use of shell is pretty basic, I'm a programmer and not an admin. I look into a workaround for one thing after another once in a while, but I really wish there was an overhaul of shell environment from ground up. For example, I don't often use shell remotely and I think that's true for many people, and it'd make sense to have a more rich, modern, fancy shell environment for use on one's desktop, with dropdowns, some light unobtrusive graphic elements, better and smarter help system and a thousand other small touches. Then the whole experience would be also more welcoming to new users. A lot of them think shell interfaces are inherently cumbersome and that's just not true, it's mainly a historical artifact. Imagine for a second that computing world did not go into guis when it did and instead all of the improvement and research and development was on shell environments in the last 20 years.. shells would keep their power, scriptability, etc but would be so much better suited for occasional users, as well.
posted by rainy at 11:14 AM on May 19, 2011 [1 favorite]


Yeah! Bespin / Skywriter looks a lot more like what I was thinking about...
posted by rainy at 11:16 AM on May 19, 2011


while also being presented as a concept rather than a finished product, which, IMO, is kind of important
To be fair to TermKit, it's been presented as nothing other than alpha.
posted by edd at 11:20 AM on May 19, 2011


Like empath said up above, if you want to look at a modern shell paradigm you really need to start by looking at Windows PowerShell. It's got the same idea as TypeKit of typed data as the communication between processes instead of text. Microsoft's been working on it for awhile now, at least some 4 years.

Yeah, it seems like an ill-thought out rip-off of a Microsoft idea.

Why can't I work with XML, or with JSON, or with HTML, or with any of the other ubiquitous modern plain-text formats in the shell as easily as I can pull data out of the columnar lsof output?

Say hello to xml-coreutils.
posted by rodgerd at 11:58 AM on May 19, 2011 [1 favorite]


I may not be as smart as most of the people on this thread but at least I'm not as angry.
posted by OverlappingElvis at 12:08 PM on May 19, 2011 [5 favorites]


As I said, there are underlying pedagogical reasons for the unix philosophy. We should imho expand upon these for GUIs. What might this look like?

Imagine a compromise between an X server and a browser. As in your browser, all software gets divided into user interface and backend components. User interface components are written in a congenial but terse functional language like Haskell, or maybe it's strict cousin ML, or even Erlang. Java isn't nearly terse enough. User interfaces are cached in various stages of compilation by your display server/browser, but critically the browser always has access to the full source. All backends are simply libraries or provide an RPC interface.

We distinguish in user interface code between operations that invoke the backends and "gui bullshit". There is a little icon below all application windows that, if you click it, opens up a little terminal for that window. If you preform actions in the gui, the corresponding backend commands scroll by in this window, much like old AutoCAD menus. Any commands you type into this window are executed by the interface's main event loop.

Vola : Any child who picks up such computer will be able to program within a matter of months. In addition, interfaces may be written "stupidly" without support for "derived" functionality because commands like "map [function] [list]" are trivial.
posted by jeffburdges at 12:27 PM on May 19, 2011


Put down your torches, you neckbeards. Think of it this way: he wants to add a few more tools to your PATH (like a PNG viewer), but he's just doing it using a mechanism other than a pipe.

I love my ASCII, too, but as has been pointed out above, sometimes I need to do things across an SSH session that this might help.

And as for DU's "A GUI or a webform to do admin?" query, I submit Sun's Ops Manager. I can't even imagine how macho I would need to become in order to do that like of wide comliance reporting and job scheduling.
posted by wenestvedt at 12:32 PM on May 19, 2011 [1 favorite]


Put down your torches, you neckbeards.

Infantile, bullying namecalling demonstrates a bankrupt intellect.
posted by rodgerd at 12:37 PM on May 19, 2011 [1 favorite]


I may not be as smart as most of the people on this thread but at least I'm not as angry.

Put down your torches, you neckbeards.

Wel, from the article:

And that process is capable of communicating only in short little grunts of text, perhaps coalescing into a cutesy little ASCII art imitation of things that grown-ups call "dialogs", "progress bars", "tables" and "graphs".

Being dismissively condescending of an entire culture that's significantly older than the blogger tends to get people's hackles up.
posted by Celsius1414 at 12:46 PM on May 19, 2011 [2 favorites]


Infantile, bullying namecalling demonstrates a bankrupt intellect.

Nuh-UH!

That said, Hypercard just ruined me on Object Oriented Programming. I kept looking for these Objects...
posted by ChurchHatesTucker at 12:52 PM on May 19, 2011


"he wants to add a few more tools to your PATH"
The problem is that he seems to be changing what tools we already have and in not particularly clear ways.
posted by edd at 12:52 PM on May 19, 2011


he's just doing it using a mechanism other than a pipe

Out-of-the-box thinking like this is what drives progress in unexpected ways. On this one, I'd be less excited about what diehard old-school CLI hackers have to say about their prejudices about how things should be done, as opposed to the people who have insights on interesting and new things that these kinds of integrative approaches can accomplish.
posted by Blazecock Pileon at 1:05 PM on May 19, 2011 [1 favorite]


> The problem is that he seems to be changing what tools we already have and in not particularly clear ways.

Well, it's not like anyone's going to start shipping OSes with the old style CLI removed just because someone is making these kinds of GUI adaptations. The either/or mentality on display in this thread is kind of silly.
posted by Horselover Phattie at 1:08 PM on May 19, 2011 [2 favorites]


> To disable pagination for all commands, set core.pager or GIT_PAGER to cat.
and
> git config --global core.pager cat

What is up with this fascination with piping everything through cat? Just set core.pager to a blank string.

Example: git config --global core.pager ''

Friends don't let friends abuse cat(1)!
posted by narwhal bacon at 1:10 PM on May 19, 2011



Well, it's not like anyone's going to start shipping OSes with the old style CLI removed just because someone is making these kinds of GUI adaptations. The either/or mentality on display in this thread is kind of silly.

It's additionally weird if you think about how very few applications are purely GUI. Most are some sort of blend of CLI and GUI. Look at Matlab, or Excel, or even Firefox - they all have a clickable user interface and expose greater function via a command line interface of some sort.

To an extent, I get why extending that metaphor to a Shell or Terminal has some appeal. But I think that those metaphors break in a large number of use cases. As a system admin, I seem to fight my tools as much as the problems I charged to solve. I'm not real excited about yet another complex tool that gets in the way of getting shit done.

That having been said, I love tools that make the CLI easier for me to use. There are some neat ideas here, although, I am far from sold on the viability of all of his ideas.
posted by Pogo_Fuzzybutt at 1:22 PM on May 19, 2011


"Well, it's not like anyone's going to start shipping OSes with the old style CLI removed just because someone is making these kinds of GUI adaptations."
Well as I stated above I've not any particular problem with adapting how the terminal interprets the output of CLI tools, but if a CLI tool is changing its output based on where it thinks it is sending it, there's a problem. I suspect in this case, based on the flow charts the CLI tools are producing an additional stream and accepting an additional stream, but this seems hairy, and potentially non-conducive to the principle of being able to glue commands together.
It's going to need a fairly clear delineation between the output to the standard tools and to the more interpretive high level terminal application. I'm not totally against the idea as I was earlier today, but it needs some careful handling and deeper thought.
posted by edd at 1:26 PM on May 19, 2011


Infantile, bullying namecalling demonstrates a bankrupt intellect.
Guilty as charged! Did you see my posts in the thread about how hard it is to read long books? I even admit to skimming the required reading!
posted by wenestvedt at 1:27 PM on May 19, 2011


enn>
I would love to be able to do something like:
for url in `xpath http://example.com/url-list.xml < /some/path[@link=unexpired]/to/urls`; do curl $url; done
The CLI fairy has granted your wish!
curl http://example.com/url-list.xml |xmlstarlet sel -t -m //*[@link=unexpired] -v ./to/urls -n |xargs wget --mirror --
xmlstarlet sel is a rather clever way to write XSLT from a few command line flags.
posted by Tobu at 2:21 PM on May 19, 2011 [2 favorites]


We distinguish in user interface code between operations that invoke the backends and "gui bullshit". There is a little icon below all application windows that, if you click it, opens up a little terminal for that window. If you preform actions in the gui, the corresponding backend commands scroll by in this window, much like old AutoCAD menus. Any commands you type into this window are executed by the interface's main event loop.

Vola : Any child who picks up such computer will be able to program within a matter of months.


Check out Squeak, the Smalltalkers' attempt to retain relevance with the next generation. It doesn't allow children to pick up programming in a matter of months, despite the claims of its proponents.

Perhaps the problem is the language? Haskell or ML would be ever so much easier for children to lean than Smalltalk, I'm sure.
posted by Slap*Happy at 2:31 PM on May 19, 2011


This guy obviously has no CS background. Javascript, really?
posted by aesacus at 2:33 PM on May 19, 2011


The fact that shell utilities communicate in short little grunts of text is a feature rather than a bug. It means, among other things, that I can feed data from shell utilities into my own programs written in python, perl, or clisp without needing to worry about XML schemas and processing.
posted by KirkJobSluder at 2:35 PM on May 19, 2011 [2 favorites]


Javascript, really?

JavaScript really what?
posted by weston at 3:05 PM on May 19, 2011


I was ready to completely dismiss this until a koala crapped a rainbow on my brain. I love it!
posted by tmt at 3:08 PM on May 19, 2011


While I think the core utilities are best kept in their current form, the terminal emulator could stand quite a bit of work and I think the man system could supplemented with a more newcomer-friendly help system with a different information structure. I'm thinking something along the lines of a man-lite which provides the least you need to know about how to use a utility.
posted by KirkJobSluder at 3:12 PM on May 19, 2011


This guy obviously has no CS background. Javascript, really?

Really who would want to use an Object Oriented language with first class functions and closures? That's not a real language!

JavaScript: The World's Most Misunderstood Programming Language
posted by papercrane at 3:18 PM on May 19, 2011 [3 favorites]


This guy obviously has no CS background. Javascript, really?

Yours seems to be about 5 years out of date.
posted by Nelson at 3:30 PM on May 19, 2011 [3 favorites]


There is a reason for C++ verbosity. Yes, C++ isn't the sublime perfection of C, but raison d'être it has in spades.

Java, umm, not so much. All it's technical contributions were compilation tricks, very nice tricks, but nothing linguistic. Ideally, Sun should've written an extensible bytecode JIT compiler targeted at Smalltalk and other OO languages, prodding the CS community towards a coherent sensible object oriented language.

I'd argue Javascript is an improvement on Java linguistically because lambda expressions are mandatory for modern higher order languages. On preview, yeah what papercrane linked.
posted by jeffburdges at 3:32 PM on May 19, 2011


And, mime types? Half of the time I drop down into bash or plain text editors is because something came across the internet with some weird special-snowflake mime type. The resulting file is almost always plain text. Should my script report the data as application/octet-stream, text/plain, or text/tab-separated-values? What happens when I feed bzip2, a utility that should take everything, a stream that it's not registered to handle? Is there a mechanism for type coercion? Does reporting the wrong mime-type have the potential for invisible and unintended side effects? Will the core utilities throw an exception the same way that my web browser does and ask me every time how I want the file to be handled?
posted by KirkJobSluder at 3:42 PM on May 19, 2011


1) Javascript has nothing in common with Java (except, perhaps, a marketing department many years ago). Don't let the name fool you.

2) Java raised to exalted levels the arts of dynamic loading and byte-code verification. Reading about the Java Virtual Machine, while professionally useless, gave me a great appreciation for what the designers had done in creating Java. There was no claim to be doing anything interesting with language.
posted by benito.strauss at 4:29 PM on May 19, 2011


The problem is that he seems to be changing what tools we already have and in not particularly clear ways.

I hate it when someone sneaks into my house and steals my copy of bash, replacing it with a visually enhanced version that I can't get rid of.

Damnit, unconed!

In all seriousness, it's really cool to see the buzz that this project is getting. I've known unconed for a few years now, and chat with him on and off when he does curious and interesting things. He's been kicking this "next generation command line" concept around for pushing a year now. He's about as far from a "Dumb things down for the users!" type as you can imagine; he's just interested in exploring ways to build new smarts on top of existing toolchains.

The explosion of interest around this early dev code has been interesting to watch, in part because of the number of people who demonstrate legitimate anger at the idea of a terminal doing anything but firehosing text.
posted by verb at 4:55 PM on May 19, 2011 [3 favorites]


Also, an interesting bit from the comment thread on unconed's blog. Eric Raymond stopped by to weigh in:
"I wrote The Art of Unix Programming and yield to no one in my appreciation of old-school Unix virtues. And I think most of the negative comments here are bunk. The Unix tradition is not a fossil or a museum exhibit, and it ill behooves those of us who value it to behave like grouchy old farts when someone brings fresh thinking to its problems.

I think the convention of using JSON for the view pipes is particularly clever and appropriate. It's an excellent way of preserving the old-school virtues of textuality while also supporting the ability to pass structured objects around.

If Steve Witten ships this for Linux I'll try it out."
posted by verb at 5:49 PM on May 19, 2011 [3 favorites]


verb, I believe the anger stems from the thinking (correct, IMO) that a terminal is a terminal, and should indeed firehose text because it is in fact a terminal and not something else.
posted by wierdo at 5:51 PM on May 19, 2011


verb, I believe the anger stems from the thinking (correct, IMO) that a terminal is a terminal, and should indeed firehose text because it is in fact a terminal and not something else.

ter·mi·nal (tûrm-nl)
adj.
  1. Of, at, relating to, or forming a limit, boundary, extremity, or end.
  2. Botany Growing or appearing at the end of a stem, branch, stalk, or similar part.
  3. Of, at, relating to, or being the end of a section or series; final. See Synonyms at last1. Relating to or occurring in a term or each term: terminal inventories.
  4. Causing, ending in, or approaching death; fatal: terminal cancer; a terminal patient.

n.
  1. A point or part that forms the end.
  2. An ornamental figure or object placed at the end of a larger structure; a finial.
  3. Electricity
    1. A position in a circuit or device at which a connection is normally established or broken.
    2. A passive conductor at such a position used to facilitate the connection.

    1. Either end of a railroad or other transportation line; a terminus.
    2. A station at the end of a transportation line or at a major junction on a transportation line.
    3. A town at the end of a transportation line.

  4. Computer Science A device, often equipped with a keyboard and a video display, through which data or information can be entered or displayed.



I understand that you're talking about traditional terminal implementations, but I'm just not seeing the "text only" imperative there. It's a proof of concept demonstration of tokenized io in a terminal application rather than simple text piping. Graphical cues and inline presentation of rich data types are some of the things that can be done with that rich data.

Better auto-completion is another interesting possibility, for example. In any case, I'm still not getting the hate. Figlet must give some of these people hives.
posted by verb at 6:06 PM on May 19, 2011


Graphical cues and inline presentation of rich data types is also something that ruins the entire purpose of the terminal. Of course, I spend much of my day dealing with computers whose only command interface is provided over a serial port, so what do I know?
posted by wierdo at 6:11 PM on May 19, 2011


Graphical cues and inline presentation of rich data types is also something that ruins the entire purpose of the terminal. Of course, I spend much of my day dealing with computers whose only command interface is provided over a serial port, so what do I know?

It's a pity that computers can only provide one interface, ever, for all devices and all users.
posted by verb at 6:12 PM on May 19, 2011 [3 favorites]


verb: I understand that you're talking about traditional terminal implementations, but I'm just not seeing the "text only" imperative there.

The imperative isn't "text only," because after all, unix pipes can handle a lot of data that's not text. The imperative is not to break interoperability in ways that burden the user. The advantage of simple text piping is that I don't have to worry about compatible mime types, parsing metadata, or an API in order to get data from any of the standard unix tools into my script. 90% of the time, I don't need a rich data model. I just need a delimited list. Keep the default simple text behavior intact, and I'll be happy. (And how do you even tokenize output from sed?)

Since we have an ls -G, I don't object to an ls --json. But here's why I'm scratching my head on this. If I need a complex data model, I'm not going to use the shell at all. I'll build the whole thing in pure python so I don't have to worry about serialization and setting metadata between each process.
posted by KirkJobSluder at 8:05 PM on May 19, 2011


90% of the time, I don't need a rich data model. I just need a delimited list. Keep the default simple text behavior intact, and I'll be happy. (And how do you even tokenize output from sed?)

That's the kind of stuff I asked unconed when he started talking about this project. He said he'd find out. Given the things he's done in the past when he got the itch for a particular problem, I'm not betting against him.
posted by verb at 8:14 PM on May 19, 2011


verb wrote: "It's a pity that computers can only provide one interface, ever, for all devices and all users."

Yeah, you're arguing against something I'm not even talking about. TermKit looks fine for what it's trying to be, but it's not a freakin' terminal.
posted by wierdo at 9:39 PM on May 19, 2011


The explosion of interest around this early dev code has been interesting to watch, in part because of the number of people who demonstrate legitimate anger at the idea of a terminal doing anything but firehosing text.
Since you opened that comment with a quotation from me, I feel somewhat obliged to point out that's the complete opposite of my concerns. I have no particular problem with a terminal that takes a data-stream and chooses to do something with it before it hits the user. I have a problem with potentially breaking tools.
Going back to cat for example, it's job is solely to take a list of arguments and consecutively send their contents to stdout. Now cat has to add content wrappers instead (guessing what that should be usually, sometimes getting it wrong), and tools at the other end have to remove the content wrappers, and not just from the outside of the content because cat may now have stuck content wrappers inside the content. On top of this, now those content wrappers have to be escaped because what if I happened to pipe a document documenting content wrappers? And now my way of thinking about a simple terminal command is making me want to cry.
I think the idea of a smarter terminal is good, but I'd be hesitant to say any implementation of a smart terminal is going to be good.
posted by edd at 1:57 AM on May 20, 2011


Training wheels don't make me angry. I know some people need them.

The insinuation that training wheels would improve my bicycle does insult me though.
posted by idiopath at 7:11 AM on May 20, 2011


Some thoughts:

The first thing that struck me while I was reading this was how painful it was going to be to write utilities that use this pipeline system. On Unix, you have two filehandles--stdin and stdout--giving you streams of bytes and that's it. (You can ignore stderr if you want to.) If you need to do something special in the middle of your pipeline--remove every line which contains exactly 17 instances of the letter 'e', for (silly) example--it's pretty easy to write a program that does this.

The Perl implementation is one line:

perl -e 'while(<>) {@x = /e/g; print unless @x == 17}'

I do stuff like this all the time when handling somewhat-formatted text. But TermKit needs me to ensure that the input has the right MIME type, set the MIME type for the output, tell the UI pipes to go away and who knows what other stuff before I do any actual work. I certainly wouldn't be able to fit it in my command.

So I agree with the folks who say that he's missing the point. The Unix command line is intended for people who are willing to spend time upfront learning how to use it. TermKit looks to me like it's trying to be easier to learn but harder to actually use once you've mastered it.

That being said, I really like the idea of merging (or at least closely integrating) the shell and the terminal. Both interfaces have their strengths and I'd like to be able to combine them.

I also agree that Unix commands should do better than outputting "somewhat parsable text". I don't think JSON (or XML) is the way to go, though. What I'd really like to see is the existing Unix commands start to get a '--well-formed' flag that formats the output as lines of tab-delimited fields in a fixed, well-documented order. That's parsable while still being close enough to the Unix Way to make for an easy transition.

So there are certainly some good ideas here but I'm dubious about TermKit ever replacing the xterm/bash combination.
posted by suetanvil at 9:38 AM on May 20, 2011 [1 favorite]


« Older Cart-pimpers and cat counselors   |   21st century Tin Pan Alley Newer »


This thread has been archived and is closed to new comments