"The sad state of web app deployment"
October 1, 2015 5:03 AM Subscribe
Oh so very much this. The entire ecosystem of web application development and deployment, regardless of platform, remains a total and utter ballache from top to bottom. So many hoops of bullshit to go through before you can even start solving the business problems.
posted by GallonOfAlan at 5:20 AM on October 1, 2015 [4 favorites]
posted by GallonOfAlan at 5:20 AM on October 1, 2015 [4 favorites]
I don't know a ton about Rails, but his greater point is that he wants everything to be better and make more sense from the lowest level possible instead of people just trying to abstract their way around problems.
Great.
I, too, want this, and I try pretty hard to not abstract away things I don't understand when I'm writing software. I even rewrite existing things that are written badly (gasp! "Reinventing the wheel you are a sinner etc."). However, I do not always have time to do this. That is the positions most people are in. Motherfucker, people have jobs and need to get things done under time pressure. What the fuck are you doing about this, blog guy?
But we haven’t even tried to solve this, and all the people who are most capable of solving it are too busy scaling Twitter or Amazon up to ten million servers or whatever.
You know why? Because they get paid money to do it so they can have apartments and shit. Are you going to get rid of capitalism? Great! I want basic income. Please do that for me.
But that's not even completely true. People try to solve this all the time. Sorry if it did not work well enough for you to install your free web app all breezy-like. Also, plenty of people do amazing things for free, even though they could be making a ton of money.
posted by ignignokt at 5:23 AM on October 1, 2015 [24 favorites]
Great.
I, too, want this, and I try pretty hard to not abstract away things I don't understand when I'm writing software. I even rewrite existing things that are written badly (gasp! "Reinventing the wheel you are a sinner etc."). However, I do not always have time to do this. That is the positions most people are in. Motherfucker, people have jobs and need to get things done under time pressure. What the fuck are you doing about this, blog guy?
But we haven’t even tried to solve this, and all the people who are most capable of solving it are too busy scaling Twitter or Amazon up to ten million servers or whatever.
You know why? Because they get paid money to do it so they can have apartments and shit. Are you going to get rid of capitalism? Great! I want basic income. Please do that for me.
But that's not even completely true. People try to solve this all the time. Sorry if it did not work well enough for you to install your free web app all breezy-like. Also, plenty of people do amazing things for free, even though they could be making a ton of money.
posted by ignignokt at 5:23 AM on October 1, 2015 [24 favorites]
Docker is last weeks news! It's all Webpack now!
posted by Artw at 5:24 AM on October 1, 2015 [5 favorites]
posted by Artw at 5:24 AM on October 1, 2015 [5 favorites]
I'm sure all this headache has nothing to do with the fact that it seems like the majority of the hip kids these days are more apt to throw up their hands in disgust and go make a whole new set of tools from scratch than they are to actually work to improve something that's good but not perfect. Why iterate when you can obviously do it better if you did it all over again to match your own personal vision?
(I swear to god, you can't throw a rock without it bouncing off a dozen different web stacks trying to solve the same problem in slightly different ways)
posted by tocts at 5:26 AM on October 1, 2015 [42 favorites]
(I swear to god, you can't throw a rock without it bouncing off a dozen different web stacks trying to solve the same problem in slightly different ways)
posted by tocts at 5:26 AM on October 1, 2015 [42 favorites]
This article basically cements that I will not touch Ruby with any keyboard in the near future.
Thanks for posting!
posted by JoeXIII007 at 5:32 AM on October 1, 2015 [13 favorites]
Thanks for posting!
posted by JoeXIII007 at 5:32 AM on October 1, 2015 [13 favorites]
So, a second look at this quote makes me think that it may be one of the causes of the problem:
You know what's was a huge installation hassle? Installing stuff on the Mac. How was that largely solved? Homebrew. It was written by a guy who Google said wasn't smart enough to work for them.
The complaining is understandable, but I am infuriated when people say that only magic elite dev troopers from Google or Somewhere Else can solve problems. I once worked with a guy who, once we were figured out that a terrible PHP open source framework in use by a project we inherited wasn't doing things correctly, shrugged and said, "Welp, that was written by men smarter than you or I."
No. No, it was not, and that is a horrible attitude that results in bad software.
posted by ignignokt at 5:34 AM on October 1, 2015 [49 favorites]
But we haven’t even tried to solve this, and all the people who are most capable of solving it are too busy scaling Twitter or Amazon up to ten million servers or whatever.This guy assumes that the people to solve this problem are at Twitter or some other elite company. "Regular" developers can't solve this.
You know what's was a huge installation hassle? Installing stuff on the Mac. How was that largely solved? Homebrew. It was written by a guy who Google said wasn't smart enough to work for them.
The complaining is understandable, but I am infuriated when people say that only magic elite dev troopers from Google or Somewhere Else can solve problems. I once worked with a guy who, once we were figured out that a terrible PHP open source framework in use by a project we inherited wasn't doing things correctly, shrugged and said, "Welp, that was written by men smarter than you or I."
No. No, it was not, and that is a horrible attitude that results in bad software.
posted by ignignokt at 5:34 AM on October 1, 2015 [49 favorites]
It's always fun to read a good rant and she/he has some reasonable points but a lot of this seems self inflicted (32-bit userspace + 64-bit kernel?). We're currently converting the Rails server application that I QA for to Docker and the first time I was handed the Docker version of the app, it took less than a half hour to install Docker and get the server running and get the old acceptance tests running. I know that I do this for a living so it seems easier to me than it probably is for a novice but it's not nearly as hard as this person makes it out to be.
posted by octothorpe at 5:40 AM on October 1, 2015 [12 favorites]
posted by octothorpe at 5:40 AM on October 1, 2015 [12 favorites]
(I swear to god, you can't throw a rock without it bouncing off a dozen different web stacks trying to solve the same problem in slightly different ways)
I don't get why this bothers you. You don't have to use any stack you don't like. I don't like, say, Ember, but it would be ridiculous of me to resent that it exists and makes people happy. Is the idea that if the makers of Ember didn't make it, they'd be working on something you like?
posted by ignignokt at 5:42 AM on October 1, 2015 [3 favorites]
I don't get why this bothers you. You don't have to use any stack you don't like. I don't like, say, Ember, but it would be ridiculous of me to resent that it exists and makes people happy. Is the idea that if the makers of Ember didn't make it, they'd be working on something you like?
posted by ignignokt at 5:42 AM on October 1, 2015 [3 favorites]
The complaining is understandable, but I am infuriated when people say that only magic elite dev troopers from Google or Somewhere Else can solve problems. I once worked with a guy who, once we were figured out that a terrible PHP open source framework in use by a project we inherited wasn't doing things correctly, shrugged and said, "Welp, that was written by men smarter than you or I."
No. No, it was not, and that is a horrible attitude that results in bad software.
I fix code for a living. The original author might or might not have been smarter, that's not the problem. It's the lack of understanding how hard building software is and that most time you cannot genius your way through it. More a problem with hubris than intelligence.
posted by KaizenSoze at 5:43 AM on October 1, 2015 [11 favorites]
I don't get why this bothers you.
It bothers me primarily because it results in a lot of effort being spent solving the same problems, over and over. And hey, I'm not your boss, you can do whatever, and if it makes you happy, that's fine. That doesn't mean I can't be annoyed that it feels like there's a lot of hard problems out there that haven't been solved, but oh hey here's another project that's totally going to be better than the current tools once it finishes re-implementing all the functionality that duplicates what already could be done (but don't worry it will totally be worth it when it's done, and will totally justify starting from scratch, I swear!).
posted by tocts at 5:50 AM on October 1, 2015 [12 favorites]
It bothers me primarily because it results in a lot of effort being spent solving the same problems, over and over. And hey, I'm not your boss, you can do whatever, and if it makes you happy, that's fine. That doesn't mean I can't be annoyed that it feels like there's a lot of hard problems out there that haven't been solved, but oh hey here's another project that's totally going to be better than the current tools once it finishes re-implementing all the functionality that duplicates what already could be done (but don't worry it will totally be worth it when it's done, and will totally justify starting from scratch, I swear!).
posted by tocts at 5:50 AM on October 1, 2015 [12 favorites]
The struggle is real, and it's by no means limited to Rails or Ruby. (Although it sounds like Rails has finally caught up to PHP in this regard. Heh.)
It sounds like much of the author's problems had to do with poorly designed software, with poor documentation. Welcome to open source. (There's plenty of fantastic open-source software, of course—and plenty of shitty commercial software. But OS is its own special kind of jungle, with all the blessings and curses that come from that.)
This is great. I've noticed this kind of cargo-cult approach to technology way more in the last few years, and it's really depressing.
This is because most people are just trying to get shit done. They don't have enough background to figure out the right answers (or understand them when they see them), and they don't have hundreds of hours to invest in gaining that expertise. Any given domain of computer technology is just too big for one person to keep up with. I do it too—we all do. I don't fully understand every command I copy-and-paste from every installation tutorial or Stack Overflow answer (although I certainly stop and look up any unfamiliar flags, and try not to run anything obviously boneheaded or suicidal). It'd be sweet if I could stop and spend several days or weeks getting up to speed with every new technology I need to use. I can't. If you know an employer who'll allow me to do that, and they're hiring, please send me a PM :)
Anyway, you arguably shouldn't have to be an expert in a given language or technology stack just to use applications written in that language, or for that stack. Installing a web forum seems like a prime example of something you should be able to accomplish without opening the hood, taking apart the engine, installing obscure and poorly documented aftermarket parts, putting it back together, and driving it sideways.
Still, it's not obvious how to solve the problem. Part of it, I think, is just inherent to open source: OS is largely written by volunteers, who want to work on the shiny fun stuff; they have no one to compel them to pay more attention to more mundane but equally important concerns; they tend to assume that everyone is as savvy with the platform in question as they are. None of these are insurmountable—many open-source projects do surmount them—but they are forces that are unique to OS. Software written this way would never fly as a commercial product.
The guy complains about having to install a thing that will help standardize the install process (Docker), but then complains that the ecosystem is such a jungle without such a thing. Well, pick one. Or fork Rails and redesign it so that kind of standardization is baked in (and accept the fact that you're breaking compatibility with existing software).
For all of its many, many flaws, WordPress actually delivers on its "famous five-minute install". That's 100% due to the way the software is written (and it's one of very few ways in which WordPress can be said to be designed well): they've specifically and consciously avoided introducing dependencies on anything remotely exotic, non-standard, or technically demanding on the user. (Sometimes this does reach absurd levels—see wp-cron, a complete [sort of] reimplementation of cron for the benefit of users who don't have command-line access to their servers. Which doesn't actually work.) But overall, it's a huge reason for its success. If you want to start a blog, and you have a modicum of technical ability (not years of experience with Linux and the rest of the LAMP stack), you can follow a tutorial with half a dozen simple steps, and they'll just work in 99.9% of environments.
posted by escape from the potato planet at 5:50 AM on October 1, 2015 [8 favorites]
It sounds like much of the author's problems had to do with poorly designed software, with poor documentation. Welcome to open source. (There's plenty of fantastic open-source software, of course—and plenty of shitty commercial software. But OS is its own special kind of jungle, with all the blessings and curses that come from that.)
This is great. I've noticed this kind of cargo-cult approach to technology way more in the last few years, and it's really depressing.
This is because most people are just trying to get shit done. They don't have enough background to figure out the right answers (or understand them when they see them), and they don't have hundreds of hours to invest in gaining that expertise. Any given domain of computer technology is just too big for one person to keep up with. I do it too—we all do. I don't fully understand every command I copy-and-paste from every installation tutorial or Stack Overflow answer (although I certainly stop and look up any unfamiliar flags, and try not to run anything obviously boneheaded or suicidal). It'd be sweet if I could stop and spend several days or weeks getting up to speed with every new technology I need to use. I can't. If you know an employer who'll allow me to do that, and they're hiring, please send me a PM :)
Anyway, you arguably shouldn't have to be an expert in a given language or technology stack just to use applications written in that language, or for that stack. Installing a web forum seems like a prime example of something you should be able to accomplish without opening the hood, taking apart the engine, installing obscure and poorly documented aftermarket parts, putting it back together, and driving it sideways.
Still, it's not obvious how to solve the problem. Part of it, I think, is just inherent to open source: OS is largely written by volunteers, who want to work on the shiny fun stuff; they have no one to compel them to pay more attention to more mundane but equally important concerns; they tend to assume that everyone is as savvy with the platform in question as they are. None of these are insurmountable—many open-source projects do surmount them—but they are forces that are unique to OS. Software written this way would never fly as a commercial product.
The guy complains about having to install a thing that will help standardize the install process (Docker), but then complains that the ecosystem is such a jungle without such a thing. Well, pick one. Or fork Rails and redesign it so that kind of standardization is baked in (and accept the fact that you're breaking compatibility with existing software).
For all of its many, many flaws, WordPress actually delivers on its "famous five-minute install". That's 100% due to the way the software is written (and it's one of very few ways in which WordPress can be said to be designed well): they've specifically and consciously avoided introducing dependencies on anything remotely exotic, non-standard, or technically demanding on the user. (Sometimes this does reach absurd levels—see wp-cron, a complete [sort of] reimplementation of cron for the benefit of users who don't have command-line access to their servers. Which doesn't actually work.) But overall, it's a huge reason for its success. If you want to start a blog, and you have a modicum of technical ability (not years of experience with Linux and the rest of the LAMP stack), you can follow a tutorial with half a dozen simple steps, and they'll just work in 99.9% of environments.
posted by escape from the potato planet at 5:50 AM on October 1, 2015 [8 favorites]
I also build and fix software for a living and have built quite a bit of software. I understand how hard it is, but also how to weigh the benefits of something that is entrenched and bad vs. introducing something new. Something is not automatically better simply because it exists.
On Docker: It's not for everything, but in case you're wondering why people bother: I generally don’t like to do what the cool kids do, I still think PHP was a good choice of language for This Is My Jam, and I still think a phone should have a physical keypad. But Docker looked really good.
posted by ignignokt at 5:52 AM on October 1, 2015 [1 favorite]
On Docker: It's not for everything, but in case you're wondering why people bother: I generally don’t like to do what the cool kids do, I still think PHP was a good choice of language for This Is My Jam, and I still think a phone should have a physical keypad. But Docker looked really good.
posted by ignignokt at 5:52 AM on October 1, 2015 [1 favorite]
Why bother working closely with others and contributing anything upstream when you can just roll your own, spew it onto github with an MIT license and then abandon it for something different a week later? I'm looking at you, anyone who thinks it would be a great idea to write a static site generator.
posted by Poldo at 5:56 AM on October 1, 2015 [6 favorites]
posted by Poldo at 5:56 AM on October 1, 2015 [6 favorites]
So ruby is still as much an impossible mess to install as it was in 2005? GTK.
posted by Annika Cicada at 6:05 AM on October 1, 2015 [3 favorites]
posted by Annika Cicada at 6:05 AM on October 1, 2015 [3 favorites]
<getoffamylawn> Most of this stuff is horribly documented, and most of these developers have no idea what good documentation even looks like. Where is the modern-day equivalent of Kernighan and Ritchie or Kernighan and Pike? Those books are classics because they were the perfect combination of brevity and lucidity, with practical real-world examples. </getoffamylawn>
posted by 1970s Antihero at 6:05 AM on October 1, 2015 [7 favorites]
posted by 1970s Antihero at 6:05 AM on October 1, 2015 [7 favorites]
I also build and fix software for a living and have built quite a bit of software. I understand how hard it is, but also how to weigh the benefits of something that is entrenched and bad vs. introducing something new. Something is not automatically better simply because it exists.
Oh, I very much agree. The new shiny is a real problem is software. Most of the time things just die and are buried in the invisible graveyard.
*Slight tangent. Javascript seems to be going through the same framework churn that Java did in the early 2000s. New popular framework every 6-12 months. Those are just the big corporate backed ones. Someone will be able to make a good living for years to come maintaining critical UIs based on dead frameworks. The modern COBOL programmer. Of course, COBOL programmers still make very good money even today. I wonder which framework will win. Spring basically for Java.
posted by KaizenSoze at 6:05 AM on October 1, 2015 [3 favorites]
Rails freak: "haha your php site is lame LOL"
PHP dude: "this site has been up for years, has withstood 3 php version bumps and 2 Web server upgrades, and performs near native. Is your demo site deployed yet?"
Rails: "... we managed to bring it up without any security or caching but our cloud hosting upgrade to Ruby point release and now two of the gems won't compile and for some reason nosql is losing data, very slowly. But PHP sucks!"
posted by clvrmnky at 6:06 AM on October 1, 2015 [43 favorites]
PHP dude: "this site has been up for years, has withstood 3 php version bumps and 2 Web server upgrades, and performs near native. Is your demo site deployed yet?"
Rails: "... we managed to bring it up without any security or caching but our cloud hosting upgrade to Ruby point release and now two of the gems won't compile and for some reason nosql is losing data, very slowly. But PHP sucks!"
posted by clvrmnky at 6:06 AM on October 1, 2015 [43 favorites]
1. Documentation is not part of move fast and break things.
2. Most programmers can't write well. The only reason I write slight better than most is the patience of my professional editor wife.
posted by KaizenSoze at 6:10 AM on October 1, 2015 [3 favorites]
I don't think we've ever had a Golden Age of Software Installation and Configuration (which the author more-or-less admits at the end). Building and configuring software has always sucked. Package managers (like Docker) and installation scripts trade flexibility for simplicity. You want to do something that isn't part of the standard configuration, YMMV.
It's true though that we haven't had any Big Ideas (at least none widely-adopted) that make any of this stuff very much easier. Docker is still basically running shell commands in a fancy chroot() jail. Maybe we should just accept the current state of affairs where a Linux VM running a single app is a LRU (line-replaceable unit, as they say in hardware).
This "ports < 1024 must be root" rule is ancient and should be deprecated, though.
I also love the comments on that link, almost every one being "Have you looked into X? I hear it's better."
posted by RobotVoodooPower at 6:10 AM on October 1, 2015 [5 favorites]
It's true though that we haven't had any Big Ideas (at least none widely-adopted) that make any of this stuff very much easier. Docker is still basically running shell commands in a fancy chroot() jail. Maybe we should just accept the current state of affairs where a Linux VM running a single app is a LRU (line-replaceable unit, as they say in hardware).
This "ports < 1024 must be root" rule is ancient and should be deprecated, though.
I also love the comments on that link, almost every one being "Have you looked into X? I hear it's better."
posted by RobotVoodooPower at 6:10 AM on October 1, 2015 [5 favorites]
Where is the modern-day equivalent of Kernighan and Ritchie or Kernighan and Pike?
Books are pretty useless for this kind of stuff these days; by the time anything can get written, edited and published, the technology will have revved three times.
posted by octothorpe at 6:11 AM on October 1, 2015 [9 favorites]
Books are pretty useless for this kind of stuff these days; by the time anything can get written, edited and published, the technology will have revved three times.
posted by octothorpe at 6:11 AM on October 1, 2015 [9 favorites]
Here's my summation of the problem:
We're all working for metric driven MBA led corporations that are built on extracting the ultimate efficiency from each individual contribution unit.
That means we sling out crap and garbage as fast as we can to solve some stupid blocker in our way and out of fear of the violating the JIRA estimated time guardrail, then we move on to the next equally senseless task to extract more money for our technocratic plantation owners.
Then we get stress related health problems and leave technology after twenty years, leaving a harried, half-broke mess for the next generation to inherent and beat their heads on out of frustration while they too also stress out about their ridiculous deadlines and metric and self assessments and quarterly reviews.
The code base sucks because the way we value the time of our employees devalues the process of creativity inefficiency to the point that we solve problems in short sighted and brittle ways.
posted by Annika Cicada at 6:15 AM on October 1, 2015 [42 favorites]
We're all working for metric driven MBA led corporations that are built on extracting the ultimate efficiency from each individual contribution unit.
That means we sling out crap and garbage as fast as we can to solve some stupid blocker in our way and out of fear of the violating the JIRA estimated time guardrail, then we move on to the next equally senseless task to extract more money for our technocratic plantation owners.
Then we get stress related health problems and leave technology after twenty years, leaving a harried, half-broke mess for the next generation to inherent and beat their heads on out of frustration while they too also stress out about their ridiculous deadlines and metric and self assessments and quarterly reviews.
The code base sucks because the way we value the time of our employees devalues the process of creativity inefficiency to the point that we solve problems in short sighted and brittle ways.
posted by Annika Cicada at 6:15 AM on October 1, 2015 [42 favorites]
Yup, that pretty much sums it up. PHP the language is a bunch of warts flying in loose formation, but PHP as a development / deployment world seems to manage things pretty well. Ruby is the converse: a beautiful little language, surrounded by awfulness.
Docker is a valiant attempt to paper over all of this, but only by reifying the steps needed to install a service into a fixed opaque chunk that requires total trust that the author of the docker image is competent not just at creating the service you want, but also all the other hard things about service delivery. Things like security, logging, service introspection, upgradeability etc etc .... these are not simple things & it’s unrealistic to expect them to be delivered by the average programmer, yet that’s what Docker files expect us to do. Madness.
posted by pharm at 6:15 AM on October 1, 2015 [8 favorites]
Docker is a valiant attempt to paper over all of this, but only by reifying the steps needed to install a service into a fixed opaque chunk that requires total trust that the author of the docker image is competent not just at creating the service you want, but also all the other hard things about service delivery. Things like security, logging, service introspection, upgradeability etc etc .... these are not simple things & it’s unrealistic to expect them to be delivered by the average programmer, yet that’s what Docker files expect us to do. Madness.
posted by pharm at 6:15 AM on October 1, 2015 [8 favorites]
> a lot of this seems self inflicted (32-bit userspace + 64-bit kernel?).
(There’s a perfectly good reason for this.)
I did keep hoping to find out what the perfectly good reason was, and not just all about the problems it caused. But really, this is just the *nix way of life and has been since forever. *nix is scrambled eggs in the first place. (If it weren't, there wouldn't be such a generous oversupply of rationales purporting to explain why it really isn't.) If, instead of building your app on just standard scrambled eggs, you would rather build it on Scrambled eggs Super-Dee-Dooper-Dee-Booper, Special deluxe a-la-Peter T. Hooper, you can just expect to have to deal with a bunch of amusing distractions from your main development task before you get your whatever running reliably.
Or as RobotVoodooPower put it more elegantly,
> You want to do something that isn't part of the standard configuration, YMMV.
posted by jfuller at 6:18 AM on October 1, 2015 [2 favorites]
(There’s a perfectly good reason for this.)
I did keep hoping to find out what the perfectly good reason was, and not just all about the problems it caused. But really, this is just the *nix way of life and has been since forever. *nix is scrambled eggs in the first place. (If it weren't, there wouldn't be such a generous oversupply of rationales purporting to explain why it really isn't.) If, instead of building your app on just standard scrambled eggs, you would rather build it on Scrambled eggs Super-Dee-Dooper-Dee-Booper, Special deluxe a-la-Peter T. Hooper, you can just expect to have to deal with a bunch of amusing distractions from your main development task before you get your whatever running reliably.
Or as RobotVoodooPower put it more elegantly,
> You want to do something that isn't part of the standard configuration, YMMV.
posted by jfuller at 6:18 AM on October 1, 2015 [2 favorites]
We're all working for metric driven MBA led corporations that are built on extracting the ultimate efficiency from each individual contribution unit.I am sympathetic to the awfulness of working on production software for any business, anywhere, but before you blame the faceless but neatly-dressed MBAs, consider that it's as much about the reality of having a business - all production web app software in a growing company is garbage because it only has a six month lifespan or so. If you don't solve your problem in a short-sighted and brittle way, congrats, it took you two years and your competitors have put you out of business.
In other words, I look forward to the revolution as well, comrade, but until then here's my minimum acceptable patch that hits the deadline ;)
(obviously this doesn't apply to stuff like flight control software or whatever but that's .1% of what programmers get paid to do)
posted by thedaniel at 6:20 AM on October 1, 2015 [7 favorites]
Most of this stuff is horribly documented, and most of these developers have no idea what good documentation even looks like.
That's because hacking is driven by "scratch the itch." Once the code runs on the happy path, then the itch goes away, and all that boring detail work -- security, maintainability, documentation -- gets laid aside because it's boring.
The *very first thing* I look at with production software is the documentation. If there isn't good documentation, it isn't production ready.
So ruby is still as much an impossible mess to install as it was in 2005? GTK.
Installation is the easy part. Try patching it!
The reason ruby/rails does not exist in my prod environments -- I ask how do we handle security patches, and the answer is always "I dunno....I'll get an answer for you." Three weeks later, I ask again, and "We're not sure yet."
BZZT. Thank you for playing.
posted by eriko at 6:22 AM on October 1, 2015 [8 favorites]
That's because hacking is driven by "scratch the itch." Once the code runs on the happy path, then the itch goes away, and all that boring detail work -- security, maintainability, documentation -- gets laid aside because it's boring.
The *very first thing* I look at with production software is the documentation. If there isn't good documentation, it isn't production ready.
So ruby is still as much an impossible mess to install as it was in 2005? GTK.
Installation is the easy part. Try patching it!
The reason ruby/rails does not exist in my prod environments -- I ask how do we handle security patches, and the answer is always "I dunno....I'll get an answer for you." Three weeks later, I ask again, and "We're not sure yet."
BZZT. Thank you for playing.
posted by eriko at 6:22 AM on October 1, 2015 [8 favorites]
Years ago, there was a writer at Linux Today who reviewed Linux distributions, and I read his articles. He really really liked anti-aliased fonts (which were bleeding-edge in Linux at the time) and he really really liked bleeding-edge KDE, installed from *source*, not a package manager, and he really really wanted it installed in /opt, not /usr or /usr/local.
So his review of every Linux distribution was entirely concerned, not with how to use the distro the way the developers wanted to, but on how easy it was to compile bleeding-edge KDE with bleeding-edge font anti-aliasing and keep it all in /opt. That was how distributions stood or fell.
The first part of this article kind of read like that. "There's a standard, easy way to do this, but my environment isn't compatible with it so I'm going to declare that other ways of installing it should be just as easy! My system has a weird mix of a 64 bit kernel and 32 bit userspace. That shouldn't be a problem but it is! Why is it a problem! The install wants to do this thing, but I think this thing is bad, so I tell it to do another thing. Why does it not like doing things this other way! It also wants to do this thing, but I disapprove of this thing, so I tell it to do a different thing. Why does it not work!! Why is it so hard to do a thing after I've rejected every easy way to do it because I don't think it should be necessary??"
And yet, if you continue farther into the article, when he analyzes why this is all so unnecessarily hard on Unix systems, he's spot on. There is a hellish dependency quagmire because you have a bunch of different "ecosystems," from the OS libraries and the OS build tools to your RVM and bundler and rubygems, each dependent on the other and yet each playing by its own rules and making different (often technically very unnecessary) demands on the system.
The systemic problems that underlie all this are absolutely real and he nails them, and they're *why* the easiest solution to the "it works on my machine... and only my machine" dependency problem is Docker, which essentially says "it works on my machine, so.... here! Have a copy of my machine to run it on, and run the whole machine as if it were a single app!"
It's a way of admitting defeat, of saying it's so impossible to modularize your shit and manage its dependencies that the only way to install it is to emulate the entire machine that it was designed to run on.
And it really does solve problems -- if you're running a whole virtual machine with limited access to the underlying machine, it doesn't *matter* how much shit runs as root in the virtual machine. It's just virtual root. It doesn't *matter* how fragile the dependencies in the virtual machine are, they exist independently of your real machine and its messy real world. It's a crazily extreme solution to a ridiculous problem, but it actually is a solution.
It's just pathetic that it's necessary.
It's like if a novelist were trying to write a story set in a canonical universe like Star Trek or something like that. But the universe is so badly put together and full of contradictions after these many years that it's impossible to write anything longer than a chapter without hitting a bunch of continuity errors. So instead, the author writes a story *about a novelist in that universe* writing a story, and it's OK if the novelist in that universe writes a story which contradicts a bunch of the canon, because the canon doesn't apply to it, because it's not a novel set in that universe, it's a *novel about a novel set in that universe*.
It's insane that anybody should ever have to do that, but it does solve the problem.
posted by edheil at 6:22 AM on October 1, 2015 [48 favorites]
So his review of every Linux distribution was entirely concerned, not with how to use the distro the way the developers wanted to, but on how easy it was to compile bleeding-edge KDE with bleeding-edge font anti-aliasing and keep it all in /opt. That was how distributions stood or fell.
The first part of this article kind of read like that. "There's a standard, easy way to do this, but my environment isn't compatible with it so I'm going to declare that other ways of installing it should be just as easy! My system has a weird mix of a 64 bit kernel and 32 bit userspace. That shouldn't be a problem but it is! Why is it a problem! The install wants to do this thing, but I think this thing is bad, so I tell it to do another thing. Why does it not like doing things this other way! It also wants to do this thing, but I disapprove of this thing, so I tell it to do a different thing. Why does it not work!! Why is it so hard to do a thing after I've rejected every easy way to do it because I don't think it should be necessary??"
And yet, if you continue farther into the article, when he analyzes why this is all so unnecessarily hard on Unix systems, he's spot on. There is a hellish dependency quagmire because you have a bunch of different "ecosystems," from the OS libraries and the OS build tools to your RVM and bundler and rubygems, each dependent on the other and yet each playing by its own rules and making different (often technically very unnecessary) demands on the system.
The systemic problems that underlie all this are absolutely real and he nails them, and they're *why* the easiest solution to the "it works on my machine... and only my machine" dependency problem is Docker, which essentially says "it works on my machine, so.... here! Have a copy of my machine to run it on, and run the whole machine as if it were a single app!"
It's a way of admitting defeat, of saying it's so impossible to modularize your shit and manage its dependencies that the only way to install it is to emulate the entire machine that it was designed to run on.
And it really does solve problems -- if you're running a whole virtual machine with limited access to the underlying machine, it doesn't *matter* how much shit runs as root in the virtual machine. It's just virtual root. It doesn't *matter* how fragile the dependencies in the virtual machine are, they exist independently of your real machine and its messy real world. It's a crazily extreme solution to a ridiculous problem, but it actually is a solution.
It's just pathetic that it's necessary.
It's like if a novelist were trying to write a story set in a canonical universe like Star Trek or something like that. But the universe is so badly put together and full of contradictions after these many years that it's impossible to write anything longer than a chapter without hitting a bunch of continuity errors. So instead, the author writes a story *about a novelist in that universe* writing a story, and it's OK if the novelist in that universe writes a story which contradicts a bunch of the canon, because the canon doesn't apply to it, because it's not a novel set in that universe, it's a *novel about a novel set in that universe*.
It's insane that anybody should ever have to do that, but it does solve the problem.
posted by edheil at 6:22 AM on October 1, 2015 [48 favorites]
What I want to see more of:
People re-implementing tools / functionality if and when they understand the predecessor and its capabilities and have a cited paper that describes the basis for their proposed improvement.
Total elimination of all things global. Everything must be scoped and local. Nothing that requires install as or run by a specific user, no files that must be installed at the top level of the file system, or even a specific fixed location (this includes all binaries and the package manager itself) and no sharing of artifacts of any sort between projects (including the build tool binary itself) unless they explicitly opt in.
Stop making tools that are too simple to cover the domain of the problem they solve. Certain problems are complex enough that a tool that properly addresses them must cover that requisite complexity. If you can't figure out the tool, you don't understand the problem, and any turn-key simplification of the solution will cause as many problems as it solves.
posted by idiopath at 6:23 AM on October 1, 2015 [1 favorite]
People re-implementing tools / functionality if and when they understand the predecessor and its capabilities and have a cited paper that describes the basis for their proposed improvement.
Total elimination of all things global. Everything must be scoped and local. Nothing that requires install as or run by a specific user, no files that must be installed at the top level of the file system, or even a specific fixed location (this includes all binaries and the package manager itself) and no sharing of artifacts of any sort between projects (including the build tool binary itself) unless they explicitly opt in.
Stop making tools that are too simple to cover the domain of the problem they solve. Certain problems are complex enough that a tool that properly addresses them must cover that requisite complexity. If you can't figure out the tool, you don't understand the problem, and any turn-key simplification of the solution will cause as many problems as it solves.
posted by idiopath at 6:23 AM on October 1, 2015 [1 favorite]
Well, finding yourself out of the standard configuration seems to be the big problem with a lot of these install methods - you get some line or lines of command line gibberish to run and it either works or it doesn't, and if it works that's great and if it doesn't then you're plunged into an ocean of opaque complications.
posted by Artw at 6:23 AM on October 1, 2015
posted by Artw at 6:23 AM on October 1, 2015
Python. Django. Django documentation. Pip. VIrtualenv. Not perfect, but it mostly works.
posted by signal at 6:25 AM on October 1, 2015 [20 favorites]
posted by signal at 6:25 AM on October 1, 2015 [20 favorites]
Honestly, for front end stuff a lot of my time I find myself pining for the days where you'd just unzip some files and plonk them in a directory.
posted by Artw at 6:26 AM on October 1, 2015 [5 favorites]
posted by Artw at 6:26 AM on October 1, 2015 [5 favorites]
The answer is to just work until your body falls apart from the stress lol.
posted by Annika Cicada at 6:29 AM on October 1, 2015 [7 favorites]
posted by Annika Cicada at 6:29 AM on October 1, 2015 [7 favorites]
Metafilter: then you're plunged into an ocean of opaque complications
posted by Multicellular Exothermic at 6:29 AM on October 1, 2015 [10 favorites]
posted by Multicellular Exothermic at 6:29 AM on October 1, 2015 [10 favorites]
If you are an application developer, and your application does not install properly and easily on a supported system, you have failed.
Why is it that this whole product can't be installed using apt-get? There are stable, standard frameworks for application procurement that don't need to be re-invented.
posted by grumpybear69 at 6:31 AM on October 1, 2015 [4 favorites]
Why is it that this whole product can't be installed using apt-get? There are stable, standard frameworks for application procurement that don't need to be re-invented.
posted by grumpybear69 at 6:31 AM on October 1, 2015 [4 favorites]
Docker is a valiant attempt to paper over all of this
The way to fix leaky abstractions is *not* to put another layer of abstraction on top.
This "ports < 1024 must be root" rule is ancient and should be deprecated, though.
Yeah, but too many OSes have that wedged in. The reason you follow it is the same reason you write your system shell scripts in /bin/sh, not other shells. You *know* /bin/sh is there and exactly how it works. Thus, when you move the script to another Unix system, it works. If you're using /usr/bin/ksh, it breaks when they have ksh-88 and you wrote it for ksh-93.
You'd want better tools? You have to bring them with you.
and he really really wanted it installed in /opt, not /usr or /usr/local.
This is a classic Unix problem. The smart people want to keep distro and added code apart, but can't agree where. The dumb people dump in into /usr and hope it doesn't break something. And everybody basically chooses the answer their first Unix variant had. Solaris guy doesn't like the BSD or Linux location.
posted by eriko at 6:31 AM on October 1, 2015 [5 favorites]
The way to fix leaky abstractions is *not* to put another layer of abstraction on top.
This "ports < 1024 must be root" rule is ancient and should be deprecated, though.
Yeah, but too many OSes have that wedged in. The reason you follow it is the same reason you write your system shell scripts in /bin/sh, not other shells. You *know* /bin/sh is there and exactly how it works. Thus, when you move the script to another Unix system, it works. If you're using /usr/bin/ksh, it breaks when they have ksh-88 and you wrote it for ksh-93.
You'd want better tools? You have to bring them with you.
and he really really wanted it installed in /opt, not /usr or /usr/local.
This is a classic Unix problem. The smart people want to keep distro and added code apart, but can't agree where. The dumb people dump in into /usr and hope it doesn't break something. And everybody basically chooses the answer their first Unix variant had. Solaris guy doesn't like the BSD or Linux location.
posted by eriko at 6:31 AM on October 1, 2015 [5 favorites]
I had a look at Ruby last year, on Linux. I like the language itself a lot, but Ruby is Rails and Rails is Ruby, and thus will it ever be. What I found jarring is that, going by the book, there's so many little DSLs to learn with Rails (Rails, the test framework, the packaging management) .
posted by GallonOfAlan at 6:33 AM on October 1, 2015 [2 favorites]
posted by GallonOfAlan at 6:33 AM on October 1, 2015 [2 favorites]
The systemic problems that underlie all this are absolutely real and he nails them, and they're *why* the easiest solution to the "it works on my machine... and only my machine" dependency problem is Docker, which essentially says "it works on my machine, so.... here! Have a copy of my machine to run it on, and run the whole machine as if it were a single app!"
And you patch this how? See, in the real world, we have to close those security holes. How do I patch that entire machine you've brought with?
This is not a glib statement. This is *absolutely* a requirement. I know that's a bad word in development, but there it is. If I can't patch it when the holes are found, the system is broken and can't be used.
If you elect to send a whole VM with, guess what? YOU are now responsible for patching that whole VM!
And you thought you were saving work? You now need to deal with system patches, then regression testing, and you need to do this fast when there's a zero-day running about.
If you're going to abstract the OS away, fine -- but you need to manage your abstraction.
posted by eriko at 6:36 AM on October 1, 2015 [18 favorites]
And you patch this how? See, in the real world, we have to close those security holes. How do I patch that entire machine you've brought with?
This is not a glib statement. This is *absolutely* a requirement. I know that's a bad word in development, but there it is. If I can't patch it when the holes are found, the system is broken and can't be used.
If you elect to send a whole VM with, guess what? YOU are now responsible for patching that whole VM!
And you thought you were saving work? You now need to deal with system patches, then regression testing, and you need to do this fast when there's a zero-day running about.
If you're going to abstract the OS away, fine -- but you need to manage your abstraction.
posted by eriko at 6:36 AM on October 1, 2015 [18 favorites]
> It's a crazily extreme solution to a ridiculous problem, but it actually is a solution.
I am presently running 17 different flavors of Linux (not all at once.) It's my ambition to have as many as there are flavors of Bertie Bott's Every Flavor Beans, includine Earwax and Booger. Every one of these runs in a VM. I won't say what the underlying host OS is, but it comes from Redmond.
posted by jfuller at 6:36 AM on October 1, 2015 [6 favorites]
I am presently running 17 different flavors of Linux (not all at once.) It's my ambition to have as many as there are flavors of Bertie Bott's Every Flavor Beans, includine Earwax and Booger. Every one of these runs in a VM. I won't say what the underlying host OS is, but it comes from Redmond.
posted by jfuller at 6:36 AM on October 1, 2015 [6 favorites]
disclaimer: rubyist
headline: user with nonstandard runtime system expects software stack with moderate complexity to execute a perfect 3 point landing on his special snowflake.
1) installers are not magical mind-reading helpful bits of artificial intelligence that can assay the entire surface of whatever habitat you land them on. they make a fair number of assumptions up-front to minimize the complexity/logic/failure modes.
2) docker exists for a few reasons, but one of them is to enforce a ceiling and a floor for environmental dependencies. most docker engines are happiest when running without much of an OS at all. installing a docker engine onto your snowflaked personal dev box or long-running carefully hand-crafted server is a form of impedance mismatch.
3) handcrafted dependency management should only be undertaken when you are intimately familiar with how those dependencies could hook together. if you've got a gem that needs to be statically linked, is possibly being compiled 64 bit, but only has 32bit libs available, yeah, you might run into problems. bundler makes a few things more convenient, but again, if you're hand-jamming dependencies, you at least need to head in with an expectation that you might have to drop down to gem or extconf.
if all these things are unacceptable, don't go for software that was crafted in the recent present. it was probably made with things from the recent present that expect to be in an environment from the recent present.
stick with what you can install from the OS, a package repo, or a language runtime environment that is familiar to you.
posted by rye bread at 6:40 AM on October 1, 2015 [5 favorites]
headline: user with nonstandard runtime system expects software stack with moderate complexity to execute a perfect 3 point landing on his special snowflake.
1) installers are not magical mind-reading helpful bits of artificial intelligence that can assay the entire surface of whatever habitat you land them on. they make a fair number of assumptions up-front to minimize the complexity/logic/failure modes.
2) docker exists for a few reasons, but one of them is to enforce a ceiling and a floor for environmental dependencies. most docker engines are happiest when running without much of an OS at all. installing a docker engine onto your snowflaked personal dev box or long-running carefully hand-crafted server is a form of impedance mismatch.
3) handcrafted dependency management should only be undertaken when you are intimately familiar with how those dependencies could hook together. if you've got a gem that needs to be statically linked, is possibly being compiled 64 bit, but only has 32bit libs available, yeah, you might run into problems. bundler makes a few things more convenient, but again, if you're hand-jamming dependencies, you at least need to head in with an expectation that you might have to drop down to gem or extconf.
if all these things are unacceptable, don't go for software that was crafted in the recent present. it was probably made with things from the recent present that expect to be in an environment from the recent present.
stick with what you can install from the OS, a package repo, or a language runtime environment that is familiar to you.
posted by rye bread at 6:40 AM on October 1, 2015 [5 favorites]
I'm not a developer, but I work with a large team of web developers at a "Saas" shop with clients who are often both extremely demanding and extremely not-technical in even the barest sense.
The number one biggest issue we have when it comes to deploying solutions is lack of process documentation. In fact it's pretty much the reason I was hired, to help our little rapidly growing firm get a handle on this one issue.
We have expert devs who really know how to do great things. We even have a few folks who probably can "Genius" their way out of a problem...but the real issue is that 95% of these very skilled coders do not have the skill set, time, nor inclination to craft and document processes. It's painful and tedious and it takes a fair amount of time to get it right, and new tools are NOT the answer.
Whenever people ask me what "tool" we should use to do solve this problem I always reply: "notepad." And when they balk at that : "I don't give a shit about the tool. If we can't all agree to to follow the same steps every time the tool won't do shit for us."
posted by Doleful Creature at 6:43 AM on October 1, 2015 [9 favorites]
The number one biggest issue we have when it comes to deploying solutions is lack of process documentation. In fact it's pretty much the reason I was hired, to help our little rapidly growing firm get a handle on this one issue.
We have expert devs who really know how to do great things. We even have a few folks who probably can "Genius" their way out of a problem...but the real issue is that 95% of these very skilled coders do not have the skill set, time, nor inclination to craft and document processes. It's painful and tedious and it takes a fair amount of time to get it right, and new tools are NOT the answer.
Whenever people ask me what "tool" we should use to do solve this problem I always reply: "notepad." And when they balk at that : "I don't give a shit about the tool. If we can't all agree to to follow the same steps every time the tool won't do shit for us."
posted by Doleful Creature at 6:43 AM on October 1, 2015 [9 favorites]
" And you patch this how? See, in the real world, we have to close those security holes. How do I patch that entire machine you've brought with? "
This is true, and yet, the VM also helps solve the "oh shit it's full of security holes" problem, in a stupid and brute-force way, because it's easier to restrict what resources a VM has access to, in terms of storage, CPU, networking, and so on, than it is to restrict what resources an application has access to within a machine. You can make sure the VM can only, ever, at all, access a single directory of real-world storage, and it can't execute any real-world programs at all ever, and so on and so forth. So the unpatched crap inside the VM can do less damage to the real world than the same unpatched crap running on the host OS.
At least I think that's how it goes. I'm pretty far out of my personal experience here and could be off base.
posted by edheil at 6:45 AM on October 1, 2015 [3 favorites]
This is true, and yet, the VM also helps solve the "oh shit it's full of security holes" problem, in a stupid and brute-force way, because it's easier to restrict what resources a VM has access to, in terms of storage, CPU, networking, and so on, than it is to restrict what resources an application has access to within a machine. You can make sure the VM can only, ever, at all, access a single directory of real-world storage, and it can't execute any real-world programs at all ever, and so on and so forth. So the unpatched crap inside the VM can do less damage to the real world than the same unpatched crap running on the host OS.
At least I think that's how it goes. I'm pretty far out of my personal experience here and could be off base.
posted by edheil at 6:45 AM on October 1, 2015 [3 favorites]
Reading the article as a Pythonista and a Django developer, I realize that I might be pretty spoiled. Our technology stack isn't sexy (NoSQL? What NoSQL?), but it is reliable and developed/maintained by some very clueful people. virtualenv is a godsend and pip actually makes installing packages and their dependencies mostly Just Work. Setting up web stacks from scratch is a breeze, and the documentation is high quality all around. (Throw in Ansible, LTS distros and some Linode boxes and it's spectacularly easy and fast.)
I haven't been following the Ruby/Rails scene for a long time now, but I assumed they would have gotten their act together by now. Perhaps not, then? Possible technological failings aside, is it still the scene of choice for tech misogynists and obnoxious brogrammer types, or have they moved on to some other new hotness yet?
posted by jklaiho at 6:47 AM on October 1, 2015 [4 favorites]
I haven't been following the Ruby/Rails scene for a long time now, but I assumed they would have gotten their act together by now. Perhaps not, then? Possible technological failings aside, is it still the scene of choice for tech misogynists and obnoxious brogrammer types, or have they moved on to some other new hotness yet?
posted by jklaiho at 6:47 AM on October 1, 2015 [4 favorites]
"this site has been up for years, has withstood 3 php version bumps and 2 Web server upgrades, and performs near native. Is your demo site deployed yet?"
This is the bottom line. The implementation which actually solves a real-world problem is infinitely superior to the one that's ideologically pure but never actually makes it to production. That doesn't mean that some production systems aren't clusterfucks—many of them are—but at least they work.
I totally understand this developer's frustration. I totally understand the urge to build clean, transparent systems. And I do everything I can to build systems that are like that. But I'm slowly, grudgingly starting to accept that the economic and logistical realities of most software development work just don't allow for the degree of purity that developers would like. Aim for the Taj Mahal, but realize that you'll be lucky to get all of the load-bearing beams in the right place. Sometimes that means choosing components and designs for their ability to withstand those economic and logistical forces, rather than their purity.
I mean, google something like "print spooler won't start" and you'll have people confidently claiming all kinds of insane steps to take because "it worked for me when I had that problem."
Ah, I totally feel you, then. The fact that someone on Stack Overflow has no idea what they're talking about will never stop them from answering anyway. It used to be the same on message boards, before Stack Overflow came along. After a while you learn to smell those answers, but that doesn't help novice users. I still get bitten sometimes when I'm dealing with an unfamiliar package or platform—people give confident, detailed answers that turn out to be in flat contradiction of facts or sane practices.
posted by escape from the potato planet at 6:51 AM on October 1, 2015 [4 favorites]
This is the bottom line. The implementation which actually solves a real-world problem is infinitely superior to the one that's ideologically pure but never actually makes it to production. That doesn't mean that some production systems aren't clusterfucks—many of them are—but at least they work.
I totally understand this developer's frustration. I totally understand the urge to build clean, transparent systems. And I do everything I can to build systems that are like that. But I'm slowly, grudgingly starting to accept that the economic and logistical realities of most software development work just don't allow for the degree of purity that developers would like. Aim for the Taj Mahal, but realize that you'll be lucky to get all of the load-bearing beams in the right place. Sometimes that means choosing components and designs for their ability to withstand those economic and logistical forces, rather than their purity.
I mean, google something like "print spooler won't start" and you'll have people confidently claiming all kinds of insane steps to take because "it worked for me when I had that problem."
Ah, I totally feel you, then. The fact that someone on Stack Overflow has no idea what they're talking about will never stop them from answering anyway. It used to be the same on message boards, before Stack Overflow came along. After a while you learn to smell those answers, but that doesn't help novice users. I still get bitten sometimes when I'm dealing with an unfamiliar package or platform—people give confident, detailed answers that turn out to be in flat contradiction of facts or sane practices.
posted by escape from the potato planet at 6:51 AM on October 1, 2015 [4 favorites]
I've noticed this kind of cargo-cult approach to technology way more in the last few years, and it's really depressing.
As a user, I think this is correct and works across a whole spectrum of expectations consumers have for society.
On the other hand, two days ago my son and I were trying to figure out if our microwave is interfering with the wireless at our house. Three minutes of research and I downloaded and installed a free, working magnetometer to my Android phone to measure the magnetic field around the microwave. Then we had a talk about correlation and causation.
This cargo cult thing is working for me!
posted by sneebler at 7:01 AM on October 1, 2015
As a user, I think this is correct and works across a whole spectrum of expectations consumers have for society.
On the other hand, two days ago my son and I were trying to figure out if our microwave is interfering with the wireless at our house. Three minutes of research and I downloaded and installed a free, working magnetometer to my Android phone to measure the magnetic field around the microwave. Then we had a talk about correlation and causation.
This cargo cult thing is working for me!
posted by sneebler at 7:01 AM on October 1, 2015
(Oh, he's also correct that there's no reason in principle a set of web forums should require all of this magic. The complexity of ruby + gems + rails + the forum code + your own configuration code + dependencies + database extensions + etc + etc, isn't all there because the technical problem of "making web forums appear on the web" *inherently requires solutions that difficult*. In principle, it could all be accomplished with one giant executable binary written in C with no dependencies, if you could find somebody who could write it. Or more humbly, it could be one big ordinary PHP script running vanilla SQL on a vanilla database.
The complexity happens for the sake of *developers,* so that the tasks of making all this shit happen can be broken up and delegated to many different developers in different times and places, and it can be made as easy as possible for each layer of developers to use the code from the previous layer of developers, so the web forum authors don't have to think about how to write a request router, and the rails authors don't have to think about how to do whatever thing those compiled gems are doing, and the libraries are there so the gem authors don't have to think about whatever those libraries are doing, and so on back through the ages.
That's where all the complexity comes from: allowing developers to re-use other developers' work so they don't have to know how to create the universe from scratch.
The way he describes it, it sounds as if that process has reached a degenerate point where the system of modularization and re-use has begun to create more problems than it sounds.
This is an eternal problem in software development, BTW. "Should I write my own shit and spend a lot of effort to reinvent the wheel, or should I leverage some existing shit set up a possibly fragile dependency on a possibly unreliable external developer, whose agenda may or may not dovetail well with my own needs?" Either choice can lead you into a world of hurt.)
posted by edheil at 7:05 AM on October 1, 2015 [5 favorites]
The complexity happens for the sake of *developers,* so that the tasks of making all this shit happen can be broken up and delegated to many different developers in different times and places, and it can be made as easy as possible for each layer of developers to use the code from the previous layer of developers, so the web forum authors don't have to think about how to write a request router, and the rails authors don't have to think about how to do whatever thing those compiled gems are doing, and the libraries are there so the gem authors don't have to think about whatever those libraries are doing, and so on back through the ages.
That's where all the complexity comes from: allowing developers to re-use other developers' work so they don't have to know how to create the universe from scratch.
The way he describes it, it sounds as if that process has reached a degenerate point where the system of modularization and re-use has begun to create more problems than it sounds.
This is an eternal problem in software development, BTW. "Should I write my own shit and spend a lot of effort to reinvent the wheel, or should I leverage some existing shit set up a possibly fragile dependency on a possibly unreliable external developer, whose agenda may or may not dovetail well with my own needs?" Either choice can lead you into a world of hurt.)
posted by edheil at 7:05 AM on October 1, 2015 [5 favorites]
signal: "Python. Django. Django documentation. Pip. VIrtualenv. Not perfect, but it mostly works."
Totally my favorite web stack. Now if I could only find an employer to pay me to work in it.
posted by octothorpe at 7:07 AM on October 1, 2015 [11 favorites]
Totally my favorite web stack. Now if I could only find an employer to pay me to work in it.
posted by octothorpe at 7:07 AM on October 1, 2015 [11 favorites]
This is why, despite it being a turgid mass of security holes, I'm still basing my web site on Wordpress. Local dev copy, ftp those changes up to the host. Life's too short for wrangling anything more complicated than "dump a bunch of PHP on the host" unless that's the entire point of your existence.
posted by egypturnash at 7:08 AM on October 1, 2015 [1 favorite]
posted by egypturnash at 7:08 AM on October 1, 2015 [1 favorite]
I'll admit that the website that I administer for our neighborhood is a Wordpress site despite my distaste for PHP. Dreamhost's admin tools handle all of the install and upgrade issues and the thing just keeps working.
posted by octothorpe at 7:19 AM on October 1, 2015
posted by octothorpe at 7:19 AM on October 1, 2015
At least I think that's how it goes. I'm pretty far out of my personal experience here and could be off base.
VMs also let you consolidate servers when the services are running on NT 4 and can't be moved to a more modern OS or would conflict if another service was sharing the machine. Windows admins "love" actual VMs because you can buy nice new hardware and just teleport your cranky 15 year-old highly customized in-house server into it along with several others.
At least that was the state of the art for virtualized consolidation in 2007.
Docker still makes crazy assumptions like having source code to the apps or them running on more than one version of linux.
posted by GuyZero at 7:26 AM on October 1, 2015 [1 favorite]
VMs also let you consolidate servers when the services are running on NT 4 and can't be moved to a more modern OS or would conflict if another service was sharing the machine. Windows admins "love" actual VMs because you can buy nice new hardware and just teleport your cranky 15 year-old highly customized in-house server into it along with several others.
At least that was the state of the art for virtualized consolidation in 2007.
Docker still makes crazy assumptions like having source code to the apps or them running on more than one version of linux.
posted by GuyZero at 7:26 AM on October 1, 2015 [1 favorite]
Here's a metaphor for how I see it:
We all have this hammer that we're expected to bang on rocks and expect fully formed, polished and usable coins to fall out.
banging on rocks part is step 1. Then we need to build the sluice, the smelt, and the dies and the machines to then cast our coins.
But no, we have his idea that the we can just bang the package on a box and with a little fiddling expect a fully working mint (aka your delivered product) to magically fall out.
Any attempts to build all the parts of the process that are required to actually ship a product are considered technical debt and we are demanded to feel bad about that time that it takes to actually get everything set up the way it should, because our bosses demand that we deliver them a fully functioning gold mine with just a hammer.
It's absurd to me because we then talk about all the versions of all the hammers that are out there and how with just a little fiddling things aren't too terribly bad and meanwhile we are basically still banging on rocks and hoping for the best.
We need to push up and demand the time required to build the tools it takes to get the entire structures built the right way so we can deliver products without the heroics involved.
posted by Annika Cicada at 7:36 AM on October 1, 2015 [6 favorites]
We all have this hammer that we're expected to bang on rocks and expect fully formed, polished and usable coins to fall out.
banging on rocks part is step 1. Then we need to build the sluice, the smelt, and the dies and the machines to then cast our coins.
But no, we have his idea that the we can just bang the package on a box and with a little fiddling expect a fully working mint (aka your delivered product) to magically fall out.
Any attempts to build all the parts of the process that are required to actually ship a product are considered technical debt and we are demanded to feel bad about that time that it takes to actually get everything set up the way it should, because our bosses demand that we deliver them a fully functioning gold mine with just a hammer.
It's absurd to me because we then talk about all the versions of all the hammers that are out there and how with just a little fiddling things aren't too terribly bad and meanwhile we are basically still banging on rocks and hoping for the best.
We need to push up and demand the time required to build the tools it takes to get the entire structures built the right way so we can deliver products without the heroics involved.
posted by Annika Cicada at 7:36 AM on October 1, 2015 [6 favorites]
(but if anyone wants to talk shop about how to securely patch several thousand docker instances via puppet memail me, that's the current problem that keeps me awake at night and I'm out of ideas)
posted by Annika Cicada at 7:43 AM on October 1, 2015 [1 favorite]
posted by Annika Cicada at 7:43 AM on October 1, 2015 [1 favorite]
Docker is last weeks news! It's all Webpack now!
You joke, but this has become one of the fundamental problems of software development: the ratio of doing-to-learning is incredibly low. This is the problem with the endless reinventing of the wheel and proliferation of multiple frameworks: unless you are the decision maker about which one to use, developers find themselves having to learn brand new frameworks at each new job and deal with additional multitudes of installation and development quirks and roadblocks inherent to each one. And the skills and understanding you gain have very little transferability to the next project using a different framework.
This post about getting docker running is just a symptom of the overall problem.
posted by deanc at 7:51 AM on October 1, 2015 [4 favorites]
You joke, but this has become one of the fundamental problems of software development: the ratio of doing-to-learning is incredibly low. This is the problem with the endless reinventing of the wheel and proliferation of multiple frameworks: unless you are the decision maker about which one to use, developers find themselves having to learn brand new frameworks at each new job and deal with additional multitudes of installation and development quirks and roadblocks inherent to each one. And the skills and understanding you gain have very little transferability to the next project using a different framework.
This post about getting docker running is just a symptom of the overall problem.
posted by deanc at 7:51 AM on October 1, 2015 [4 favorites]
You joke, but this has become one of the fundamental problems of software development: the ratio of doing-to-learning is incredibly low.
The gap between what actual SREs do at work and what the nice developer relations people go to conferences about seems to get wider by the day.
posted by GuyZero at 7:57 AM on October 1, 2015 [4 favorites]
The gap between what actual SREs do at work and what the nice developer relations people go to conferences about seems to get wider by the day.
posted by GuyZero at 7:57 AM on October 1, 2015 [4 favorites]
Rhymes with "piss horse"??
posted by boo_radley at 7:59 AM on October 1, 2015 [3 favorites]
posted by boo_radley at 7:59 AM on October 1, 2015 [3 favorites]
Salesforce?
posted by boo_radley at 8:02 AM on October 1, 2015 [3 favorites]
posted by boo_radley at 8:02 AM on October 1, 2015 [3 favorites]
Rhymes with "piss horse"??
It's "Discourse." I hadn't heard of it either, but figured it out from a link down the page.
posted by justkevin at 8:05 AM on October 1, 2015 [2 favorites]
It's "Discourse." I hadn't heard of it either, but figured it out from a link down the page.
posted by justkevin at 8:05 AM on October 1, 2015 [2 favorites]
Open source Web code isn't messy because it's written by volunteers; it's now written (mostly) by salaried employees, and then open-sourced in service of their employer's pr, hr, and engineering departments (in that order). If it's poorly documented it's because those devs, at that company, probably talked to each other and helped each other and all had a similar build environment, so why would they spend cycles writing docs that would distract them from coding.
And it's hard to install because the devs were not told "make it easy to install on random machines" but instead "make it easy to install on 100 - 10000 roughly identical machines."
posted by zippy at 8:05 AM on October 1, 2015 [2 favorites]
And it's hard to install because the devs were not told "make it easy to install on random machines" but instead "make it easy to install on 100 - 10000 roughly identical machines."
posted by zippy at 8:05 AM on October 1, 2015 [2 favorites]
I think this guy's problem is also a lot to do with trying to work within a shared hosting environment where he is fundamentally out of control of key parts of the system.
He's trying to argue that the onus is on developers of the web forum to make it work, but they have even less control over his system than he does, and cannot anticipate the issues that may come up.
I have had a small website on a shared hosting server for ages, and got tripped up at one point when I realised they ran Apache + BenSSL instead of Apache + OpenSSL and that was just one of a million little things that has made me think that trying to replicate custom environments is pointless, and the mantra of "cattle not pets" around servers and the trend towards containers is really a pragmatic response to dealing with this stuff.
But it means the best you can do for shared hosting is VMs or docker containers or something else where you exist in an entirely controllable environment.
posted by pulposus at 8:10 AM on October 1, 2015 [4 favorites]
He's trying to argue that the onus is on developers of the web forum to make it work, but they have even less control over his system than he does, and cannot anticipate the issues that may come up.
I have had a small website on a shared hosting server for ages, and got tripped up at one point when I realised they ran Apache + BenSSL instead of Apache + OpenSSL and that was just one of a million little things that has made me think that trying to replicate custom environments is pointless, and the mantra of "cattle not pets" around servers and the trend towards containers is really a pragmatic response to dealing with this stuff.
But it means the best you can do for shared hosting is VMs or docker containers or something else where you exist in an entirely controllable environment.
posted by pulposus at 8:10 AM on October 1, 2015 [4 favorites]
To be honest, I think the problem is "this should be easy." You want a web forum. Seems simple. You can say it in one sentence. "I want a web forum." But it is incredibly complex. Before you even think about the actual logic, you need a database, a web server for static assets, an application server, possibly a caching system. And you need some kind of middleware or abstraction layer so that people can run your application with their preferred web server and database (or at least their preferred versions) and on different operating systems and what have you. And each of these choices imposes limitations on the other elements of the system. Not only that, but you need to think about how the system will scale or else it will be useless to the people who might actually pay you for your work, and that means a whole additional layer of complexity and limitations. Before you write a single line of code you have a fairly complicated install process that requires several external dependencies of specific versions, now all of which you can test.
But then, and this is the real kicker, most of those dependencies were written by people similarly unprepared for the complexity of what they were doing. Or worse, under deadlines where a project manager was telling them "it's just a web server, you get a request and return a page, we have 3 months, what's the problem?"
posted by Nothing at 8:15 AM on October 1, 2015 [7 favorites]
But then, and this is the real kicker, most of those dependencies were written by people similarly unprepared for the complexity of what they were doing. Or worse, under deadlines where a project manager was telling them "it's just a web server, you get a request and return a page, we have 3 months, what's the problem?"
posted by Nothing at 8:15 AM on October 1, 2015 [7 favorites]
OCaml to the rescue?
Or maybe we should all be running Oberon.
GNU/Linux is a real problem though; that stuff is just way too complex. For instance, /bin/sh *actually means* /bin/bash, although Debian's working on that one. The kernel's way too large, but the alternative is a BSD where the TCP code is descended from the stuff Bill Joy wrote in the early 80s.
The author was being silly; he's essentially complaining that he can't fit a big-block Chevy in a heavily customized Dodge without fabricating motor mounts and bending a bunch of sheetmetal.
posted by Monday, stony Monday at 8:15 AM on October 1, 2015 [2 favorites]
Or maybe we should all be running Oberon.
GNU/Linux is a real problem though; that stuff is just way too complex. For instance, /bin/sh *actually means* /bin/bash, although Debian's working on that one. The kernel's way too large, but the alternative is a BSD where the TCP code is descended from the stuff Bill Joy wrote in the early 80s.
The author was being silly; he's essentially complaining that he can't fit a big-block Chevy in a heavily customized Dodge without fabricating motor mounts and bending a bunch of sheetmetal.
posted by Monday, stony Monday at 8:15 AM on October 1, 2015 [2 favorites]
No way Salesforce runs on anything invented after 1998. That's enterprise, the other slower moving kind of hell.
posted by Artw at 8:20 AM on October 1, 2015 [3 favorites]
posted by Artw at 8:20 AM on October 1, 2015 [3 favorites]
An example of how things *should* work: Redis is extremely easy to install with make and gcc, and has no external dependencies, not even autoconf. The quick start is a couple of pages and takes you through build, command-line testing, and permanent install. There's just a single config file to tweak.
Compare this to the Docker image, where everyone is complaining about their preferred configuration not being supported, or how they found a better/smaller image, and there's a plea from the maintainers to not fork their image.
This is a cherry-picked example, but just demonstrates that building from source and configuring without an image doesn't always have to hurt.
posted by RobotVoodooPower at 8:20 AM on October 1, 2015 [2 favorites]
Compare this to the Docker image, where everyone is complaining about their preferred configuration not being supported, or how they found a better/smaller image, and there's a plea from the maintainers to not fork their image.
This is a cherry-picked example, but just demonstrates that building from source and configuring without an image doesn't always have to hurt.
posted by RobotVoodooPower at 8:20 AM on October 1, 2015 [2 favorites]
I will however call it Piss Horse from now on.
posted by Artw at 8:21 AM on October 1, 2015 [3 favorites]
posted by Artw at 8:21 AM on October 1, 2015 [3 favorites]
+++1 EXCELLENT RANT WOULD LISTEN AGAIN
posted by quonsar II: smock fishpants and the temple of foon at 8:30 AM on October 1, 2015 [5 favorites]
posted by quonsar II: smock fishpants and the temple of foon at 8:30 AM on October 1, 2015 [5 favorites]
It actually didn't occur to me that the real name for what they were talking about wasn't "Piss Horse".
posted by octothorpe at 8:35 AM on October 1, 2015 [5 favorites]
posted by octothorpe at 8:35 AM on October 1, 2015 [5 favorites]
Meanwhile, in Java land, looks like you can be assured of nothing much changing ever: Oracle has lost interest in Java
They want to be cloud competitors with Piss Horse instead.
posted by Artw at 8:49 AM on October 1, 2015 [1 favorite]
They want to be cloud competitors with Piss Horse instead.
posted by Artw at 8:49 AM on October 1, 2015 [1 favorite]
Ah, signal, I believe you about Django and virtualenv but am currently unable to install them on a system with Enthought and venv.
posted by clew at 8:58 AM on October 1, 2015
posted by clew at 8:58 AM on October 1, 2015
Python. Django. Django documentation. Pip. VIrtualenv. Not perfect, but it mostly works.
This. We're moving to managing servers with Chef, and for some reason it's written in Ruby. I'm most familiar with Django applications, and a middling of Enterprise Java. I needed to write a template for a config file, and so decided to look up the syntax & documentation. Apparently instead of a variety of template languages, Ruby has one that everyone uses. And it is poorly documented. Which might be reasonable, since it has practically no features. It's more or less the ruby version of PHP.
We have a body of student developers primarily writing Python code. They seem to really hate Django; as best I can tell, new developers believe the fastest, best way to understand code is to have written it yourself. So the more features Django offers, the less they like it and it's feature set, and the more Flask projects we end up with. Which take longer, have fewer features, and are harder to manage.
posted by pwnguin at 9:15 AM on October 1, 2015 [2 favorites]
This. We're moving to managing servers with Chef, and for some reason it's written in Ruby. I'm most familiar with Django applications, and a middling of Enterprise Java. I needed to write a template for a config file, and so decided to look up the syntax & documentation. Apparently instead of a variety of template languages, Ruby has one that everyone uses. And it is poorly documented. Which might be reasonable, since it has practically no features. It's more or less the ruby version of PHP.
We have a body of student developers primarily writing Python code. They seem to really hate Django; as best I can tell, new developers believe the fastest, best way to understand code is to have written it yourself. So the more features Django offers, the less they like it and it's feature set, and the more Flask projects we end up with. Which take longer, have fewer features, and are harder to manage.
posted by pwnguin at 9:15 AM on October 1, 2015 [2 favorites]
Related: Choose Boring Technology:
posted by pwnguin at 9:17 AM on October 1, 2015 [19 favorites]
every company gets about three innovation tokens. You can spend these however you want, but the supply is fixed for a long while. You might get a few more after you achieve a certain level of stability and maturity, but the general tendency is to overestimate the contents of your wallet.
posted by pwnguin at 9:17 AM on October 1, 2015 [19 favorites]
...Once the code runs on the happy path...
Ay Dios Mio! I want to erase that phrase from the entire lexicon. ElGuapo's golden hammer for beating project mangers over the head is inscribed: "The first 85% takes 15% of the effort, and the last 15% takes 85% of the effort."
posted by ElGuapo at 9:20 AM on October 1, 2015 [2 favorites]
Ay Dios Mio! I want to erase that phrase from the entire lexicon. ElGuapo's golden hammer for beating project mangers over the head is inscribed: "The first 85% takes 15% of the effort, and the last 15% takes 85% of the effort."
posted by ElGuapo at 9:20 AM on October 1, 2015 [2 favorites]
to be clear, fuck this guy
OMG I love that article , thanks pwnguin!
posted by Annika Cicada at 9:24 AM on October 1, 2015
OMG I love that article , thanks pwnguin!
posted by Annika Cicada at 9:24 AM on October 1, 2015
Apparently instead of a variety of template languages, Ruby has one that everyone uses.
Uh. So, ignoring HAML (since it's pretty tightly coupled to HTML), I see three template languages in my current ruby project. Erb, liquid, and heredoc-plus-interpolation.
My Chef is fairly limited, but I'd be shocked if you couldn't drop in another (ruby-based) templating language pretty easily. The Ruby world definitely has its favorite tools, but swapping in alternatives is a pretty common thing.
posted by spaceman_spiff at 9:29 AM on October 1, 2015 [1 favorite]
Uh. So, ignoring HAML (since it's pretty tightly coupled to HTML), I see three template languages in my current ruby project. Erb, liquid, and heredoc-plus-interpolation.
My Chef is fairly limited, but I'd be shocked if you couldn't drop in another (ruby-based) templating language pretty easily. The Ruby world definitely has its favorite tools, but swapping in alternatives is a pretty common thing.
posted by spaceman_spiff at 9:29 AM on October 1, 2015 [1 favorite]
Server-side on back is basically solved if you want a stable, run-almost-anywhere platform:
Spring + Java 7/8 + non-ORM JDBC-based DB mapping framework + (Postgres/DB2/Oracle, pick according to size and budget).
No, it's not sexy. And there are a ton of startup hipsters that will chime in and tell you how you should be using Node/Rails/some other "cool" platform. They'll tell you how that's "old" (as if age is inversely proportional to quality.)
But what they won't tell you is that the above will run, reliably, and easily on all major OSs and most minor ones. It has mature, feature-complete tooling for development and scales well from a one-man project to a project involving hundreds of developers. You can deploy it on a single-core VM on a PaaS or on IBM Big Iron and if used correctly, will work fine on those and anything in between with what are basically just minor config changes. You can handle a hundred requests a second or a hundred thousand without replacing any of those technologies.
So yeah, I'm gonna call the server-side portion a solved problem. 'cause it is. Mainly because it was written by adults with a reasonable attention span.
Now the front end... that's where the hipster magpies have gravitated.
The front end is a complete clusterfuck. There're a couple frameworks with very good ideas (Mithril gets the notion of controller->view->controller control flow right, as well as the separation of state, but it's still very much a small preproduction project). For most of them the execution -- particularly from the "big" ones is lacking. Way too much complexity for solving anything that's not the specific business use cases that the creators face. In other words: React is great for implementing Facebook + Instagram, but not so great for someone who doesn't have to deal with a huge development team and a massive codebase. And even if you do learn a front-end framework, it's old and "deprecated" within a year as the rest of the children move on to the new shiny.
And let's not even talk about design language and its associated ball of library mud...
I am so, so glad I work solely on the back end of things. Then again, I'm old and grumpy. ;)
posted by -1 at 9:30 AM on October 1, 2015 [12 favorites]
Spring + Java 7/8 + non-ORM JDBC-based DB mapping framework + (Postgres/DB2/Oracle, pick according to size and budget).
No, it's not sexy. And there are a ton of startup hipsters that will chime in and tell you how you should be using Node/Rails/some other "cool" platform. They'll tell you how that's "old" (as if age is inversely proportional to quality.)
But what they won't tell you is that the above will run, reliably, and easily on all major OSs and most minor ones. It has mature, feature-complete tooling for development and scales well from a one-man project to a project involving hundreds of developers. You can deploy it on a single-core VM on a PaaS or on IBM Big Iron and if used correctly, will work fine on those and anything in between with what are basically just minor config changes. You can handle a hundred requests a second or a hundred thousand without replacing any of those technologies.
So yeah, I'm gonna call the server-side portion a solved problem. 'cause it is. Mainly because it was written by adults with a reasonable attention span.
Now the front end... that's where the hipster magpies have gravitated.
The front end is a complete clusterfuck. There're a couple frameworks with very good ideas (Mithril gets the notion of controller->view->controller control flow right, as well as the separation of state, but it's still very much a small preproduction project). For most of them the execution -- particularly from the "big" ones is lacking. Way too much complexity for solving anything that's not the specific business use cases that the creators face. In other words: React is great for implementing Facebook + Instagram, but not so great for someone who doesn't have to deal with a huge development team and a massive codebase. And even if you do learn a front-end framework, it's old and "deprecated" within a year as the rest of the children move on to the new shiny.
And let's not even talk about design language and its associated ball of library mud...
I am so, so glad I work solely on the back end of things. Then again, I'm old and grumpy. ;)
posted by -1 at 9:30 AM on October 1, 2015 [12 favorites]
It's like if a novelist were trying to write a story set in a canonical universe like Star Trek or something like that. But the universe is so badly put together and full of contradictions after these many years that it's impossible to write anything longer than a chapter without hitting a bunch of continuity errors. So instead, the author writes a story *about a novelist in that universe* writing a story, and it's OK if the novelist in that universe writes a story which contradicts a bunch of the canon, because the canon doesn't apply to it, because it's not a novel set in that universe, it's a *novel about a novel set in that universe*
I love that metaphor about canon becoming unworkable.
posted by thelonius at 9:34 AM on October 1, 2015 [2 favorites]
I love that metaphor about canon becoming unworkable.
posted by thelonius at 9:34 AM on October 1, 2015 [2 favorites]
clew: "am currently unable to install them on a system with Enthought and venv."
You can't get django working? It's dead simple, has very few dependencies, is pure python, etc. Is your Pip installation working?
posted by signal at 9:35 AM on October 1, 2015
You can't get django working? It's dead simple, has very few dependencies, is pure python, etc. Is your Pip installation working?
posted by signal at 9:35 AM on October 1, 2015
non-ORM JDBC-based DB mapping framework
which means myBatis, and what else is comparable?
I'm not too big a believer in ORM myself........but our team seems to have lost pretty completely
posted by thelonius at 9:37 AM on October 1, 2015
which means myBatis, and what else is comparable?
I'm not too big a believer in ORM myself........but our team seems to have lost pretty completely
posted by thelonius at 9:37 AM on October 1, 2015
So the article pwnguin linked to is fantastic, particularly for its very last sentence:
*Not that this is a bar with which to measure importance of any saying, I'm just saying it's a really fine statement
posted by JoeXIII007 at 9:41 AM on October 1, 2015 [5 favorites]
Technology for its own sake is snake oil.Need a projector and a large building in a big city's wall to put this quote on... this is music to my ears and someone said it*.
*Not that this is a bar with which to measure importance of any saying, I'm just saying it's a really fine statement
posted by JoeXIII007 at 9:41 AM on October 1, 2015 [5 favorites]
RobotVoodooPower: "Docker is still basically running shell commands in a fancy chroot() jail."... except the door to the jail is wide open.
edheil: "it doesn't *matter* how much shit runs as root in the [Docker] virtual machine. It's just virtual root."No, it's actual root.
Was I the only one who read this, linked from the essay?
posted by brokkr at 9:43 AM on October 1, 2015 [3 favorites]
Spring + Java 7/8 + non-ORM JDBC-based DB mapping framework + (Postgres/DB2/Oracle, pick according to size and budget).
I love how I've been working in web development, largely backend, for like 5 years now, and I have experience with roughly 3 of those words.
posted by pan at 9:49 AM on October 1, 2015
I love how I've been working in web development, largely backend, for like 5 years now, and I have experience with roughly 3 of those words.
posted by pan at 9:49 AM on October 1, 2015
So, some background:
I've written a lot of PHP, but currently work at a Java/Scala shop and also work on a lot of front-end stuff.
Ruby (and its ecosystem) have always perplexed me. The language itself seems decently solid, but so much of the surrounding community and tooling seems.... inexplicably bad? Between the major incompatibilities introduced in minor releases, the difficulty of getting rbenv (or rvm, or whatever is fashionable right now) working on a Linux box, and the overall lackluster performance of the interpreter... I just don't know why you'd use it. (SASS in particular irks me for how bad the documentation is, and how painfully slow the native (Ruby) implementation is)
I understand the desire to move quickly, but I just can't imagine how you're supposed to use Ruby in a professional setting.
PHP, on the other hand, has more than its fair share of baggage. It's been around forever, and the early versions made some.... questionable decisions about architecture. However, the documentation is great, and backward compatibility has always been pretty decent. On the other hand, the governance structure of PHP has seemingly done everything in its power to strangle the language. New developers to the core were treated with hostility, there were a few major direction shifts that went nowhere, and bugs routinely stay open for years at a time.
Meanwhile, in spite of itself, PHP 5.4 started evolving into a pretty nice language. Much of the rationale for the old monolithic frameworks went away, and we started seeing lots of interesting experiments with microframeworks, and finally Laravel emerged as a really good framework that understood the failings of its predecessors, as well as the reasons that people were moving away from PHP in droves. [Interestingly, it emerged long after PHP became uncool.]
Laravel has great documentation, great community. Laravel still has lots of forward momentum, but also an understanding that people are building their businesses on the framework. In short, it's everything that you want in an open-source project.
I could also write extensively about it, but the JavaScript ecosystem has also started understanding the need for good governance. While there's still a little too much trend-thrashing for my taste, it's an awesome space to work in. The Node/IO.js debacle underscored the community's desire for effective governance, and... we won? It was actually pretty remarkable to see how the community managed to acknowledge that there was a problem, and actually worked through the issues to reach a conclusion that was satisfactory to virtually everyone involved.
Now that I'm writing Java, I'm increasingly concerned that Oracle appear to be more interested in litigating Java than they appear to be interested in furthering the language. Like the PHP core, they appear to be caught in an older paradigm of governance and management that is hostile to newcomers, and is leading to stagnation. (However, like PHP 5.4, Java 8 is a really solid improvement that makes it feel a lot more like a modern language. I enjoy developing Java far more than I thought I would.)
On the other hand, I don't even know what to think about Scala. So far, it's sounding a lot like a language for people who are too clever for their own good (eg. like Perl, you can write some very clever one-liners that are impossible to grok after the fact). Scala's community is always talking about how great Scala is.
posted by schmod at 9:51 AM on October 1, 2015 [6 favorites]
I've written a lot of PHP, but currently work at a Java/Scala shop and also work on a lot of front-end stuff.
Ruby (and its ecosystem) have always perplexed me. The language itself seems decently solid, but so much of the surrounding community and tooling seems.... inexplicably bad? Between the major incompatibilities introduced in minor releases, the difficulty of getting rbenv (or rvm, or whatever is fashionable right now) working on a Linux box, and the overall lackluster performance of the interpreter... I just don't know why you'd use it. (SASS in particular irks me for how bad the documentation is, and how painfully slow the native (Ruby) implementation is)
I understand the desire to move quickly, but I just can't imagine how you're supposed to use Ruby in a professional setting.
PHP, on the other hand, has more than its fair share of baggage. It's been around forever, and the early versions made some.... questionable decisions about architecture. However, the documentation is great, and backward compatibility has always been pretty decent. On the other hand, the governance structure of PHP has seemingly done everything in its power to strangle the language. New developers to the core were treated with hostility, there were a few major direction shifts that went nowhere, and bugs routinely stay open for years at a time.
Meanwhile, in spite of itself, PHP 5.4 started evolving into a pretty nice language. Much of the rationale for the old monolithic frameworks went away, and we started seeing lots of interesting experiments with microframeworks, and finally Laravel emerged as a really good framework that understood the failings of its predecessors, as well as the reasons that people were moving away from PHP in droves. [Interestingly, it emerged long after PHP became uncool.]
Laravel has great documentation, great community. Laravel still has lots of forward momentum, but also an understanding that people are building their businesses on the framework. In short, it's everything that you want in an open-source project.
I could also write extensively about it, but the JavaScript ecosystem has also started understanding the need for good governance. While there's still a little too much trend-thrashing for my taste, it's an awesome space to work in. The Node/IO.js debacle underscored the community's desire for effective governance, and... we won? It was actually pretty remarkable to see how the community managed to acknowledge that there was a problem, and actually worked through the issues to reach a conclusion that was satisfactory to virtually everyone involved.
Now that I'm writing Java, I'm increasingly concerned that Oracle appear to be more interested in litigating Java than they appear to be interested in furthering the language. Like the PHP core, they appear to be caught in an older paradigm of governance and management that is hostile to newcomers, and is leading to stagnation. (However, like PHP 5.4, Java 8 is a really solid improvement that makes it feel a lot more like a modern language. I enjoy developing Java far more than I thought I would.)
On the other hand, I don't even know what to think about Scala. So far, it's sounding a lot like a language for people who are too clever for their own good (eg. like Perl, you can write some very clever one-liners that are impossible to grok after the fact). Scala's community is always talking about how great Scala is.
posted by schmod at 9:51 AM on October 1, 2015 [6 favorites]
Note that the security problem mentioned with Docker is not that a process inside the docker container can break out. It is that a user on the host system with access to run docker can get access to host resources that their user should not be able to access. This is a vulnerability, but it is a different vulnerability.
posted by Nothing at 9:54 AM on October 1, 2015 [1 favorite]
posted by Nothing at 9:54 AM on October 1, 2015 [1 favorite]
The next step was [XXX] which is usually a breeze in [YYY], by which I mean I have never once had it actually work without descending into a hellish nightmare and this time was no exception.
This description applies to so much of the stuff I use, with pretty much arbitrary values for XXX and YYY.
(And for astronomy, at least, Python, numpy, scipy, matplotlib, and astro-py are making a real dent in the problem, but dependency hell still abounds.)
posted by RedOrGreen at 9:59 AM on October 1, 2015 [2 favorites]
This description applies to so much of the stuff I use, with pretty much arbitrary values for XXX and YYY.
(And for astronomy, at least, Python, numpy, scipy, matplotlib, and astro-py are making a real dent in the problem, but dependency hell still abounds.)
posted by RedOrGreen at 9:59 AM on October 1, 2015 [2 favorites]
I gave up on Python when I had to run
Docker is a good tool that is being over- and mis-used. The learning curve is very nice, albeit dangerous (you can get a very intuitive feel for how it works very quickly without knowing why).
I don't use Docker on any of my current projects, but I'd probably start any new project with it. The elevator pitch for me is this:
There are other container systems out there, and there are some compelling reasons to use them over Docker (again, the cargo-cult mentality), but it's hard to argue against Docker's ease of use and ubiquity.
Note that I didn't mention security or virtualization. Docker is still useful if you don't consider it secure, and accordingly, it's not a replacement for virtualization.
posted by schmod at 10:04 AM on October 1, 2015 [3 favorites]
easy-install pipDocker is a good tool that is being over- and mis-used. The learning curve is very nice, albeit dangerous (you can get a very intuitive feel for how it works very quickly without knowing why).
I don't use Docker on any of my current projects, but I'd probably start any new project with it. The elevator pitch for me is this:
Docker provides a clean and intuitive way to to set up the exact environment that your application should run in, and provides a fairly-decent guarantee that your application won't have access to anything outside of the exact things you specify.You can effectively recreate docker in 100 lines of bash. At its core, Docker adds very little other than an intuitive abstraction of some already-existing Linux tools. It's also a very light layer of abstraction, and I wouldn't discount it because it adds yet another layer, particularly because it helps reduce complexity elsewhere.
There are other container systems out there, and there are some compelling reasons to use them over Docker (again, the cargo-cult mentality), but it's hard to argue against Docker's ease of use and ubiquity.
Note that I didn't mention security or virtualization. Docker is still useful if you don't consider it secure, and accordingly, it's not a replacement for virtualization.
posted by schmod at 10:04 AM on October 1, 2015 [3 favorites]
/rings bell
Webpack is for cavemen, JSPM.io is the new hotness.
posted by Artw at 10:05 AM on October 1, 2015 [5 favorites]
Webpack is for cavemen, JSPM.io is the new hotness.
posted by Artw at 10:05 AM on October 1, 2015 [5 favorites]
ALSO: Since there are developer-people in here, note that the latest Mac OS X update will make you very unhappy if you've placed files outside of the following directories:
posted by schmod at 10:06 AM on October 1, 2015 [4 favorites]
- /Library
- ~/Library
- /usr/local
- /Applications
posted by schmod at 10:06 AM on October 1, 2015 [4 favorites]
brokkr, I understood that article to mean, not that the programs inside the container can root their way all over the filesystem, but that *the person using the docker command* can root his way all over the filesystem with the docker command.
If I was mistaken in that, I stand corrected.
posted by edheil at 10:06 AM on October 1, 2015
If I was mistaken in that, I stand corrected.
posted by edheil at 10:06 AM on October 1, 2015
JSPM is interesting, and I don't know why it isn't catching on more quickly. If nothing else, it deserves to win because it's based around an established standard. That said, I do find the interplay between ES6, JSPM, and SystemJS to be needlessly confusing, and Webpack currently has better tooling for 3rd-party libraries, etc...
posted by schmod at 10:08 AM on October 1, 2015
posted by schmod at 10:08 AM on October 1, 2015
The "patching" thing can be reduced to a single requirement:
But usually you can get someone thinking by specifically asking about the SSL library.
And don't get me started on Java: ever try to unpack all the JARs for a single app completely on your disk? Usually there's more logging frameworks than layers, and you'll have duplicate bundles of essential utility modules. How do you even begin to audit a dependency quagmire like that?
posted by rum-soaked space hobo at 10:09 AM on October 1, 2015 [1 favorite]
Okay, so when the crypto or auth library you use has a security update, how do I apply that update?There are lots of other pieces that need updating: compression libraries, language runtime interpreters, libc updates because that language runtime was written in C, etc!
But usually you can get someone thinking by specifically asking about the SSL library.
And don't get me started on Java: ever try to unpack all the JARs for a single app completely on your disk? Usually there's more logging frameworks than layers, and you'll have duplicate bundles of essential utility modules. How do you even begin to audit a dependency quagmire like that?
posted by rum-soaked space hobo at 10:09 AM on October 1, 2015 [1 favorite]
The damnedest thing is, PHP + Apache with mod_php absolutely solved the deployment problem. You write your code, you scp it up to a shared host, and you're done. The admins there take care of keeping the underlying system patched and up to date, and everything is great.
Go and Rust are doing a great job of compiling down to single binaries that you can just scp and run, but so much software I want to run is written in other languages.
And for those other languages, as far as I can tell Heroku is the only thing that has ever come close. I haven't seen anything that offers a Heroku-like experience with the ease or expense of shared hosting and mod_php. Well, things like CoreOS and Deis are out there, but they don't reasonably scale down to single-node clusters.
This complexity keeps so many people from autonomy. It's absolutely infuriating.
and omg, don't even talk to me about external DNS and service discovery on a multi-node CoreOS cluster for a hobbyist developer...
posted by SemiSophos at 10:18 AM on October 1, 2015 [5 favorites]
Go and Rust are doing a great job of compiling down to single binaries that you can just scp and run, but so much software I want to run is written in other languages.
And for those other languages, as far as I can tell Heroku is the only thing that has ever come close. I haven't seen anything that offers a Heroku-like experience with the ease or expense of shared hosting and mod_php. Well, things like CoreOS and Deis are out there, but they don't reasonably scale down to single-node clusters.
This complexity keeps so many people from autonomy. It's absolutely infuriating.
and omg, don't even talk to me about external DNS and service discovery on a multi-node CoreOS cluster for a hobbyist developer...
posted by SemiSophos at 10:18 AM on October 1, 2015 [5 favorites]
I gave up on Python when I had to run
easy-install pip
Wait until you get to
easy-install pip
Wait until you get to
pip install another-python-module-managerposted by GuyZero at 10:21 AM on October 1, 2015 [2 favorites]
i prefer oryx --reup tempo-dev --modulemanager npm --trystrategy optimistic -trm -Oo:3
posted by boo_radley at 10:45 AM on October 1, 2015
posted by boo_radley at 10:45 AM on October 1, 2015
I guess it's largely Docker's fault for framing it this way, but it's always strange to me to hear people talking about containerization as if its primary purpose is dependency management. I think of it as a nice interface around containers, which exist primarily for resource isolation.
posted by invitapriore at 10:56 AM on October 1, 2015 [2 favorites]
posted by invitapriore at 10:56 AM on October 1, 2015 [2 favorites]
And don't get me started on Java: ever try to unpack all the JARs for a single app completely on your disk? Usually there's more logging frameworks than layers, and you'll have duplicate bundles of essential utility modules. How do you even begin to audit a dependency quagmire like that?
So I've been using Ruby for about 5 years now. Unlike the author I've only needed to run it on my Mac laptop and AWS Linux instances, which rvm runs fine on. (I stopped running Ubuntu on custom hardware about 10 years ago, and have been much happier since.) Bundler is one of the things that makes me happy to use the Ruby/Rails ecosystem. (Some things make me unhappy but I guess I've drunk the Kool-aid, I like the ecosystem in general.) You can lock down the version of each gem (library) in as coarse or fine grained a way as you like, and it does all the dependency management itself, only stopping and warning you when there's an incompatibility somewhere. It stores its record of the version dependencies in a separate, human-readable file. If need be, you can reference a github repo for a gem, which is useful when you have a library/gem in a private repo and don't want to share the gem via gemfury or whatever.
I had to use gradle for a Java project about a year ago and was surprised by how inflexible it was. I had to write custom commands for building a fat jar and also for pulling down the source for, and then building, a separate jar file from one of my team's private repos, and getting the custom commands right was like pulling teeth. At least it does let you specify the versions for all the jar files you use.
So while there are things I wouldn't use Ruby or Rails for, I feel like that ecosystem represents some amount of progress towards the way things should be.
posted by A dead Quaker at 11:01 AM on October 1, 2015 [1 favorite]
So I've been using Ruby for about 5 years now. Unlike the author I've only needed to run it on my Mac laptop and AWS Linux instances, which rvm runs fine on. (I stopped running Ubuntu on custom hardware about 10 years ago, and have been much happier since.) Bundler is one of the things that makes me happy to use the Ruby/Rails ecosystem. (Some things make me unhappy but I guess I've drunk the Kool-aid, I like the ecosystem in general.) You can lock down the version of each gem (library) in as coarse or fine grained a way as you like, and it does all the dependency management itself, only stopping and warning you when there's an incompatibility somewhere. It stores its record of the version dependencies in a separate, human-readable file. If need be, you can reference a github repo for a gem, which is useful when you have a library/gem in a private repo and don't want to share the gem via gemfury or whatever.
I had to use gradle for a Java project about a year ago and was surprised by how inflexible it was. I had to write custom commands for building a fat jar and also for pulling down the source for, and then building, a separate jar file from one of my team's private repos, and getting the custom commands right was like pulling teeth. At least it does let you specify the versions for all the jar files you use.
So while there are things I wouldn't use Ruby or Rails for, I feel like that ecosystem represents some amount of progress towards the way things should be.
posted by A dead Quaker at 11:01 AM on October 1, 2015 [1 favorite]
It's like we've built up a higher order of technical debt, sort of like the National Debt. There should be a running scoreboard with "hours lost to deployment debt" spinning wildly somewhere in Silicon Valley...
posted by lubujackson at 11:02 AM on October 1, 2015 [3 favorites]
posted by lubujackson at 11:02 AM on October 1, 2015 [3 favorites]
Rails isn't startup hipster cool anymore. It's brogrammer status now.
posted by Apocryphon at 11:08 AM on October 1, 2015 [10 favorites]
posted by Apocryphon at 11:08 AM on October 1, 2015 [10 favorites]
I think the important point to take away from this article is that the problem is not dependencies, but concurrency and managing the contingencies of interactions between systems.
If the author had been using a more vanilla linux install, then Docker would have worked fine and there wouldn't have been a story. Similarly, the trouble with RVM was rooted in the specific way that the OS was configured.
Point by point you can go through this article and see that the author is making decisions that the writers of the software wouldn't have. This divergence is cumulative; each decision adds complications.
More broadly, as the author notes, there are many resources which an operating system treats as singular (eg port 80 on a network interface or the file-system path "/") for logical reasons, but can be made plural with some technical jiggery pokery (eg virtualization). Containerization is a half step in that direction.
Linux containers return the process environment to a sanitary condition, so that prior decisions made by the operator of the system have as little effect on the container's process environment as is practically achievable.
These are exactly the reasons that Discourse recommends that one install it via Docker. Effectively it reduces the preparation required to install the application to something like "Provide a relatively modern 64 bit linux"
That's not to say its how things should be, the thing that bulids the thing that builds the thing that builds the thing is a blight.
posted by ethansr at 11:10 AM on October 1, 2015 [3 favorites]
If the author had been using a more vanilla linux install, then Docker would have worked fine and there wouldn't have been a story. Similarly, the trouble with RVM was rooted in the specific way that the OS was configured.
Point by point you can go through this article and see that the author is making decisions that the writers of the software wouldn't have. This divergence is cumulative; each decision adds complications.
More broadly, as the author notes, there are many resources which an operating system treats as singular (eg port 80 on a network interface or the file-system path "/") for logical reasons, but can be made plural with some technical jiggery pokery (eg virtualization). Containerization is a half step in that direction.
Linux containers return the process environment to a sanitary condition, so that prior decisions made by the operator of the system have as little effect on the container's process environment as is practically achievable.
These are exactly the reasons that Discourse recommends that one install it via Docker. Effectively it reduces the preparation required to install the application to something like "Provide a relatively modern 64 bit linux"
That's not to say its how things should be, the thing that bulids the thing that builds the thing that builds the thing is a blight.
posted by ethansr at 11:10 AM on October 1, 2015 [3 favorites]
I had to use gradle for a Java project about a year ago and was surprised by how inflexible it was.
Oh God, gradle. Such a wolf in sheep's clothing. Try to deviate from a widely-used template and you'll be debugging method calls in other people's plugins in no time. Upgrading Gradle or its dependencies pretty much always breaks your scripts. Really, if your build scripts can execute arbitrary code, maybe you should step back and think about what you've done.
posted by RobotVoodooPower at 11:19 AM on October 1, 2015 [3 favorites]
Oh God, gradle. Such a wolf in sheep's clothing. Try to deviate from a widely-used template and you'll be debugging method calls in other people's plugins in no time. Upgrading Gradle or its dependencies pretty much always breaks your scripts. Really, if your build scripts can execute arbitrary code, maybe you should step back and think about what you've done.
posted by RobotVoodooPower at 11:19 AM on October 1, 2015 [3 favorites]
I kind of agree that gradle and sbt do *way* too much magic in their respective proprietary (and profoundly undocumented) DSLs.
In fact, I'm pretty "meh" about the whole "code as your configuration" paradigm for build tooling. It just doesn't seem maintainable in the long-term.
But, if not Gradle... then what? Maven?
Can Maven even be considered an "active" project at this point? Personally, I'd rather have seen somebody try to fix Maven's problems, but alas that doesn't seem to have happened.
posted by schmod at 11:47 AM on October 1, 2015 [2 favorites]
In fact, I'm pretty "meh" about the whole "code as your configuration" paradigm for build tooling. It just doesn't seem maintainable in the long-term.
But, if not Gradle... then what? Maven?
Can Maven even be considered an "active" project at this point? Personally, I'd rather have seen somebody try to fix Maven's problems, but alas that doesn't seem to have happened.
posted by schmod at 11:47 AM on October 1, 2015 [2 favorites]
Piping to a shell script is not significantly different than setup.exe or installing from a DMG file.
But code signing? Yes, because no attacker ever got or stole a signing cert.
Docker ain't perfect but it's literally "package up all the dependencies needed to run this thing and run it without spawning a whole separate kernel" which is a bit silly because kernels are sometimes the one thing you want to segment but some future docker will clearly let you flip a "give this its own kernel" switch so why not.
Point being, release engineering sucks, and doing release engineering for someone else's code sucks even more. Dude second guessed the release engineer all to hell and it blew up in his face. Yes, that happens.
Though he's totally completely right about the port 80 only to root garbage. Lunacy from another decade, that, even for those who are well aware that the entire user model of Unix (and most OS's) doesn't reflect reality in any way any longer.
posted by effugas at 11:52 AM on October 1, 2015 [1 favorite]
But code signing? Yes, because no attacker ever got or stole a signing cert.
Docker ain't perfect but it's literally "package up all the dependencies needed to run this thing and run it without spawning a whole separate kernel" which is a bit silly because kernels are sometimes the one thing you want to segment but some future docker will clearly let you flip a "give this its own kernel" switch so why not.
Point being, release engineering sucks, and doing release engineering for someone else's code sucks even more. Dude second guessed the release engineer all to hell and it blew up in his face. Yes, that happens.
Though he's totally completely right about the port 80 only to root garbage. Lunacy from another decade, that, even for those who are well aware that the entire user model of Unix (and most OS's) doesn't reflect reality in any way any longer.
posted by effugas at 11:52 AM on October 1, 2015 [1 favorite]
Lunacy from another decade, that, even for those who are well aware that the entire user model of Unix (and most OS's) doesn't reflect reality in any way any longer.
What's that line from the The Big Lebowski?
Say what you will about the user security models of mid-80's UNIX systems, dude, at least it's an ethos.
Piping curl output to bash, that's nihilism.
posted by GuyZero at 11:57 AM on October 1, 2015 [13 favorites]
What's that line from the The Big Lebowski?
Say what you will about the user security models of mid-80's UNIX systems, dude, at least it's an ethos.
Piping curl output to bash, that's nihilism.
posted by GuyZero at 11:57 AM on October 1, 2015 [13 favorites]
The guy complains about having to install a thing that will help standardize the install process (Docker), but then complains that the ecosystem is such a jungle without such a thing. Well, pick one. Or fork Rails and redesign it so that kind of standardization is baked in (and accept the fact that you're breaking compatibility with existing software)
This.
posted by Going To Maine at 11:59 AM on October 1, 2015 [4 favorites]
This.
posted by Going To Maine at 11:59 AM on October 1, 2015 [4 favorites]
Yeah, privileged ports are utter nonsense. If I have sshd disabled on my Mac, why shouldn't any random program I've downloaded from the internet be able to listen on port 22 and email my password to Russia when I forget it's disabled and try to ssh in?
posted by mubba at 12:12 PM on October 1, 2015 [3 favorites]
posted by mubba at 12:12 PM on October 1, 2015 [3 favorites]
Meanwhile, in spite of itself, PHP 5.4 started evolving into a pretty nice language.
If you think this you have been using PHP too long. I wont dispute that it is easy to get set up though, or that good work has been done in bringing the ecosystem up to speed.
posted by atoxyl at 12:22 PM on October 1, 2015 [1 favorite]
If you think this you have been using PHP too long. I wont dispute that it is easy to get set up though, or that good work has been done in bringing the ecosystem up to speed.
posted by atoxyl at 12:22 PM on October 1, 2015 [1 favorite]
Privileged ports are like windows screens - they won't keep real attackers out, but they keep the flies out.
posted by GuyZero at 12:28 PM on October 1, 2015
posted by GuyZero at 12:28 PM on October 1, 2015
This is another one of those articles where the author attempts to show how confusing and complicated some form of software development is, and I think to myself "who could possibly be confused by this"?
I don't know, maybe I'm been in the game too long, but if getting a Rails app running in Docker container is too much for you, perhaps a career change is in order.
I'm interviewing someone later today, and I'm going to walk them through a similar story. If getting nginx/Postgres/Rails going seems to be the least be daunting to them, I'm going to stop the interview and thank them for their time.
posted by sideshow at 12:31 PM on October 1, 2015 [1 favorite]
I don't know, maybe I'm been in the game too long, but if getting a Rails app running in Docker container is too much for you, perhaps a career change is in order.
I'm interviewing someone later today, and I'm going to walk them through a similar story. If getting nginx/Postgres/Rails going seems to be the least be daunting to them, I'm going to stop the interview and thank them for their time.
posted by sideshow at 12:31 PM on October 1, 2015 [1 favorite]
Put me on team Docker.
Back in the dark ages, to run a new service, that service would get grafted on top of an existing physical machine, and we just hoped real hard there was no dependency conflicts between the two. There was no easy undo in those days. Then virtualization became feasible, and a brand new era emerged. Instead of grafting stuff on top of each other, a fresh new machine could be created out of thin air, and the new service put on top of that.
In practice unfortunately, a machine, virtual or physical, and its services, is an on-going process. It's not a simple fire-and-forget. So recording the exact set of steps to install a piece of software is a little hairy. So you end up with a carefully curated library of virtual machines, with snapshots for every step, but it would be hard, if not impossible to setup from scratch. Especially when the application's developer has their own (virtual) machine they've been tweaking while writing the application, who no one else has access to, and the VM has gone through several versions of the software organically.
Rather than the developer having a VM that picks up a bunch of weird tweaks that they've forgotten, and only comes out when QA can't even run the software, is it encourages the developer to develop using a Dockerfile, which is trivial to copy around, and everyone who runs the software gets the same, known-good target.
Docker doesn't solve the fundamental issue, which is that software gets complicated very quickly, there are a lot of moving parts, and all those pieces have to agree. But it's a good, if small step in the right direction.
posted by fragmede at 12:37 PM on October 1, 2015 [2 favorites]
Back in the dark ages, to run a new service, that service would get grafted on top of an existing physical machine, and we just hoped real hard there was no dependency conflicts between the two. There was no easy undo in those days. Then virtualization became feasible, and a brand new era emerged. Instead of grafting stuff on top of each other, a fresh new machine could be created out of thin air, and the new service put on top of that.
In practice unfortunately, a machine, virtual or physical, and its services, is an on-going process. It's not a simple fire-and-forget. So recording the exact set of steps to install a piece of software is a little hairy. So you end up with a carefully curated library of virtual machines, with snapshots for every step, but it would be hard, if not impossible to setup from scratch. Especially when the application's developer has their own (virtual) machine they've been tweaking while writing the application, who no one else has access to, and the VM has gone through several versions of the software organically.
Rather than the developer having a VM that picks up a bunch of weird tweaks that they've forgotten, and only comes out when QA can't even run the software, is it encourages the developer to develop using a Dockerfile, which is trivial to copy around, and everyone who runs the software gets the same, known-good target.
Docker doesn't solve the fundamental issue, which is that software gets complicated very quickly, there are a lot of moving parts, and all those pieces have to agree. But it's a good, if small step in the right direction.
posted by fragmede at 12:37 PM on October 1, 2015 [2 favorites]
One of the things I love about MetaFilter is the diversity I get from reading other people's thoughts, which helps me get a feel for where I sit. That said, there's lots to unpack here, and I'm going to assume some familiarity with web deployment as I go.
I read the original article a few days ago, I had Many Thoughts, and I'm going to try to avoid the "someone's wrong on the Internet thing".
1) A touch of irony. Discourse (the software being referred to) is trying to be a better type of forum software. There are lots of things that they do super well, and the software is a delight to interact with. One of the side goals of the project was to make Ruby web apps easier to deploy. They have posts a plenty about it. One of the things that is difficult is not just getting ruby or Rails set up; it's also setting up and configuring a database. Then setting up Redis. Then..well, yeah. They settled on Docker to do that, and if you use Docker it is fairl simple. If you don't though...
2) A touch of sadness. The Ruby ecosystem does many things well, but occasionally some things just don't work. Some of this is down to the fact that the language evolves at a reasonable pace, and some of it is down to libraries that rely on C code to do things. When that goes wrong, things can go south fast.
Signal talked about the ease of Python upthread: Python. Django. Django documentation. Pip. VIrtualenv. Not perfect, but it mostly works.
You can say the same thing in this case: Ruby. Rails. Rails guides. Bundler. Not perfect, but it mostly works. In this case, it didn't. But this isn't unique.
The programming language for web stuff that I think is is best at ease of set up on a local machine is PHP. One of the things that has made PHP so dominate in the shared hosting space (as well as everywhere else) is that the expected stack is well understood. Your hosting provider will have PHP of a certain version installed (updates to the language used to be slow, so it didn't matter what version), MySQL installed and configured (why would you use anything else?), and Apache installed with mod_php. Magic. Beautiful. Easy. If you don't stray from this path (like, you didn't want to use a different database for example), this largely works.
Even in this case though, it doesn't always. Some PHP libraries required compiled support for optional in the language. Using a project that required mb_string? Or a XML parsing? Or some web access features? Sorry - your hosting provider might not provide that. And that ignores that many hosting providers still ship with an end-of-life PHP 5.3, and newer software projects are using features only available in PHP 5.4 or above.
So - no one gets it right all the time. More importantly, any experience like this is terrible, and Eevee is absolutely right to point it out. Some of the anger at the beginning is misplaced, and some of the solutions that Eevee wants to work up are *kinda* being provided by things like Docker, though other containerised formats or virtual hosts would work as well.
We should be able to do better, and there are people all over the place (including Discourse) that are trying to make it better.
posted by sarcas at 1:22 PM on October 1, 2015 [3 favorites]
I read the original article a few days ago, I had Many Thoughts, and I'm going to try to avoid the "someone's wrong on the Internet thing".
1) A touch of irony. Discourse (the software being referred to) is trying to be a better type of forum software. There are lots of things that they do super well, and the software is a delight to interact with. One of the side goals of the project was to make Ruby web apps easier to deploy. They have posts a plenty about it. One of the things that is difficult is not just getting ruby or Rails set up; it's also setting up and configuring a database. Then setting up Redis. Then..well, yeah. They settled on Docker to do that, and if you use Docker it is fairl simple. If you don't though...
2) A touch of sadness. The Ruby ecosystem does many things well, but occasionally some things just don't work. Some of this is down to the fact that the language evolves at a reasonable pace, and some of it is down to libraries that rely on C code to do things. When that goes wrong, things can go south fast.
Signal talked about the ease of Python upthread: Python. Django. Django documentation. Pip. VIrtualenv. Not perfect, but it mostly works.
You can say the same thing in this case: Ruby. Rails. Rails guides. Bundler. Not perfect, but it mostly works. In this case, it didn't. But this isn't unique.
The programming language for web stuff that I think is is best at ease of set up on a local machine is PHP. One of the things that has made PHP so dominate in the shared hosting space (as well as everywhere else) is that the expected stack is well understood. Your hosting provider will have PHP of a certain version installed (updates to the language used to be slow, so it didn't matter what version), MySQL installed and configured (why would you use anything else?), and Apache installed with mod_php. Magic. Beautiful. Easy. If you don't stray from this path (like, you didn't want to use a different database for example), this largely works.
Even in this case though, it doesn't always. Some PHP libraries required compiled support for optional in the language. Using a project that required mb_string? Or a XML parsing? Or some web access features? Sorry - your hosting provider might not provide that. And that ignores that many hosting providers still ship with an end-of-life PHP 5.3, and newer software projects are using features only available in PHP 5.4 or above.
So - no one gets it right all the time. More importantly, any experience like this is terrible, and Eevee is absolutely right to point it out. Some of the anger at the beginning is misplaced, and some of the solutions that Eevee wants to work up are *kinda* being provided by things like Docker, though other containerised formats or virtual hosts would work as well.
We should be able to do better, and there are people all over the place (including Discourse) that are trying to make it better.
posted by sarcas at 1:22 PM on October 1, 2015 [3 favorites]
Fucking architecture astronauts.
posted by 1970s Antihero at 1:24 PM on October 1, 2015 [2 favorites]
posted by 1970s Antihero at 1:24 PM on October 1, 2015 [2 favorites]
jfuller: "> It's a crazily extreme solution to a ridiculous problem, but it actually is a solution.
I am presently running 17 different flavors of Linux (not all at once.) It's my ambition to have as many as there are flavors of Bertie Bott's Every Flavor Beans, includine Earwax and Booger. Every one of these runs in a VM. I won't say what the underlying host OS is, but it comes from Redmond."
Weak sauce. Not impressed unless you have them all running at once, with a screen cast to prove it.
Seriously though, how in the blistering green fuck do you end up with a 64 bit kernal and a 32 bit userland? I mean I know how it works in Windows, but not so much Ubuntu.
posted by Samizdata at 1:26 PM on October 1, 2015
I am presently running 17 different flavors of Linux (not all at once.) It's my ambition to have as many as there are flavors of Bertie Bott's Every Flavor Beans, includine Earwax and Booger. Every one of these runs in a VM. I won't say what the underlying host OS is, but it comes from Redmond."
Weak sauce. Not impressed unless you have them all running at once, with a screen cast to prove it.
Seriously though, how in the blistering green fuck do you end up with a 64 bit kernal and a 32 bit userland? I mean I know how it works in Windows, but not so much Ubuntu.
posted by Samizdata at 1:26 PM on October 1, 2015
It has come to managements attention that your MacBook does not have the required number of stickers. Please upgrade the number of stickers on your device at the soonest convenience. Please consult GitHub statistics as to the most active frameworks to add stickers for.
posted by Artw at 1:27 PM on October 1, 2015 [13 favorites]
posted by Artw at 1:27 PM on October 1, 2015 [13 favorites]
I don't think a specific stack makes a whole lot of difference in how software gets delivered--a point repeated alluded to in the rant/essay. I also agree with his broader contention that delivery is the least thought about and lowest priority item when it comes to resources. This may be colored somewhat by my experience as one who works in SCM/CI/CD. Times are changing though. As trendy and hip as it is to talk about devops we're finally talking about it. In the past year I've participated in moving a human-intensive deployment strategy to something that is virtually push button. The process is both highly efficient and our stability on release is nearly iron clad.
Ultimately I think OSS generally and those who have pets instead of cattle will get the least benefit from these efforts. But I do thing that as we get more experience with application delivery frameworks and really go at them hammer and tongs instead of as afterthoughts, this area of the SLC is going to blossom.
posted by Fezboy! at 1:29 PM on October 1, 2015
Ultimately I think OSS generally and those who have pets instead of cattle will get the least benefit from these efforts. But I do thing that as we get more experience with application delivery frameworks and really go at them hammer and tongs instead of as afterthoughts, this area of the SLC is going to blossom.
posted by Fezboy! at 1:29 PM on October 1, 2015
Artw: "It has come to managements attention that your MacBook does not have the required number of stickers. Please upgrade the number of stickers on your device at the soonest convenience. Please consult GitHub statistics as to the most active frameworks to add stickers for."
Dude, they don't have enough pieces of digital flair? Are they not committed?
posted by Samizdata at 1:44 PM on October 1, 2015 [1 favorite]
Dude, they don't have enough pieces of digital flair? Are they not committed?
posted by Samizdata at 1:44 PM on October 1, 2015 [1 favorite]
Artw: "It has come to managements attention that your MacBook does not have the required number of stickers. Please upgrade the number of stickers on your device at the soonest convenience. Please consult GitHub statistics as to the most active frameworks to add stickers for."
OK, that made me laugh. Half of our office has MacBooks and the other half of us have HP EliteBooks and I notice that none of us Windows users have stickers and all of the MacBook users have multiple ones.
posted by octothorpe at 1:47 PM on October 1, 2015
OK, that made me laugh. Half of our office has MacBooks and the other half of us have HP EliteBooks and I notice that none of us Windows users have stickers and all of the MacBook users have multiple ones.
posted by octothorpe at 1:47 PM on October 1, 2015
Windows laptop stickers come with the machine.
posted by GuyZero at 1:50 PM on October 1, 2015 [3 favorites]
posted by GuyZero at 1:50 PM on October 1, 2015 [3 favorites]
This "ports < 1024 must be root" rule is ancient and should be deprecated, though.
I kind of disagree. Reserving the lower chunk of ports for sufficiently privileged installs - people with the root access - doesn't strike me as unreasonable and it's a complaint I found a little confusing in the article, given it going back and forth about shared hosting. Why should the person who gets to bind port 80 simply be the user who got their first?
Now, the lack of an ability to delegate that port to an app that doesn't otherwise have root access - yeah, that's insane.
The guy complains about having to install a thing that will help standardize the install process (Docker), but then complains that the ecosystem is such a jungle without such a thing. Well, pick one.
It's not really a complaint about needing Docker, though, it's a complaint about needing a more recent version and the difficulty in installing a sufficiently recent version without a number of other compounding issues. The fact is that this challenge and package dependencies - even with things that make it easier like apt - can cause a real stew inside a machine you need to keep running other things with their own dependencies.
I personally like the look of Docker, though the speed it seems to be evolving puts me off. I also have yet to find a very good narrative that would lead me though how it would actually get used in an ongoing development. All the descriptions are either "here's how to use this already built big-ass thing for deploying an existing thing" or "here's how to see it work with hello world." Since I'm not being hired on to a place with people using this where I can garner institutional knowledge I'm going to need a more hand-holdey picture of how this is going to work in a life cycle.
I think a lot of the problems this guy is describing also has a lot to do with the ascendency of things like AWS. Manage one server to run a dozen things? Fuck it, just spin up another instance and let em talk to each other. There's a lot to like about that way of solving problems but I do think sometimes that I am the only person who wants to fuck around with projects without throwing a bunch of money out the window to Amazon/Google/Heroku just to test some ideas.
posted by phearlez at 1:58 PM on October 1, 2015 [1 favorite]
I kind of disagree. Reserving the lower chunk of ports for sufficiently privileged installs - people with the root access - doesn't strike me as unreasonable and it's a complaint I found a little confusing in the article, given it going back and forth about shared hosting. Why should the person who gets to bind port 80 simply be the user who got their first?
Now, the lack of an ability to delegate that port to an app that doesn't otherwise have root access - yeah, that's insane.
The guy complains about having to install a thing that will help standardize the install process (Docker), but then complains that the ecosystem is such a jungle without such a thing. Well, pick one.
It's not really a complaint about needing Docker, though, it's a complaint about needing a more recent version and the difficulty in installing a sufficiently recent version without a number of other compounding issues. The fact is that this challenge and package dependencies - even with things that make it easier like apt - can cause a real stew inside a machine you need to keep running other things with their own dependencies.
I personally like the look of Docker, though the speed it seems to be evolving puts me off. I also have yet to find a very good narrative that would lead me though how it would actually get used in an ongoing development. All the descriptions are either "here's how to use this already built big-ass thing for deploying an existing thing" or "here's how to see it work with hello world." Since I'm not being hired on to a place with people using this where I can garner institutional knowledge I'm going to need a more hand-holdey picture of how this is going to work in a life cycle.
I think a lot of the problems this guy is describing also has a lot to do with the ascendency of things like AWS. Manage one server to run a dozen things? Fuck it, just spin up another instance and let em talk to each other. There's a lot to like about that way of solving problems but I do think sometimes that I am the only person who wants to fuck around with projects without throwing a bunch of money out the window to Amazon/Google/Heroku just to test some ideas.
posted by phearlez at 1:58 PM on October 1, 2015 [1 favorite]
It has come to our attention your MacBook is not a MacBook. Jesus, how do you expect this shit to run on your crapbox, peon? Your running Mint? Whatevs, we never tested this outside of BSD.
posted by Artw at 2:47 PM on October 1, 2015 [1 favorite]
posted by Artw at 2:47 PM on October 1, 2015 [1 favorite]
Reading this thread with a blend of amusement and sadness.
In nearly 18 years of my second career in web and webapp development, I've learned and used Perl, Java, JS (client-side), dot-NET, PHP, Python ... and up to several frameworks in each. I'm not a sysadmin but did get to help one set up several servers, and probably the best takeaway was that server admin and software deployment are specialties in themselves, not some afterthought that you toss at the developer as s/he starts packing up late Friday afternoon.
I did get a good handle on setting up Jenkins to automate deploys from svn repos.
As I look for my next contract, it's a tad discouraging to see so many startup ads looking for Ruby/Rails. Oh well, I guess it will be another contract trying to maintain/update yet another abandoned one-off pos POS system in PHP4...
posted by Artful Codger at 2:55 PM on October 1, 2015 [4 favorites]
In nearly 18 years of my second career in web and webapp development, I've learned and used Perl, Java, JS (client-side), dot-NET, PHP, Python ... and up to several frameworks in each. I'm not a sysadmin but did get to help one set up several servers, and probably the best takeaway was that server admin and software deployment are specialties in themselves, not some afterthought that you toss at the developer as s/he starts packing up late Friday afternoon.
I did get a good handle on setting up Jenkins to automate deploys from svn repos.
As I look for my next contract, it's a tad discouraging to see so many startup ads looking for Ruby/Rails. Oh well, I guess it will be another contract trying to maintain/update yet another abandoned one-off pos POS system in PHP4...
posted by Artful Codger at 2:55 PM on October 1, 2015 [4 favorites]
Aw, signal, I can't believe you want to debug this. I don't want to and it's my volunteer enthusiasm. (The actual failure is building greenlet, pip 6.0.6 seemed fine but won't upgrade itself, and I'm suspicious of virtualenv and back ported venv together. )
Years ago when I left the 'soft we were in "DLL hell", and I had some smart colleagues trying to fix it by modularizing COM objects. How'd they do?
posted by clew at 2:56 PM on October 1, 2015 [1 favorite]
Years ago when I left the 'soft we were in "DLL hell", and I had some smart colleagues trying to fix it by modularizing COM objects. How'd they do?
posted by clew at 2:56 PM on October 1, 2015 [1 favorite]
Artful Codger, if you've got that background, you'll pick up Ruby in a week. I literally just watched a new employee at my shop do that two months ago. We like Ruby and use Rails frequently. I disagree that Rails is the only option available though; we're deploying a complex Sinatra app (think Flask, if you're into Python) tomorrow evening. That said, I think this:
server admin and software deployment are specialties in themselves
is incredibly important and true. I realize sysadmins are hard to find, but ours is worth his weight in gold. I tend to think DevOps is less "the wave of the future!" and more just a cost-cutting measure because companies don't realize the value of a good sysadmin. Even though we deploy 95% of our stuff to AWS, having him around as a backstop lets me and the other devs just write code and act as the junior partners when it comes to deployment.
posted by protocoach at 3:36 PM on October 1, 2015 [2 favorites]
server admin and software deployment are specialties in themselves
is incredibly important and true. I realize sysadmins are hard to find, but ours is worth his weight in gold. I tend to think DevOps is less "the wave of the future!" and more just a cost-cutting measure because companies don't realize the value of a good sysadmin. Even though we deploy 95% of our stuff to AWS, having him around as a backstop lets me and the other devs just write code and act as the junior partners when it comes to deployment.
posted by protocoach at 3:36 PM on October 1, 2015 [2 favorites]
So much of my time is sunk into environment issues these days. Getting to do actual dev work is a rare thing to be treasured.
posted by Artw at 3:46 PM on October 1, 2015 [4 favorites]
posted by Artw at 3:46 PM on October 1, 2015 [4 favorites]
The other convenience factor of PHP is that it will happily sit on top of a regular web server, and things generally tend to follow an intuitive directory structure.
There are drawbacks to this approach, but it's very easy to get up and going.
posted by schmod at 4:01 PM on October 1, 2015
There are drawbacks to this approach, but it's very easy to get up and going.
posted by schmod at 4:01 PM on October 1, 2015
You know what's was a huge installation hassle? Installing stuff on the Mac. How was that largely solved? Homebrew.
That was the *third* effort to solve installing Unix-style software on a Mac. Fink and MacPorts are just as easy, and in many cases (most?) superior. Brew's a faddish nightmare.
posted by Mo Nickels at 4:15 PM on October 1, 2015 [1 favorite]
That was the *third* effort to solve installing Unix-style software on a Mac. Fink and MacPorts are just as easy, and in many cases (most?) superior. Brew's a faddish nightmare.
posted by Mo Nickels at 4:15 PM on October 1, 2015 [1 favorite]
That was the *third* effort to solve installing Unix-style software on a Mac. Fink and MacPorts are just as easy, and in many cases (most?) superior. Brew's a faddish nightmare.
I have to admit I never quite figured out all the differences between these other than putting things in /opt or not. I certainly find it extremely annoying that there are three of them.
posted by atoxyl at 4:42 PM on October 1, 2015 [1 favorite]
I have to admit I never quite figured out all the differences between these other than putting things in /opt or not. I certainly find it extremely annoying that there are three of them.
posted by atoxyl at 4:42 PM on October 1, 2015 [1 favorite]
We should make a new mac software installer, to establish a standard...
posted by kaibutsu at 5:13 PM on October 1, 2015 [3 favorites]
posted by kaibutsu at 5:13 PM on October 1, 2015 [3 favorites]
Homebrew always seemed to work with less fuss than the alternatives for me.
I actually had no idea that MacPorts and Fink were both still going. Didn't one (or both) of them go dormant for a time?
And... oh gosh. Fink still uses CVS for its source control.
posted by schmod at 6:09 PM on October 1, 2015
I actually had no idea that MacPorts and Fink were both still going. Didn't one (or both) of them go dormant for a time?
And... oh gosh. Fink still uses CVS for its source control.
posted by schmod at 6:09 PM on October 1, 2015
kaibutsu: "We should make a new mac software installer, to establish a standard..."
How could I tell the link without clicking?
posted by Samizdata at 6:48 PM on October 1, 2015 [3 favorites]
How could I tell the link without clicking?
posted by Samizdata at 6:48 PM on October 1, 2015 [3 favorites]
"So much of my time is sunk into environment issues these days. Getting to do actual dev work is a rare thing to be treasured."
Artw, I think we may be coworkers.
posted by device55 at 7:42 PM on October 1, 2015 [1 favorite]
Artw, I think we may be coworkers.
posted by device55 at 7:42 PM on October 1, 2015 [1 favorite]
I had some smart colleagues trying to fix it [DLL Hell] by modularizing COM objects. How'd they do?
not bad actually. though .net is fully com-compatible with bindings (RCWs), in a lot of cases unless you're bound to some system32 or low-level stuff, you can mostly dodge it. the global assembly cache is a better-but-still-version-sensitive-registry-sorta-thing.
the portable libraries and katana web server infrastructure are hella-good.
i've deployed pretty nifty[and very sophisticated] web apis across several windows hosting environments using installshield to package and deploy. easy-peasy. not cheap by any measure.
posted by j_curiouser at 8:03 PM on October 1, 2015 [1 favorite]
not bad actually. though .net is fully com-compatible with bindings (RCWs), in a lot of cases unless you're bound to some system32 or low-level stuff, you can mostly dodge it. the global assembly cache is a better-but-still-version-sensitive-registry-sorta-thing.
the portable libraries and katana web server infrastructure are hella-good.
i've deployed pretty nifty[and very sophisticated] web apis across several windows hosting environments using installshield to package and deploy. easy-peasy. not cheap by any measure.
posted by j_curiouser at 8:03 PM on October 1, 2015 [1 favorite]
Being in the Windows and .NET worlds has probably spoiled me. As long as you avoid the GAC, you can deploy your dependencies alongside your application, and you're pretty much good to go.
posted by a snickering nuthatch at 8:45 PM on October 1, 2015 [1 favorite]
posted by a snickering nuthatch at 8:45 PM on October 1, 2015 [1 favorite]
unless you are the decision maker about which one to use, developers find themselves having to learn brand new frameworks at each new job and deal with additional multitudes of installation and development quirks and roadblocks inherent to each one.
Which becomes an even worse problem when you're not working in some hipster startup in San Fran or New York, but rather in a (to pull an example completely at random) a staid, hidebound traditional bank with an IT infrastructure going back as far as IT has existed where the cowboy mentality of fresher developers clashes hideously with the ordnung muss sein mindset of the organisation as a whole, before we even get into such horrors from beyond time and space as having internal employees depending on IE8 as their browser of necessity, completely incapable of running all the fancy new gear you're busy implementing, with the decision to migrate the in house work management system which your stuff needs to be integrated in to Chrome probably will be taken sometime in 2016, so that in the meantime the workaround will probably be to just copy and paste the urls of your app from IE8 and start it in Chrome manually.
posted by MartinWisse at 11:13 PM on October 1, 2015 [1 favorite]
Which becomes an even worse problem when you're not working in some hipster startup in San Fran or New York, but rather in a (to pull an example completely at random) a staid, hidebound traditional bank with an IT infrastructure going back as far as IT has existed where the cowboy mentality of fresher developers clashes hideously with the ordnung muss sein mindset of the organisation as a whole, before we even get into such horrors from beyond time and space as having internal employees depending on IE8 as their browser of necessity, completely incapable of running all the fancy new gear you're busy implementing, with the decision to migrate the in house work management system which your stuff needs to be integrated in to Chrome probably will be taken sometime in 2016, so that in the meantime the workaround will probably be to just copy and paste the urls of your app from IE8 and start it in Chrome manually.
posted by MartinWisse at 11:13 PM on October 1, 2015 [1 favorite]
If you have Chrome installed at the local machine level (by an admin account) you could give users a desktop or home folder shortcut to a Chrome-specific web target like so (from the new shortcut dialog box):
"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" http://www.your_url.stuff
posted by aydeejones at 11:58 PM on October 1, 2015
"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" http://www.your_url.stuff
posted by aydeejones at 11:58 PM on October 1, 2015
Not that you don't know that, but it's that sort of little extra step to make a contrived work environment slightly easier that helped me stand out. Why not give it a CTRL+ALT+Key shortcut to boot, and tell users about it via a global email, making sure it's copied to all of their C:\Users\Public\Desktop folders
posted by aydeejones at 12:00 AM on October 2, 2015
posted by aydeejones at 12:00 AM on October 2, 2015
So much of my time is sunk into environment issues these days. Getting to do actual dev work is a rare thing to be treasured.
Too often also true for testing work. Not just fighting the software you're nominally testing, but the infrastructure that should make testing it possible.
posted by MartinWisse at 12:25 AM on October 2, 2015 [3 favorites]
Too often also true for testing work. Not just fighting the software you're nominally testing, but the infrastructure that should make testing it possible.
posted by MartinWisse at 12:25 AM on October 2, 2015 [3 favorites]
Nothing: "Note that the security problem mentioned with Docker is not that a process inside the docker container can break out. It is that a user on the host system with access to run docker can get access to host resources that their user should not be able to access. This is a vulnerability, but it is a different vulnerability."And then what happens when the web app you're running inside docker turns out to have an arbitrary code execution vulnerability?
posted by brokkr at 12:47 AM on October 2, 2015
We all use electricity, but few of us build our own generators. It's easier to buy electricity from the electric company.
I think most companies should do the same with computing; don't build your own servers and production environments, but buy them. With AWS, Azure, Google's Cloud Platform and others, you can do that now. Get the plumbing out of the way so you can start writing code.
posted by Triplanetary at 2:24 AM on October 2, 2015 [1 favorite]
I think most companies should do the same with computing; don't build your own servers and production environments, but buy them. With AWS, Azure, Google's Cloud Platform and others, you can do that now. Get the plumbing out of the way so you can start writing code.
posted by Triplanetary at 2:24 AM on October 2, 2015 [1 favorite]
It was written by a guy who Google said wasn't smart enough to work for them.
So what does "inverting a binary tree" even mean?
posted by effbot at 3:02 AM on October 2, 2015
So what does "inverting a binary tree" even mean?
posted by effbot at 3:02 AM on October 2, 2015
Server-side on back is basically solved if you want a stable, run-almost-anywhere platform:
Spring + Java 7/8 + non-ORM JDBC-based DB mapping framework + (Postgres/DB2/Oracle, pick according to size and budget).
The big sites tend to be JVM-based, though some of them are turning to Scala rather than Java. (Twitter moved from Rails to Scala once the fail-whale became embarrassingly common, and the Guardian is a huge Scala shop.) Java's committee-designed conservatism tends to lag a bit (it only recently got lambdas), and there is the impression that it's a good platform mostly for large institutional projects which aren't meant to change rapidly, when you have large teams of graduate programmers plodding along and ticking off tickets. Meanwhile, Scala has the advantage of attracting those who are more passionate about programming and cognizant of different paradigms (Scala itself can do reasonably solid functional programming; i.e., in the futures-and-options-are-monads sense, rather than the here's-an-array-you-can-map-and-reduce-over sense, and while it is pragmatic about the use of the paradigm, you can go all the way towards Haskell-style statelessness if that's your kink; meanwhile, the Play framework borrows agent-oriented programming from Erlang). Some ten years earlier, they called this the “Python effect”: back then if you decided to develop in Python rather than, say, Perl or Java or something from Microsoft, you'd get a more knowledgeable class of developer.
Meanwhile, on the horizon, there's Kotlin, a new JVM-based language from the JetBrains people (best known for the IDEA family of IDEs); it'll be interesting to see how it goes.
posted by acb at 3:50 AM on October 2, 2015 [3 favorites]
Spring + Java 7/8 + non-ORM JDBC-based DB mapping framework + (Postgres/DB2/Oracle, pick according to size and budget).
The big sites tend to be JVM-based, though some of them are turning to Scala rather than Java. (Twitter moved from Rails to Scala once the fail-whale became embarrassingly common, and the Guardian is a huge Scala shop.) Java's committee-designed conservatism tends to lag a bit (it only recently got lambdas), and there is the impression that it's a good platform mostly for large institutional projects which aren't meant to change rapidly, when you have large teams of graduate programmers plodding along and ticking off tickets. Meanwhile, Scala has the advantage of attracting those who are more passionate about programming and cognizant of different paradigms (Scala itself can do reasonably solid functional programming; i.e., in the futures-and-options-are-monads sense, rather than the here's-an-array-you-can-map-and-reduce-over sense, and while it is pragmatic about the use of the paradigm, you can go all the way towards Haskell-style statelessness if that's your kink; meanwhile, the Play framework borrows agent-oriented programming from Erlang). Some ten years earlier, they called this the “Python effect”: back then if you decided to develop in Python rather than, say, Perl or Java or something from Microsoft, you'd get a more knowledgeable class of developer.
Meanwhile, on the horizon, there's Kotlin, a new JVM-based language from the JetBrains people (best known for the IDEA family of IDEs); it'll be interesting to see how it goes.
posted by acb at 3:50 AM on October 2, 2015 [3 favorites]
As I recall, MacPorts grew out of the original *BSD ports effort, which had been around for, what, a decade before OSX heaved itself over the horizon & onto our machines? As such, it has a slightly more old-school “lets get things right” attitude that often goes with projects that have been around for the long haul. That meant proper dependency management & the occasional wait for new things to be added to the port system whilst bugs were squashed & dependencies managed.
Meanwhile Fink was an attempt to piggy-back on the while Debian/Apt package management system, but they made they made the classic error of thinking that the tools were what made the whole thing work. This is a fallacy of course: The quality that Debian embodies lies not in the package manager itself, but in the hours of effort put in by a horde of maintainers to make sure that their packages fit into the system as a whole. Same with the Ports system really, and *Ports has a poorer reputation for reliability only because they can’t match the peoplepower that Debian can unleash. Fink ported the tools & then discovered that they didn’t really have the time to make all the packages actually work properly, so the whole project pretty much ground to a halt.
Meanwhile Homebrew was the 17 year old kid who says “sod that: I want the new stuff now”. Which works fine, until you get into a situation where upgrading a particular package breaks half the rest of your installed software. That tension between “want it now” and “lets make sure everything works as expected” never goes away of course, but it’s pretty obvious which of those two choices the ruby/rails crowd was going to make. New and shiny wins out again!
posted by pharm at 4:12 AM on October 2, 2015 [2 favorites]
Meanwhile Fink was an attempt to piggy-back on the while Debian/Apt package management system, but they made they made the classic error of thinking that the tools were what made the whole thing work. This is a fallacy of course: The quality that Debian embodies lies not in the package manager itself, but in the hours of effort put in by a horde of maintainers to make sure that their packages fit into the system as a whole. Same with the Ports system really, and *Ports has a poorer reputation for reliability only because they can’t match the peoplepower that Debian can unleash. Fink ported the tools & then discovered that they didn’t really have the time to make all the packages actually work properly, so the whole project pretty much ground to a halt.
Meanwhile Homebrew was the 17 year old kid who says “sod that: I want the new stuff now”. Which works fine, until you get into a situation where upgrading a particular package breaks half the rest of your installed software. That tension between “want it now” and “lets make sure everything works as expected” never goes away of course, but it’s pretty obvious which of those two choices the ruby/rails crowd was going to make. New and shiny wins out again!
posted by pharm at 4:12 AM on October 2, 2015 [2 favorites]
(I think inverting a binary tree was meant to mean swapping the order of all the elements in a binary search tree. The algorithm to do that is trivial. However, if that was the actual question asked, then it was horribly ambiguous. There was quite a big discussion on r/programming when that whole kerfuffle blew up IIRC, worth a quick peruse if you care about the answer to your question.)
posted by pharm at 4:15 AM on October 2, 2015
posted by pharm at 4:15 AM on October 2, 2015
The article is pretty accurate... aside of the small technical issues, which tend to divert the discussion.
Rails is easily my favourite web development platform because it just does so many things right. I tried some Python frameworks, but they were clearly lagging behind on some of the features. (Ruby's activerecord is just fantastic to develop for, compared to Python's database libraries - ugh, Django ORM is limited and SQLAlchemy just doesn't seem to be able to decide what it is. My poor brain fried when I tried to mess my way around the documentation do a simple SQL COUNT() in both of these. Ruby? You might find it hard to believe, but it has, like, a .count method.)
Which is a shame, because I can't actually deploy Rails applications on any shared-host type deal, because shared-host stuff has limitations.
And I guess I now realise much of this is about the somewhat duct-tapey way the security model of these shared hosts was established - "Cronjobs are (sometimes) out, dedicated application servers are (definitely) out - you can serve static HTML, and anything you can run through CGI, and PHP of course, unless PHP is somehow hobbled, because of course it is".
Part of me hopes that J2EE open source had gotten its shit together a decade earlier than did and that JRuby hadn't been slightly neglected after Oracle took over Java. If only shared hosts would provide Java application servers as one of the possible choices, and you could deploy Rails apps straight offa .war files...
Deploying any web application to any server sucks.
And don't get me started on how easy PHP is. Sure, I can update Drupal with a few clicks, but I also have a 20-step command-line checklist for updating MediaWiki. Which changes every now and then because this Composer thingy (blessing? curse? who the hell knows yet) was invented.
Oh, and it kind of annoys me that the web application I've had least amount of issues with, ever, is Movable Type 4 and 5. Written in Perl 5, which people really don't recommend for new apps. And of course, the open source project is dead, so good luck with that. There ain't no justice.
posted by wwwwolf at 5:47 AM on October 2, 2015
Rails is easily my favourite web development platform because it just does so many things right. I tried some Python frameworks, but they were clearly lagging behind on some of the features. (Ruby's activerecord is just fantastic to develop for, compared to Python's database libraries - ugh, Django ORM is limited and SQLAlchemy just doesn't seem to be able to decide what it is. My poor brain fried when I tried to mess my way around the documentation do a simple SQL COUNT() in both of these. Ruby? You might find it hard to believe, but it has, like, a .count method.)
Which is a shame, because I can't actually deploy Rails applications on any shared-host type deal, because shared-host stuff has limitations.
And I guess I now realise much of this is about the somewhat duct-tapey way the security model of these shared hosts was established - "Cronjobs are (sometimes) out, dedicated application servers are (definitely) out - you can serve static HTML, and anything you can run through CGI, and PHP of course, unless PHP is somehow hobbled, because of course it is".
Part of me hopes that J2EE open source had gotten its shit together a decade earlier than did and that JRuby hadn't been slightly neglected after Oracle took over Java. If only shared hosts would provide Java application servers as one of the possible choices, and you could deploy Rails apps straight offa .war files...
Deploying any web application to any server sucks.
And don't get me started on how easy PHP is. Sure, I can update Drupal with a few clicks, but I also have a 20-step command-line checklist for updating MediaWiki. Which changes every now and then because this Composer thingy (blessing? curse? who the hell knows yet) was invented.
Oh, and it kind of annoys me that the web application I've had least amount of issues with, ever, is Movable Type 4 and 5. Written in Perl 5, which people really don't recommend for new apps. And of course, the open source project is dead, so good luck with that. There ain't no justice.
posted by wwwwolf at 5:47 AM on October 2, 2015
wwwwolf: "Deploying any web application to any server sucks."
For simple projects, Heroku one line deployment just-works. At least from a Django perspective, the only (minor) gotcha is that uploaded files don't persist, so you have to set up storage on AWS or something if you're gonna have users or admins uploading files.
posted by signal at 6:05 AM on October 2, 2015
For simple projects, Heroku one line deployment just-works. At least from a Django perspective, the only (minor) gotcha is that uploaded files don't persist, so you have to set up storage on AWS or something if you're gonna have users or admins uploading files.
posted by signal at 6:05 AM on October 2, 2015
Meanwhile, Scala has the advantage of attracting those who are more passionate about programming and cognizant of different paradigms
Translation: Most Scala code is too clever by half, and it's non-trivial to work on somebody else's Scala code.
I nearly threw my monitor across the room when I realized that
posted by schmod at 7:22 AM on October 2, 2015 [4 favorites]
Translation: Most Scala code is too clever by half, and it's non-trivial to work on somebody else's Scala code.
I nearly threw my monitor across the room when I realized that
? and ! are methods on a few of the classes that Akka uses, while trying to puzzle through what the hell actor ! f(operation) was supposed to do.posted by schmod at 7:22 AM on October 2, 2015 [4 favorites]
So what does "inverting a binary tree" even mean?
In general it means you are in an interview and you better have done your CS-for-interview-use-only brush up.
posted by Artw at 7:27 AM on October 2, 2015 [1 favorite]
In general it means you are in an interview and you better have done your CS-for-interview-use-only brush up.
posted by Artw at 7:27 AM on October 2, 2015 [1 favorite]
(Because your actual job will mostly be dealing with sodding docker/gradel/awful CMS/the terrible "enterprise" time tracking software that sucks up Friday afternoon/whatever. And meetings. So many meeting. Hey, we're agile now!)
posted by Artw at 7:29 AM on October 2, 2015 [12 favorites]
posted by Artw at 7:29 AM on October 2, 2015 [12 favorites]
Studying up on my trees/heaps/hashes/sorts/searches/etc right now as I'm preparing for an interview in a few weeks. Tech has this weird thing where you have this set of skills that you learn in school but only ever use for job interviews and never on the job you get if you pass that interview.
For example, how to do a sort:
* In a job interview you show that you can code either a recursive or iterative implementation of a quick sort or merge sort in Java or C++ on a white board. This takes a lot of thought and a dozen or so lines of code.
* On the job you type something like "thing = thing.sort()" or "sort(thing)" and you're done.
posted by octothorpe at 8:08 AM on October 2, 2015 [2 favorites]
For example, how to do a sort:
* In a job interview you show that you can code either a recursive or iterative implementation of a quick sort or merge sort in Java or C++ on a white board. This takes a lot of thought and a dozen or so lines of code.
* On the job you type something like "thing = thing.sort()" or "sort(thing)" and you're done.
posted by octothorpe at 8:08 AM on October 2, 2015 [2 favorites]
It actually didn't occur to me that the real name for what they were talking about wasn't "Piss Horse".
Things our customers have said about Discourse
"Piss Horse" is on the milder side.
posted by flabdablet at 8:10 AM on October 2, 2015
Things our customers have said about Discourse
"Piss Horse" is on the milder side.
posted by flabdablet at 8:10 AM on October 2, 2015
Where is the modern-day equivalent of Kernighan and Ritchie or Kernighan and Pike? Those books are classics because they were the perfect combination of brevity and lucidity, with practical real-world examples.
Buried under decades of Wirth's Law.
Also, none of the "real-world" examples in either of those books were anywhere near production-ready even when those books were new. Software is inherently complicated, and every attempt to abstract that fact away adds more layers of complication.
All code written by people who think pointers are a difficult and confusing concept is cargo cult code. Unfortunately, in 2015, that's most of it.
posted by flabdablet at 8:22 AM on October 2, 2015 [2 favorites]
Buried under decades of Wirth's Law.
Also, none of the "real-world" examples in either of those books were anywhere near production-ready even when those books were new. Software is inherently complicated, and every attempt to abstract that fact away adds more layers of complication.
All code written by people who think pointers are a difficult and confusing concept is cargo cult code. Unfortunately, in 2015, that's most of it.
posted by flabdablet at 8:22 AM on October 2, 2015 [2 favorites]
My poor brain fried when I tried to mess my way around the documentation do a simple SQL COUNT() in both of these.
Surely this is not the edge case that fried your brain.
Django's count function. Documented as far back as 1.4, but I'm pretty sure I saw it in the original Django book covering 0.96 / 1.0. The top hit when you search for 'count' on the Django docs page is Aggregation, who's first example is counting. It is literally just Book.objects.count().
posted by pwnguin at 10:00 AM on October 2, 2015
Surely this is not the edge case that fried your brain.
Django's count function. Documented as far back as 1.4, but I'm pretty sure I saw it in the original Django book covering 0.96 / 1.0. The top hit when you search for 'count' on the Django docs page is Aggregation, who's first example is counting. It is literally just Book.objects.count().
posted by pwnguin at 10:00 AM on October 2, 2015
Tech has this weird thing where you have this set of skills that you learn in school but only ever use for job interviews and never on the job you get if you pass that interview.
The worst part about this tendency is that there are a subset of people who seem to take immense pleasure in torturing interviewees with these kinds of questions, and base way too much of their personal self worth on having memorized the answers.
We get it, Bob: if there was some sort of apocalyptic loss of the entire standard library of the language of your choice, you could personally re-implement a big chunk of it without having to reference any of the literature on algorithms. Meanwhile, the rest of us are over here building software that actually pays the bills, and assuming that this is probably not going to happen (or, if it did, that we'd have more than 10 minutes to fix it, and maybe take a minute to refresh our memory from a book or some crazy such thing).
posted by tocts at 10:51 AM on October 2, 2015 [5 favorites]
The worst part about this tendency is that there are a subset of people who seem to take immense pleasure in torturing interviewees with these kinds of questions, and base way too much of their personal self worth on having memorized the answers.
We get it, Bob: if there was some sort of apocalyptic loss of the entire standard library of the language of your choice, you could personally re-implement a big chunk of it without having to reference any of the literature on algorithms. Meanwhile, the rest of us are over here building software that actually pays the bills, and assuming that this is probably not going to happen (or, if it did, that we'd have more than 10 minutes to fix it, and maybe take a minute to refresh our memory from a book or some crazy such thing).
posted by tocts at 10:51 AM on October 2, 2015 [5 favorites]
All code written by people who think pointers are a difficult and confusing concept is cargo cult code. Unfortunately, in 2015, that's most of it.
"this is a golden nugget of truth," he noted after watching his pair, yet again, paste that same validation block.
posted by j_curiouser at 1:41 PM on October 2, 2015
"this is a golden nugget of truth," he noted after watching his pair, yet again, paste that same validation block.
posted by j_curiouser at 1:41 PM on October 2, 2015
No, it's not sexy. And there are a ton of startup hipsters that will chime in and tell you how you should be using Node/Rails/some other "cool" platform. They'll tell you how that's "old" (as if age is inversely proportional to quality.)
Total bullshit. Not sure this deserves much more of a response, but to counter some blatant FUD:
A) Plenty of old people use Node
B) Their attention spans are fine, and they, too, have had careers in C++ and Java
C) I have never ever heard a Node developer say, "Oh, that sucks cuz it's old!" Have you ever talked to these people? Do you imagine they're all Poochie from the Simpsons? They generally love the shit of out Unix and the Unix philosophy.
D) Plenty of non-"sexy" non-startup companies like Wal-Mart and PayPal use Node in production
E) It works fucking fine
F) At some point, someone said the same shit about Java. Goddamn Java hipsters
posted by ignignokt at 1:57 PM on October 2, 2015 [3 favorites]
Total bullshit. Not sure this deserves much more of a response, but to counter some blatant FUD:
A) Plenty of old people use Node
B) Their attention spans are fine, and they, too, have had careers in C++ and Java
C) I have never ever heard a Node developer say, "Oh, that sucks cuz it's old!" Have you ever talked to these people? Do you imagine they're all Poochie from the Simpsons? They generally love the shit of out Unix and the Unix philosophy.
D) Plenty of non-"sexy" non-startup companies like Wal-Mart and PayPal use Node in production
E) It works fucking fine
F) At some point, someone said the same shit about Java. Goddamn Java hipsters
posted by ignignokt at 1:57 PM on October 2, 2015 [3 favorites]
I like Node a lot, it just has the misfortune of being the current silver bullet solution for everything.
posted by Artw at 2:17 PM on October 2, 2015 [1 favorite]
posted by Artw at 2:17 PM on October 2, 2015 [1 favorite]
"(Because your actual job will mostly be dealing with sodding docker/gradel/awful CMS/the terrible "enterprise" time tracking software that sucks up Friday afternoon/whatever. And meetings. So many meeting. Hey, we're agile now!)"
Reads Annika Cicada on a Friday afternoon as she looks with disdain at the other browser tab, Jira, open, demanding time to be recorded. She looks at the traffic on the freeway gaining density by the minute as she peers longingly out the tinted office windows, across the sprawling new-urbanist inspired commercial center, dying to get home to her sweetheart and a cold beer. "But the time must be entered" says she, and quietly puts her head down back to work.
posted by Annika Cicada at 2:21 PM on October 2, 2015 [4 favorites]
Reads Annika Cicada on a Friday afternoon as she looks with disdain at the other browser tab, Jira, open, demanding time to be recorded. She looks at the traffic on the freeway gaining density by the minute as she peers longingly out the tinted office windows, across the sprawling new-urbanist inspired commercial center, dying to get home to her sweetheart and a cold beer. "But the time must be entered" says she, and quietly puts her head down back to work.
posted by Annika Cicada at 2:21 PM on October 2, 2015 [4 favorites]
Nine of the project categories make sense? Why do none of the categories make sense? Who set this up? Yet got forbid you enter anything without a project category.
posted by Artw at 2:41 PM on October 2, 2015 [1 favorite]
posted by Artw at 2:41 PM on October 2, 2015 [1 favorite]
On the other hand, two days ago my son and I were trying to figure out if our microwave is interfering with the wireless at our house. Three minutes of research and I downloaded and installed a free, working magnetometer to my Android phone to measure the magnetic field around the microwave. Then we had a talk about correlation and causation.
This cargo cult thing is working for me!
Except a magnetometer isn't what you need, you need an RF signal meter tuned to 2.4GHz. The magnetron will have a magnetic field around it when it's running because of the AC current (changing electrical currents induce magnetic fields) but that's true regardless if the microwave is leaking microwaves (read, electromagnetic energy in the microwave spectrum.)
The simplest way to tell? Get a wifi signal meter, run it, put device about 5' from microwave. Start microwave. If you see a big loss of signal or big jump in background noise, your microwave is leaking microwaves.
posted by eriko at 4:00 PM on October 2, 2015 [2 favorites]
This cargo cult thing is working for me!
Except a magnetometer isn't what you need, you need an RF signal meter tuned to 2.4GHz. The magnetron will have a magnetic field around it when it's running because of the AC current (changing electrical currents induce magnetic fields) but that's true regardless if the microwave is leaking microwaves (read, electromagnetic energy in the microwave spectrum.)
The simplest way to tell? Get a wifi signal meter, run it, put device about 5' from microwave. Start microwave. If you see a big loss of signal or big jump in background noise, your microwave is leaking microwaves.
posted by eriko at 4:00 PM on October 2, 2015 [2 favorites]
Tech has this weird thing where you have this set of skills that you learn in school but only ever use for job interviews and never on the job you get if you pass that interview.
I dunno, having been both interviewer and interviewee in these environments, odds are that when the interviewer asks you to solve a small programming problem on the whiteboard, they're looking for a whole bunch of skills that they expect you to use every day if you pass that interview. (Including the very basic one of not stomping off to Twitter and ranting about how about everyone else is stupid because they asked you to do something you didn't know how to do :-)
There was quite a big discussion on r/programming when that whole kerfuffle blew up IIRC
I tend to avoid kerfuffle discussions on reddit and hacker news; they're mostly ranging from that mix of engineer's disease and dunning kruger where you think that since you have colleagues that are slightly worse than yourself, you're clearly a genius, and all the way to outright mythomania. Which can be entertaining, of course, but usually just gives me a headache.
posted by effbot at 3:41 AM on October 3, 2015 [1 favorite]
I dunno, having been both interviewer and interviewee in these environments, odds are that when the interviewer asks you to solve a small programming problem on the whiteboard, they're looking for a whole bunch of skills that they expect you to use every day if you pass that interview. (Including the very basic one of not stomping off to Twitter and ranting about how about everyone else is stupid because they asked you to do something you didn't know how to do :-)
There was quite a big discussion on r/programming when that whole kerfuffle blew up IIRC
I tend to avoid kerfuffle discussions on reddit and hacker news; they're mostly ranging from that mix of engineer's disease and dunning kruger where you think that since you have colleagues that are slightly worse than yourself, you're clearly a genius, and all the way to outright mythomania. Which can be entertaining, of course, but usually just gives me a headache.
posted by effbot at 3:41 AM on October 3, 2015 [1 favorite]
That's not cargo cult, that's just basic scientific exploration.
I think that's the tech industry usage of cargo cult, which makes sense. I was thinking about the ritual search for a solution, first by petitioning the mighty Google for the name of an app that sort of maybe might head us in the right direction (or at least help with the terminology), followed by a trip to the altar of Play to learn if it really exists, and finally praying for a working application. All to use a tool whose principles I barely understand (yesterday I couldn't spell magnetometer...). And who (as a generic user) would have guessed that someone could re-purpose the physical compass in a smart phone to measure magnetic fields?
posted by sneebler at 8:40 AM on October 3, 2015
I think that's the tech industry usage of cargo cult, which makes sense. I was thinking about the ritual search for a solution, first by petitioning the mighty Google for the name of an app that sort of maybe might head us in the right direction (or at least help with the terminology), followed by a trip to the altar of Play to learn if it really exists, and finally praying for a working application. All to use a tool whose principles I barely understand (yesterday I couldn't spell magnetometer...). And who (as a generic user) would have guessed that someone could re-purpose the physical compass in a smart phone to measure magnetic fields?
posted by sneebler at 8:40 AM on October 3, 2015
All to use a tool whose principles I barely understand
That is the key issue right there.
Object-oriented programming was supposed to reduce software development costs by promoting encapsulation and re-usability. But the trouble with software development is that, being largely unconstrained by physics, encapsulation and re-use is a pattern that allows systems to grow to the point where nobody can understand how the software actually works any more.
The typical moderate-sized software project in 2015 has so many layers of abstraction between the coder and the machine that most coders not only do not but cannot understand what the code they write actually does. Which, in turn, means that defects creep into every project at every level in the abstraction stack purely because nobody knows enough to look for them. And once you have a sufficiently tall abstraction stack - and every moderate-sized project in 2015 does - your project will get into a state where bugs never get fixed; they merely move around.
Every time you sit down to write some code, there's a chance you're going to introduce a bug. As project complexity grows, two things happen: (1) the sheer size of the codebase guarantees that the number of bugs it contains will be relatively large; (2) programmers fixing bugs in one part of the codebase have a greater chance of either introducing new bugs in other parts that depend on the part they're fixing, or creating new bugs by exercising lower layers in ways not considered by their original designers.
You can hold the consequences of this truth at bay for a while by going in hard with unit tests, to help find regressions introduced by apparently unrelated bug fixes. But as the number of test cases grows, so does the likelihood of bugs in the test cases themselves.
When I started in the software industry in the early Eighties, machines were not particularly capable and we expected very little of them compared to what's normal today. In 2015, we're routinely using computers easily a million times as capable as those I started out with. We expect much more of them, but do they work a million times better than the Apple II I had at the age of 17? I don't believe they're anywhere close to that.
In 2015, computing devices are still seen by the vast majority of people as mysterious and magical, just as they were in 1979. And in 2015 there is still a minority of people with tech skills who keep the bastard things working for everybody else. The difference is that in 1979, most of that minority had a far deeper degree of insight into what the machines were actually doing than most of today's tech priesthood does. Even folks as smart as Ray Kurzweil have found reason to give up and start treating IT as something essentially supernatural.
I don't think it is. I don't think our brains are, either. What I do think is that when we do eventually get access to machinery of the same order of complexity as the human brain, we're going to find out that fixing it is every bit as difficult, ad-hoc and time-consuming a process as fixing brains is. Anything complicated enough to contemplate fixing itself is too complicated to fix.
posted by flabdablet at 10:29 PM on October 4, 2015 [4 favorites]
That is the key issue right there.
Object-oriented programming was supposed to reduce software development costs by promoting encapsulation and re-usability. But the trouble with software development is that, being largely unconstrained by physics, encapsulation and re-use is a pattern that allows systems to grow to the point where nobody can understand how the software actually works any more.
The typical moderate-sized software project in 2015 has so many layers of abstraction between the coder and the machine that most coders not only do not but cannot understand what the code they write actually does. Which, in turn, means that defects creep into every project at every level in the abstraction stack purely because nobody knows enough to look for them. And once you have a sufficiently tall abstraction stack - and every moderate-sized project in 2015 does - your project will get into a state where bugs never get fixed; they merely move around.
Every time you sit down to write some code, there's a chance you're going to introduce a bug. As project complexity grows, two things happen: (1) the sheer size of the codebase guarantees that the number of bugs it contains will be relatively large; (2) programmers fixing bugs in one part of the codebase have a greater chance of either introducing new bugs in other parts that depend on the part they're fixing, or creating new bugs by exercising lower layers in ways not considered by their original designers.
You can hold the consequences of this truth at bay for a while by going in hard with unit tests, to help find regressions introduced by apparently unrelated bug fixes. But as the number of test cases grows, so does the likelihood of bugs in the test cases themselves.
When I started in the software industry in the early Eighties, machines were not particularly capable and we expected very little of them compared to what's normal today. In 2015, we're routinely using computers easily a million times as capable as those I started out with. We expect much more of them, but do they work a million times better than the Apple II I had at the age of 17? I don't believe they're anywhere close to that.
In 2015, computing devices are still seen by the vast majority of people as mysterious and magical, just as they were in 1979. And in 2015 there is still a minority of people with tech skills who keep the bastard things working for everybody else. The difference is that in 1979, most of that minority had a far deeper degree of insight into what the machines were actually doing than most of today's tech priesthood does. Even folks as smart as Ray Kurzweil have found reason to give up and start treating IT as something essentially supernatural.
I don't think it is. I don't think our brains are, either. What I do think is that when we do eventually get access to machinery of the same order of complexity as the human brain, we're going to find out that fixing it is every bit as difficult, ad-hoc and time-consuming a process as fixing brains is. Anything complicated enough to contemplate fixing itself is too complicated to fix.
posted by flabdablet at 10:29 PM on October 4, 2015 [4 favorites]
I have never ever heard a Node developer say, "Oh, that sucks cuz it's old!" Have you ever talked to these people? Do you imagine they're all Poochie from the Simpsons? They generally love the shit of out Unix and the Unix philosophy.No, it's usually a more subtle derision of older tech. Or just assuming that a project is dead because it hasn't had a release in a month (which, in the eyes of people that believe that, effectively skips any software that could be considered polished and done.)
And yes, many of them do love the Unix philosophy to the point of religious zeal. Which in and of itself can be a problem.
At some point, someone said the same shit about Java. Goddamn Java hipstersIncluding me, back many years ago when people were arguing that it was the way of the future for desktop applications and I was writing in C (mostly). I don't feel like I was on the wrong side of that one either.
Node is a reasonable solution to certain sets of problems. The problem is it's currently "sexy", so -- just like Java c. the late 90s, early 00s -- it gets used as a solution for all sorts of stuff that it is a very, very poor fit for.
posted by -1 at 5:25 PM on October 23, 2015 [1 favorite]
in the eyes of people that believe that, effectively skips any software that could be considered polished and done
One under-appreciated effect of inexorably rising expectations is the reduction in the amount of software that could be considered polished and done to approximately none. In 2015, nobody with a clue chooses to hang the success of their business on software that isn't under active development; it's widely understood that any such choice will eventually bite back hard.
The flip side of this is the hidden assumption that change and progress are identical by definition.
Progress was once held to be inevitable: a kind of ultimately benign juggernaut. In 2015, it looks to me like a juggernaut with millions of dedicated followers of fashion churning the ground under its wheels into a sucking quagmire.
posted by flabdablet at 8:10 PM on October 23, 2015 [2 favorites]
One under-appreciated effect of inexorably rising expectations is the reduction in the amount of software that could be considered polished and done to approximately none. In 2015, nobody with a clue chooses to hang the success of their business on software that isn't under active development; it's widely understood that any such choice will eventually bite back hard.
The flip side of this is the hidden assumption that change and progress are identical by definition.
Progress was once held to be inevitable: a kind of ultimately benign juggernaut. In 2015, it looks to me like a juggernaut with millions of dedicated followers of fashion churning the ground under its wheels into a sucking quagmire.
posted by flabdablet at 8:10 PM on October 23, 2015 [2 favorites]
Nods hipsters are absolutely the worst thing about Node, but it remains a very good thing despite them.
posted by Artw at 8:12 PM on October 23, 2015 [1 favorite]
posted by Artw at 8:12 PM on October 23, 2015 [1 favorite]
As I mentioned earlier, I'm a "mature" developer currently looking for my next gig. Alot of places I've spoken to have gone Ruby/Rails or Node.js. Just this week I was speaking to the HR person of a larger global digital house and in discussing technology stacks, the HR person effectively said that they'd gone into the newer technologies in large part to attract developers who were on the leading edge and would be attracted by a place 'hip' enough to be into the newer stuff.
So, yeah. The only constant is change. I probably have to learn yet another paradigm and framework. #6 for me, I think. And a whole new palette of idiosyncracies and problems to solve. Good news is that many places are also saying that they're open to any good dev and they'll help them pick it up; one place has even specified that formal training in their CMS of choice is part of the ramp-up after hiring.
posted by Artful Codger at 12:50 PM on October 24, 2015 [1 favorite]
So, yeah. The only constant is change. I probably have to learn yet another paradigm and framework. #6 for me, I think. And a whole new palette of idiosyncracies and problems to solve. Good news is that many places are also saying that they're open to any good dev and they'll help them pick it up; one place has even specified that formal training in their CMS of choice is part of the ramp-up after hiring.
posted by Artful Codger at 12:50 PM on October 24, 2015 [1 favorite]
« Older Stealing All My Dreams | Fifty years and two documentaries since the... Newer »
This thread has been archived and is closed to new comments
posted by Artw at 5:17 AM on October 1, 2015 [7 favorites]