A Better Way to Code – Mike Bostock – Medium

May 1, 2017 2:39 PM

It’s great that journalists and scientists are sharing data and code. But code on GitHub is not always easy to run: you need to reproduce the necessary environment, the operating system, the application, the packages, etc. If your code is already running in the browser, it runs in any other browser; that’s the beauty of the web. Mike Bostock writes 4300 words for Medium (with LOTS of pretty pictures).
posted by cgc373 (61 comments total)

And lo, BASIC has returned.
posted by grumpybear69 at 2:46 PM on May 1, 2017 [5 favorites]

(It’s perhaps worth acknowledging up top that this is Bostock pitching d3.express. Which is fine! But this isn’t a purely academic take - It’s an advertisement of a solution that he is developing.)
posted by Going To Maine at 2:47 PM on May 1, 2017 [7 favorites]

This is why the Shelx software package is coded with the "0 Dependency" philosophy. Every bit of code needed to run it is already included. You point just about any Fortran compiler and it, and it will work on the target OS, and has for a very, very long time. (It was first written in the 60s).

I'm not sure why the idea of highly portable code is a new one. Is it really that hard to make installed software portable?
posted by Canageek at 3:02 PM on May 1, 2017 [1 favorite]

If your code is already running in the browser, it runs in any other browser

lolz
posted by thelonius at 3:07 PM on May 1, 2017 [73 favorites]

> I'm not sure why the idea of highly portable code is a new one. Is it really that hard to make installed software portable?

If you want to do math, string and data manipulation that outputs to a text file? No, it's not that hard.

If you want the sort of interactive graphical media that people have been taking for granted for the past couple decades, using only the platform portable tools currently available, then yeah, it gets trickier.
posted by ardgedee at 3:10 PM on May 1, 2017 [14 favorites]

/me checks n-gate.com for it
/me disappointed
posted by runcifex at 3:11 PM on May 1, 2017 [3 favorites]

And incidentally the road to hell is lavishly paved with already well-known examples.
posted by ardgedee at 3:11 PM on May 1, 2017 [14 favorites]

Agreed 100% with argedee, and even with tools that are cross platform and compatible (like web browsers), it's a constant fight between cutting edge features / fashionable design (which are always using incompatible and incidental features of each platform) and compatibility. And if nobody prioritizes compatibility features and fashion win.
posted by idiopath at 3:19 PM on May 1, 2017 [1 favorite]

Containers make this a lot easier.

Pull repo for container. Build image, create container from image, run container.

Container realizes it doesn't have an app, so it pulls the app repo, make, make test, make install, ./kick-hypnotoad.sh

Then on another computer checkout the test suite, and run that.

Easy peasy
posted by mikelieman at 3:23 PM on May 1, 2017 [2 favorites]

>I'm not sure why the idea of highly portable code is a new one.

It's not. It's an ancient dream, taking another turn on the Wheel of Technical Reincarnation.

Previous incarnations include COBOL-60, NEGFFBIBM* and FORTRAN-66 (whose union begat your beloved SHELX), p-code, ISO programming language standards, and JAVA.

>Is it really that hard to make installed software portable?

Yes. It's definitely an unnatural act.

*Nobody Ever Got Fired For Buying IBM
posted by the Real Dan at 3:35 PM on May 1, 2017 [8 favorites]

Yeah, of those 4300 words, 342 of them are in the section that relates to the pullquote in the FPP. The actual topic of this article is "hey look I started developing yet another reactive visualization framework, come help me work on it!" I'm wondering if the timing of this announcement relates at all to Airbnb's announcement two weeks ago.

Meanwhile it looks like he's proposing enhancements to js syntax (while ignoring ES6) in a way that kind of reminds me of when people decided perl wasn't DOM-y enough, resulting in the abortion that is PHP.

If you really want the ecumenical in-browser development environment described in the pullquote, go hang out with the WebAssembly project instead.
posted by 7segment at 3:40 PM on May 1, 2017 [5 favorites]

/me checks n-gate.com for it
/me disappointed

This blog post was posted to HackerNews 3 days ago and got like 3 comments.

Also the "doing shit right in the browser" is tired as hell, but I guess it's finally tricking down into the non-technical sites like MeFi.
posted by sideshow at 3:42 PM on May 1, 2017

Medium offers no editing services.
posted by boo_radley at 3:45 PM on May 1, 2017

See also: your local neighborhood JavaScript browser console.
posted by grumpybear69 at 3:48 PM on May 1, 2017 [1 favorite]

There was that medium post that went around a little while ago about how Electron is the worst, so perhaps we can consider this part of the blowback .
posted by Going To Maine at 4:02 PM on May 1, 2017 [1 favorite]

Webapp is so 2017.

Real hipster runs docker.js in browser with debian-image.json.
posted by runcifex at 4:10 PM on May 1, 2017 [8 favorites]

Guess who's heard "C is cross platform" as an argument in the last couple months. In reference to the front end for a web app.
posted by The Gaffer at 4:28 PM on May 1, 2017 [9 favorites]

Yeah, we've got a C weirdo at our shop. He's just mentioned it a couple of times so far so I'm waiting to see where he's going with it.
posted by Artw at 5:06 PM on May 1, 2017 [3 favorites]

I can't remember exactly which talk I was watching over the weekend that introduced me to this, but it made oi laff. And here we are.
posted by Devonian at 5:08 PM on May 1, 2017 [8 favorites]

could .cgi *anything*

I believe the dating site Plenty of Fish was C or C++ and the efficiency of the code allowed the site to run much more economically than most contemporary interpretive frameworks.
posted by sammyo at 6:41 PM on May 1, 2017

I maintain a very high traffic web site that is built in C (inherited legacy thing... I am not a C weirdo). It is a terrifying monster that exists only to devour web requests and spit up HTML. I was benchmarking a PHP framework a few months ago and Apache would become unresponsive at around the 8,000 requests/min mark. But C just kept on pumping up through 10,000 req/min. I don't really even know the ceiling for it as I've never tried to max it out.

But oh my God does it suck to update. When a dev comes to me to say 'Hey you know that cookie we rely on everywhere? Well over on this PHP/JS part of the site, we are updating it to look like ' I then sigh and say I'll have to go look at the memory allocation for that var, re-compile, and re-deploy.

Thankfully it looks like we'll just throw more hardware at things and retire C in favor of an interpreted language later this year. I'd take ease of maintenance at this point over speed.
posted by boubelium at 6:45 PM on May 1, 2017 [7 favorites]

boubelium, please give elixir a look. It's not perfect, but such a relief.
posted by The Gaffer at 6:58 PM on May 1, 2017 [3 favorites]

If you want to do math,

Pretty sure that you can't really, really do math portably. Integers are probably mostly well behaved but FP? I'm not smart enough to know that here be dragons, but I'm smart enough to know... here be dragons.
posted by wotsac at 7:06 PM on May 1, 2017

FP? I'm not smart enough to know that here be dragons, but I'm smart enough to know... here be dragons.

Plenty o' dragons, which is why they came up with a standard more than 30 years ago.

Say what you will about a language where every number is a float, but at least they're standardized floats.
posted by neckro23 at 7:25 PM on May 1, 2017 [3 favorites]

Seems like a lot of the snarky comments here are from legit developers, but I'm not sure that this is supposed to be for you lot. As someone who works in data exploration and analytics space but is not that talented a coder, this looks very cool and exciting. I think Bostock may mostly be targeting data journalism and PhD types for whom this is a simpler, more flexible version of Jupyter notebooks.
posted by Aizkolari at 7:26 PM on May 1, 2017

Still looks a lot more complicated than RStudio / RMarkdown notebooks to me, but then I'm incredibly, incredibly biased.
posted by Jimbob at 7:30 PM on May 1, 2017 [1 favorite]

Pretty sure that you can't really, really do math portably. Integers are probably mostly well behaved but FP?

That's applied math.
posted by escabeche at 7:42 PM on May 1, 2017 [3 favorites]

Aizkolari: "I think Bostock may mostly be targeting data journalism and PhD types for whom this is a simpler, more flexible version of Jupyter notebooks."

Any world that asks these people to learn javascript is a bad one. You don't give foot-shooting guns to newbies.
posted by TypographicalError at 7:43 PM on May 1, 2017 [6 favorites]

Any world that asks these people to learn javascript is a bad one.

Yep. Data is something you want to do things with. Simple imperative/functional paradigms work great. Object oriented stuff adds an unnecessary layer of complexity. Going on to force an event-driven language full of callback functions and such rubbish distracts hugely from the task at hand.

Reactive programming can also be fun, but dealing with scientific data every day, the number of times I've actually found a use for the reactive abilities of, for example, R Shiny, is tiny. And I usually have to short-circuit all its automation to get it to do what I need anyway.
posted by Jimbob at 8:09 PM on May 1, 2017 [1 favorite]

I believe the dating site Plenty of Fish was C or C++ and the efficiency of the code allowed the site to run much more economically than most contemporary interpretive frameworks.

It wasn't in 2009 and it feels very, very unlikely that they'd have moved from ASP.net to C/C++. Parts of Amazon were written in C++ as well as early Yahoo!, IIRC.

Compiled code can be more efficient on servers than interpreted languages or CLRs but very often the cost of programmers that can safely wield C++ online is significantly greater than the cost of just adding more machines. Safer languages are often faster to iterate in and speed to market can be key. Ruby on Rails is notoriously low performance but runs quite a few important sites because you can get an application running quickly and easily with more or less commodity programmers.

C/C++ can also have severe security vulnerabilities from a typo and the internet provides a pen test pretty much 24/7 to any site where the performance gains compared to Java or .Net would matter.

At this point, if you need intense performance with a compiled language, you're going to be reaching for Go, which is designed for the modern internet from the ground up.
posted by Candleman at 8:50 PM on May 1, 2017 [4 favorites]

You guys, d3 is a genuinely good and useful platform for visualisations, and anything Bostock says is worth taking seriously. If I wanted this level of coder-snarking I'd read HN.
posted by mikelynch at 8:55 PM on May 1, 2017 [17 favorites]

I think Bostock may mostly be targeting data journalism and PhD types for whom this is a simpler, more flexible version of Jupyter notebooks.

Yup. d3 squarely focuses on this space (Bostock is one of the founders of d3), so people snarking about him are not exactly coming from an informed place. It's not at all meant to be a general programming panacea, but for what it does it's really a lovely thing.
posted by vanar sena at 9:59 PM on May 1, 2017

Well, if you title your essay "A Better Way to Code" you had better have something pretty impressive or you're just painting a snark target on yourself.
posted by Pyry at 10:02 PM on May 1, 2017 [9 favorites]

"You don't give foot-shooting guns to newbies."

All guns are foot-shooting guns, unfortunately.
posted by el io at 10:11 PM on May 1, 2017 [9 favorites]

Yeah, when it comes over to mefi without the background context, it is indeed a nebulous title. Taken within the context of the rest of Bostock's Medium blog, which is almost entirely about data visualisation and analysis with d3.js, the title is unsurprising.

You don't give foot-shooting guns to newbies.

I'm afraid that ship sailed with VisiCalc.
posted by vanar sena at 10:17 PM on May 1, 2017 [3 favorites]

D3, of course, is beautiful, and powers a lot of Jupyter stuff I do to make a living. I normally dismiss any medium post about Javascript as being for those weird front-end people, but this caught my attention. In the early years of my career, I "invented" a similar reactive framework for a Python CAD program (Invented in quotes because I lacked the perspective to realize how many times this wheel has been invented). I remember fondly how glorious it is to nudge a control and watch the shapes on the screen reconfigure themselves, recomputing only that slice of the dependency graph defined by the intersection of things that depend on the altered input and those things that might possibly affect an output rendered on screen. That was really fun.
posted by qxntpqbbbqxl at 11:16 PM on May 1, 2017

d3 is a genuinely good and useful platform for visualisations

Agreed.

anything Bostock says is worth taking seriously.

Does not follow from premise.

If I wanted this level of coder-snarking I'd read HN.

MeFi should potentially look into limiting the degree of crossposting from there as they have with /., k5, digg, reddit, and similar sites over the years.
posted by 7segment at 11:25 PM on May 1, 2017 [2 favorites]

The close parallels I can think of to this are Light Table and nteract/hydrogen. None of these are runnable in the browser, though that’s certainly not impossible - it’s just a little further away than using JavaScript directly.

I think my only real complaint about all of this is that there remains no universal data tool, so going all “let’s learn JavaScript!” makes me sad. But then, lots of the hip new viz tools (Altair, Vega, etc.) are based around javascript, so perhaps we should just settle for it as a sub-optimal, universal standard that can be used directly in a browser.

(That said, Bostock’s interests have been beautiful, interactive content. And like other folks, I wonder about the universal merits of interactivity here. Further, I wonder if a few years from now we’re going to have to all get on board the good ship Julia. Perhaps the forward-thinking thing would be to design new visualization tools specifically for it.)
posted by Going To Maine at 11:33 PM on May 1, 2017 [1 favorite]

Unfortunately, even in the WebAssembly-everywhere future, Javascript's going to have a leg up simply because it will be the only language to have an interpreter shipped with the browser. Everyone else is likely going to have to distribute a runtime along with their app.

OTOH it may compel webassembly runtimes to shrink to within the size range of the huge JS bundles used by apps these days, who knows.
posted by vanar sena at 11:57 PM on May 1, 2017 [1 favorite]

MeFi should potentially look into limiting the degree of crossposting from there as they have with /., k5, digg, reddit, and similar sites over the years.

If Hacker News is the only place you can imagine coming across this piece, you either aren't interested in the topic, or you should spend some time expanding your reading sources instead of snarking about it.
posted by the agents of KAOS at 12:33 AM on May 2, 2017 [1 favorite]

I can imagine coming across it plenty of other places. I was not the one to bring up hacker news.
posted by 7segment at 12:41 AM on May 2, 2017

Every time somebody responds to "here's something in Javascript" with "Webassembly is better", I hear: "I heard you like to ride bicycles, so I put a truck factory in your back yard. Trucks are much better than bicycles, and this lets you make any truck you want, after you've hired the labor, sourced the supply chains, designed a vehicle and engineered its components."
posted by ardgedee at 3:27 AM on May 2, 2017 [8 favorites]

This is neat, but I feel like it falls into an uncanny valley where the people who need the interface won't understand the code, and the people who understand the code won't need the interface.
posted by empath at 4:37 AM on May 2, 2017 [2 favorites]

This is neat, but I feel like it falls into an uncanny valley where the people who need the interface won't understand the code, and the people who understand the code won't need the interface.

This and the other 'lol, people with PhDs and journalists' comments kind of feel like they're coming from people who have neither used d3, nor know what sort of thing the users of d3 tend to do. d3 is pretty ineffable and the fastest (sometimes only) way to get anything done is to hack away at one of Bostock's examples. Lo and behold that requires... knowing something about Javascript. And then there's the ubiquity of d3 + angular + flask. It seems somehow plausible that the users "understand the code".
posted by hoyland at 5:12 AM on May 2, 2017

Now, there is an argument that some of what makes d3 ineffable is where it deviates from standard Javascript (I seem to recall d3.min and d3.max behave subtly differently to what you'd expect). See also someone's comment above ignoring ES6.
posted by hoyland at 5:15 AM on May 2, 2017 [2 favorites]

This month one of my tasks is a web GUI control which is basically an on-the-fly pie chart type of visualization, but the slices are selectable and the selected slice gets a handle on one edge that can be dragged to alter its position (and a corresponding data object).

I found several "charting" JS libraries but none with the interactivity I wanted, and they didn't look very extensible. I'm now using a combination of fabric.js plus a custom event handler.

I am a JS hater... mostly it's just a personal dislike, but I particularly don't like how stuff has grown around it like cancer, few of these libraries interoperate that well, and there's a new damn paradigm every two years. /old guy rant
posted by Artful Codger at 6:36 AM on May 2, 2017 [2 favorites]

I don't see where this is using non-ES2015 syntax, though he does have some of the new hotness like generators in there.
posted by whir at 7:00 AM on May 2, 2017

 → Unfortunately, even in the WebAssembly-everywhere future, Javascript's going to have a leg up simply because it will be the only language to have an interpreter shipped with the browser.

So JS is the modern equivalent of Fortran, which the Unix/compsci types hated but every new system used to ship with a Fortran compiler because you could do paid work with that language. So the Unix/compsci held their noses and wrote hacky bootstraps in Fortran to get their C compilers hosted and thus spread the virus of slow and overheating computers worldwide …

Bostock does lovely things, but it's a shade rich of him to go "code reuse is the best" when associated bostockiana crows about him creating a whole new JSON dialect (ndjson) and writing all his new command line tools to use (only) it. Cynical me thinks he'd have saved a bunch of time and effort learning to use GDAL rather than creating tonnes of code and new, lightly-tested data processing tools.

I guess it just proves that code has little value to write, yet it can do a bunch of damage to data and no-one bats an eyelid. Data holds the real value.
posted by scruss at 8:11 AM on May 2, 2017 [4 favorites]

hoyland: "This and the other 'lol, people with PhDs and journalists' comments kind of feel like they're coming from people who have neither used d3, nor know what sort of thing the users of d3 tend to do. d3 is pretty ineffable and the fastest (sometimes only) way to get anything done is to hack away at one of Bostock's examples."

I haven't used d3, but this isn't d3. It's d3 express, which is meant to be for data analytics (apparently). Now, I have done lots of data analytics, and I've been unfortunate enough to do it in node for a period. Maybe this visualization is the hottest thing that's ever existed, but it doesn't matter because 99% of data exploration is munging your data, and Javascript is awful for data manipulation. So offering that this is "just like a jupyter notebook" is really disingenuous--Javascript will fuck you hard in many different ways if you aren't being super paranoid about it. It's not contempt for people to understand that Javascript is full of nasty corners, especially when you do data work.
posted by TypographicalError at 10:08 AM on May 2, 2017 [6 favorites]

I haven't used d3, but this isn't d3. It's d3 express, which is meant to be for data analytics (apparently). Now, I have done lots of data analytics, and I've been unfortunate enough to do it in node for a period. Maybe this visualization is the hottest thing that's ever existed, but it doesn't matter because 99% of data exploration is munging your data, and Javascript is awful for data manipulation. So offering that this is "just like a jupyter notebook" is really disingenuous--Javascript will fuck you hard in many different ways if you aren't being super paranoid about it. It's not contempt for people to understand that Javascript is full of nasty corners, especially when you do data work.

As always, https://www.destroyallsoftware.com/talks/wat , starting at 1:26
posted by The Gaffer at 12:08 PM on May 2, 2017

You did not link your URL, the text denies that the speaker is responsible for their own their words and I can't watch the video.

So explicitly stating the point you would like to make would be awesome, thanks!
posted by ardgedee at 2:32 PM on May 2, 2017

Copy and paste are great, the text is not entirely earnest, and the video illustrates that javascript does indeed have some surprising pitfalls. I am trying to make the point that this is a fun video to watch, with some pertinence to the quoted comment.
posted by The Gaffer at 2:54 PM on May 2, 2017 [1 favorite]

(Oh, as another data point here, we should probably consider Bokeh I’d say it remains ugly as sin and a reminder of why interactivity is often not what one wants, but it does give you interactive javascript in a jupyter notebook - though it always seems pretty dang janky, and I don’t think it plays well in a rendered github page. (An example app, not in a notebook.)
posted by Going To Maine at 8:40 PM on May 2, 2017

I am a JS hater... mostly it's just a personal dislike, but I particularly don't like how stuff has grown around it like cancer, few of these libraries interoperate that well, and there's a new damn paradigm every two years. /old guy rant

I totally feel this, to the extent that I purposely avoided writing anything larger than a few lines of JS in the last twenty years. That changed recently when I found myself having to prototype a new system as an SPA. After a few months of cursing the name of anyone who ever released a piece of javascript tooling, I'm at a spot now with Vue.js/Vuex and d3 where I'm grudgingly enjoying myself. I suspect this is the best I can hope for with the current generation of browser dev tools.
posted by vanar sena at 11:34 PM on May 2, 2017

It was linked here last October, but perhaps it’s still relevant: “How it feels to learn JavaScript in 2016”
posted by Going To Maine at 12:11 AM on May 3, 2017 [3 favorites]

Reading that thread, I've decided we need to have this discussion here once a year, just to keep a record of the 20% of frameworks that have succumbed to fashion in that period.
posted by vanar sena at 12:44 AM on May 3, 2017 [2 favorites]

It was linked here last October, but perhaps it’s still relevant:

How could anything possibly still be "relevant" after six months?
posted by thelonius at 3:51 AM on May 3, 2017 [1 favorite]

How could anything possibly still be "relevant" after six months?

Bourbon is still relevant, if not more so.

( Locally, the price of a handle of Wild Turkey 101 has dropped over 3 dollars since then. )
posted by mikelieman at 4:53 AM on May 3, 2017 [1 favorite]

Not to abuse the edit window. Shoot, I thought this was the politics thread window, but given the rapid change in application frameworks, yea, it's an appropriate comment.
posted by mikelieman at 4:54 AM on May 3, 2017

bourbon is no longer relevant in my crib, I'm afraid
posted by thelonius at 6:13 AM on May 3, 2017

bourbon is no longer relevant in my crib, I'm afraid

So, you gave up development? Good for you. Not a day goes by I don't dream of being back in the pretzel stand. "You want a coke with that?" being the extent of my responsibilty
posted by mikelieman at 6:28 AM on May 3, 2017