Playing with Dark Magic

January 18, 2011 4:00 PM

What the Heck is Shadow DOM? Browser developers realized that coding the appearance and behavior of HTML elements completely by hand is a) hard and b) silly. So they sort of cheated. They created a boundary between what you, the Web developer can reach and what’s considered implementation details, thus inaccessible to you. The browser however, can traipse across this boundary at will.
posted by netbros (38 comments total)

WARNING: Pseudocode, not a real API.
posted by Artw at 4:08 PM on January 18, 2011

I need a bit more to go on. I read the first half, skimmed the rest. What's the tweet version take away on why this is cool?
posted by cjorgensen at 4:27 PM on January 18, 2011

As far as i can figure out... some browsers render form elements as nested HTML? But don't give you a way to mess with it? And he thinks it would be a good idea to mess with it because ???

/shrugs. Possibly I am tired right now, because people are responding in comments as if he has written something reasonable and not the timecubey mess I am reading it as.
posted by Artw at 4:32 PM on January 18, 2011 [3 favorites]

From the article: "The ability to write your widget and not worry about some random selector fiddling with your style seems … downright intoxicating."

Actually, it seems exactly the opposite, more of a nauseating "this is why it takes five revisions in Photoshop to show off our custom <select> widget".

I appreciate what all the browser developers are doing with regards to native widgets, but this battle was already fought and more-or-less lost after the days of having custom scrollbar colors in IE, Netscape 4, etc.

The moment you start sandboxing UI away from the designer/integrator is the moment you start getting hideous flash scrollbar widgets that serve little purpose other than to confuse and distract your users. I like the ability to style my "Post Comment" button to be yellow, with a little two-pixel bevel. Taking that away from me by only allowing some attributes to be styled via a "Shadow DOM" is basically saying, "well, go ahead and create <input type="image"> once again because you can't do what you really want."

If the author of the article is worried about "some random selector fiddling with your style" then whose fault is it really? The designer who puts in <style> i { font-style: normal; } </style> or the widget maker? HTML, and now CSS, has always been a "test visually" language, almost an np-problem, so it makes more sense to allow the maximum amount of visual styling possible.
posted by mark242 at 4:33 PM on January 18, 2011 [4 favorites]

Oh, that's what he's trying to do? yeah, that's a no.
posted by Artw at 4:39 PM on January 18, 2011

Are we talking about 2 spaces after a period again?
posted by humboldt32 at 4:40 PM on January 18, 2011 [1 favorite]

With the exception of SVG (more on that later), today’s Web platform offers only one built-in mechanism to isolate one chunk of code from another — and it ain’t pretty. Yup, I am talking about iframes. For most encapsulation needs, frames are too heavy and restrictive.

Yeah... this guy has no idea what he's talking about. Javascript is an incredibly flexible language and you can do most OO things with it, although it's true that it doesn't have private members... there's a huge difference between actual secure isolation between components (which is what iFrames give you) and encapsulation.

Basically, encapsulation is a way of preventing programming mistakes. If you prevent code from accessing parts of other objects then you can change the way an object is implemented in the future, while being sure that your other code won't break. And, you can have different implementations at the same time.

The only difference between Javascript and something like Java and C++ is that Javascript won't catch your mistakes for you, being a weakly typed language. But there are ways to 'cheat' the type system in both Java and C++ if you want. They are not for actual security (In theory, you can't cheat in java if your running code from an untrusted security context). But if you do cheat you have to do something 'weird', something that would be unlikely to be used if in normal development.
posted by delmoi at 4:47 PM on January 18, 2011

Taking that away from me by only allowing some attributes to be styled via a "Shadow DOM" is basically saying, "well, go ahead and create once again because you can't do what you really want."

speaking as a user, I'd really rather you gave up the fantasy that you (designer) get to have complete control over something that is ultimately happening on my computer, and just let the browser draw normal widgets the way the native OS wants to draw them.
posted by Mars Saxman at 5:04 PM on January 18, 2011 [25 favorites]

Preventing web developers from making sites that drive users away due to their miserable overdesign is not the purpose of a web browser. In fact, this tends to be a problem that moderates itself without intervention.
posted by ardgedee at 5:24 PM on January 18, 2011 [2 favorites]

Man, I have enough troubles just getting divs to look the same across browsers. What is this guy on about?

speaking as a user, I'd really rather you gave up the fantasy that you (designer) get to have complete control over something that is ultimately happening on my computer, and just let the browser draw normal widgets the way the native OS wants to draw them.

Everyone just needs to relax and go back to what they were doing. This is a non-starter.
posted by dubitable at 5:37 PM on January 18, 2011 [1 favorite]

Wait, so if I spend an extra $10k and tinker with the engine of my consumer market automobile and blow it out drag racing at the old airstrip on a Friday night, my warranty is voided? What about the spoiler and the running lights?
posted by jsavimbi at 6:05 PM on January 18, 2011

although it's true that it doesn't have private members

That's not true at all. All variables declared within the scope of an object's construction are private, accessible only to other privately-scoped functions in the same object.

function MyObject() {
  var private = 0;
  this.public = 1;
}
var temp = new MyObject();
alert(temp.private); // undefined
alert(temp.public); // 1

posted by Civil_Disobedient at 6:19 PM on January 18, 2011

speaking as a user, I'd really rather you gave up the fantasy that you (designer) get to have complete control over something that is ultimately happening on my computer, and just let the browser draw normal widgets the way the native OS wants to draw them.

Yup, that pretty much sums it up.
posted by thsmchnekllsfascists at 6:29 PM on January 18, 2011

Not sure what he expects, at a base level input controls are just windowless controls. For a long time people bumped into the fact that select was not implemented as a windowless control, It was impossible to place a div over a visible select control, the select control always displayed on top
posted by Ad hominem at 6:54 PM on January 18, 2011 [1 favorite]

Oh , a bit farther afield but it is possible to use silverlight in windowed and windowless mode. Windowed mode is faster, in windowless mode you can position DIVs over the plugin.
posted by Ad hominem at 6:57 PM on January 18, 2011

speaking as a user, I'd really rather you gave up the fantasy that you (designer) get to have complete control over something that is ultimately happening on my computer, and just let the browser draw normal widgets the way the native OS wants to draw them.

See also the way IGN webpages must be navigated by the mouse, since the keyboard browser controls mysteriously do nothing or return you to the top of the page.
posted by Pope Guilty at 7:28 PM on January 18, 2011 [2 favorites]

The moment you start sandboxing UI away from the designer/integrator is the moment you start getting

...consistent and clear interfaces that "designers" and "integrators" can't repeatedly fuck up.
posted by eriko at 7:39 PM on January 18, 2011 [2 favorites]

Abstraction layers... abstract things?
posted by jeffamaphone at 8:24 PM on January 18, 2011

Civil_Disobedient:

That's not true at all. All variables declared within the scope of an object's construction are private, accessible only to other privately-scoped functions in the same object.

I would submit that is not "private" in the sense that most object-oriented frameworks would understand the term. To declare "var private" in that function would mean that the private variable does not exist anyplace but that function and any code directly contained therein. In most OO languages, a private variable is one that may be accessed only by class methods which is not the case in your example for javascript.

As someone who does a fair share of programming on the web, there are some things which really demand that you work with the DOM in javascript (the "shadow" DOM, as it were). It's the age-old problem: computers are meant to automate things. Maybe it feels more open to hack some things by hand, but after a while I imagine most wonder to themselves if they aren't missing the point.

That is not to say anything about people deliberately hiding HTML that they very well could have written by hand in javascript, but you can generally ignore such people as they are clearly deranged. It may be easier to automate some code in javascript, but it's not as straightforward as just writing out some HTML if that's all you need.

A final point to make, if it hasn't already been made: isn't this a browser issue more than a bad-programmers-hide-things problem? I mean, the browser knows what's in the DOM even if it's different than what the HTML source would indicate. There's nothing that keeps it from showing you in some kind of source view what is actually in the DOM. (Doesn't Chrome do this now?)
posted by moz at 8:31 PM on January 18, 2011

I think he is wrong about the webkit video control. He states that it is HTML and CSS in a "shadow dom". Accoding to this it is based on this. Reading the "software rendering" portion of the webkit docs this seems like the QT equiv of an owner drawn control. Is he sure there is a "shadow dom" or just some QT widgets that have some properties accessible to the browser.

Any QT developers around? Webkit is a branch of kthml that famously used QT right?
posted by Ad hominem at 8:40 PM on January 18, 2011

Err I mean KHTML
posted by Ad hominem at 8:41 PM on January 18, 2011

But there are ways to 'cheat' the type system in both Java and C++ if you want. They are not for actual security (In theory, you can't cheat in java if your running code from an untrusted security context). But if you do cheat you have to do something 'weird', something that would be unlikely to be used if in normal development.

Except it's my experience that sooner or later, every reasonably large Java program starts to use reflection in increasingly unusual ways, usually to implement various design patterns for dynamically creating different types of objects at runtime or to make certain JavaBeans behavior work or to handle serialization or any number of other reasons. In other words, large released Java apps "cheat" the type system all the time.
posted by zachlipton at 8:50 PM on January 18, 2011

Agreed, .net also use reflection for some very common things, such as the XmlSerializer class.
Various ORMs such as hibernate (according to the FAQ) use reflection extensively. This is all normal development, but I agree with you that if you are using reflection to poke at the internals of a component you are going to get burned.
posted by Ad hominem at 9:07 PM on January 18, 2011

He's not talking about encapsulating JavaScript, guys: he's talking about encapsulating HTML/CSS "code".

If I want to make a reusable doohickey out of HTML and CSS that gets dropped into a larger web page by somebody else (like, a date-picker or a status-of-your-shopping-cart widget), I have to worry about their CSS selectors reaching in and messing with my rendering, not to mention their JavaScript noticing that my doohickey's DOM node has all sorts of internal structure and maybe starting to rely on the specific implementation.

So then he points out that a lot of things that web designers think of as simple native controls (like that slider) are actually just DOM subtrees with some kind of encapsulation barrier to make them appear structureless from the outside. And wouldn't it be nice (he concludes) if there were some way to get similar isolation for arbitrary parts of the page?
posted by hattifattener at 9:26 PM on January 18, 2011 [3 favorites]

If I want to make a reusable doohickey out of HTML and CSS that gets dropped into a larger web page by somebody else (like, a date-picker or a status-of-your-shopping-cart widget), I have to worry about their CSS selectors reaching in and messing with my rendering

So, namespace your CSS selectors. Unless you mean 'messing with it on purpose', which is not something you're going to convince an overzealous web designer is bad.

rely on the specific implementation

It's not like you're able to provide accessors for discrete pieces of your HTML widget. It's simply the nature of the markup-language beast that the internals of your widget are exposed -- and the hacky pseudoselectors outlined in the article still allow downstreamers to become reliant on your 'shadow DOM details'... they merely have to type a few arcane symbols, from the looks of it, and you're right back where you started.

Meanwhile, someone has to write this silly API and get it implemented across the board. I'm sure that's going to come down the web standards conveyor belt even quicker than the last ten things I was promised.
posted by zvs at 9:38 PM on January 18, 2011

I would submit that is not "private" in the sense that most object-oriented frameworks would understand the term. To declare "var private" in that function would mean that the private variable does not exist anyplace but that function and any code directly contained therein. In most OO languages, a private variable is one that may be accessed only by class methods which is not the case in your example for javascript.

Spidermonkey disagrees with you:

stephen@jellyfish:~$ js
js> function foo() {
 var privateThing = 0;
 this.getPrivateThing = function() { return privateThing; }
 this.setprivateThing = function(val) { privateThing = val; }
 this.visibleThing = 2;
}
js> var bar = new foo();
js> bar.getPrivateThing();
0
js> bar.setPrivateThing(3);
js> print(bar.getPrivateThing());
3
js> print(bar.privateThing);
undefined
js> print(bar.visibleThing);
2
js> bar.visibleThing = 99
99
js> print(bar.visibleThing);
99
js> var gum = new foo();
js> print(bar.getPrivateThing());
3
js> print(gum.getPrivateThing());
0
js> print(bar.visibleThing);
99
js> print(gum.visibleThing);
2
js> ^C
stephen@jellyfish:~$ 

posted by flabdablet at 11:37 PM on January 18, 2011

Obligatory Douglas Crockford: Private Members in JavaScript
posted by teraflop at 11:37 PM on January 18, 2011 [1 favorite]

To be fair access to privateThing is limited to members function foo due to scope. The closeure that contains the foo ctor still exists when you call foo's member functions. This isn't really what people think of when they think of member variables marked private.
posted by Ad hominem at 12:11 AM on January 19, 2011

to members function foo due to scope

I mean member functions of foo.
posted by Ad hominem at 12:14 AM on January 19, 2011

Also, if you need truly custom controls, you can write them yourself using HTML+JS, or even <canvas>
posted by delmoi at 2:16 AM on January 19, 2011

If it walks like a private variable, and it quacks like a private variable, I don't see how hairsplitting about implementation details disqualifies it.
posted by flabdablet at 6:55 AM on January 19, 2011

I think most of the people commenting here are misunderstanding his point. He's not saying that authors should have more control over styling native controls. That pony has long left the stable, and if you search you can find all sorts of articles about how to write custom-styled input/select/etc elements.

The place he's coming from is the standpoint of a Javascript library author, someone writing jQuery UI or the like. As it is now, if you want to make sure that your widgets and styles don't clash with other libraries/frameworks, the best you can do is to manually namespace the IDs/classes. There is no way to prevent some other code in a complex page from walking all over the elements that your library is responsible for, making them render incorrectly. This is analogous to, but separate from, all the namespace problems that you get when you try to combine two JS libraries and they both want to override Object.prototype to do their magic.
posted by Rhomboid at 8:28 AM on January 19, 2011 [2 favorites]

I see the author's point: It would be nice if the jQuery UI Calendar I placed on my page wouldn't display like crap because I specified a generic td { background-color: gray; } style that wasn't intended to apply to the packageable widgets I use. The "shadow DOM" is basically way for a DOM element to say "do not cascade styles down this branch of the tree".
posted by The Lurkers Support Me in Email at 8:37 AM on January 19, 2011

The day jQuery UI starts doing this is the day I drop jQuery UI.
posted by Artw at 10:45 AM on January 19, 2011

And doing something like td { background-color: gray; } is just asking for trouble - even without JQuery UI. If you're putting a lot of specific styling in selectors for things like TD, DIV and SPAN I'd be surprised if you're not having to put in classes for all kinds of special cases that have a bunch of !important stuff in them, even before any UI libraries.
posted by Artw at 10:48 AM on January 19, 2011

hattifattener and Rhomboid have it: He's talking about extending the same sort of privilege that browser makers have in being able to protect the appearance of their elements out to framework authors.

For every ArtW there are 20 HTML authors who ask for trouble. Wouldn't it be nice if a framework could meet them halfway, offering a set of widgets as protected from inadvertent tampering as the OS elements are?

I realise the answer is of course "fuck off, they should drop everything and spend 9 years in Tibet learning how to do it properly before daring to knock up their quick and dirty" which is the attitude that will also ensure this will never happen.
posted by bonaldi at 12:54 PM on January 19, 2011

I think that'd be a case of a solution that's far, far worse than the problem when there is an abundance of other solutions available, and that won't even cure all of the problems you are likely to be caused if you are insistent on coding like that.
posted by Artw at 3:10 PM on January 19, 2011

Artw, I get where you're coming from. Most of the time, it's a feature that you can style down as far as you like in the DOM tree and having developers seal stuff off arbitrarily would be annoying (much like it's annoying when certain stripes of OOP enthusiasts are more enthusiastic about their final and private declarations than they are about actually writing classes/objects that are so well-considered that nobody cares to subclass or poke around inside).

But there are reasons why we isolate code into modules / objects / whatever when we're writing code in Real Programming Languages (TM), and I think some of those reasons travel well into the world of collaborative styling/markup projects. If you've ever worked on, say, a big ol' hacked Wordpress Install where there are 15 different style sheets created by 20 different authors (not to mention, of course, the additional handful of inline style declarations that go on for 2-3 pages each)... it's sure easy to think that it would indeed be convenient sometimes if there were some way to isolate rulesets. Or at least say "start me over" without having to re-specify a rule every last default for the browser all over again, or having to pour !important all over everything, or start adding additional selectors for the sole purpose of manipulating specificity weights.

So, maybe something like:

div.someclass#myspecialdombranch {
   default-ruleset: browser;
}

In essence, a "reset" for a given branch of the DOM tree.

There's some question in my mind whether you'd really want a "protect me from all outside delcarations" kind of rule, or just a "start over" (the former is the one for I'd worry about abuse, the latter might be too weak unless you've got control of attached style declarations or want to (again) resort to the extra !importants or specificity formulation). But yeah, I can see how this might be really helpful, particularly in a group context.
posted by weston at 4:18 PM on January 19, 2011