The point of black triangles
June 9, 2019 5:19 AM Subscribe
“We’ve got ten months to deliver two games to Sony, and they are cheering over a black triangle? THAT took them nearly a month to develop?” If it may not look like much to the casual observer, but you know how important it is - it might be a black triangle. Games developer Jay Barnson on the project breakthroughs that don't give you much to show off.
Via Hot Garbage
Via Hot Garbage
A significant and increasing part of my job as I have progressed from a Mid-level Software Engineer to a Senior and now Principal SWE with lots of Project Management duties has been managing expectations and figuring out how to explain to non-technical (and in my case often government) personnel the significance of an achievement that doesn't have clear visual clues and not to lose themselves over relatively trivial changes that mistakenly appear significant. While I never used "black triangle" as a metaphor, it's in the same vein as the methods I have used.
posted by mystyk at 5:43 AM on June 9, 2019 [5 favorites]
posted by mystyk at 5:43 AM on June 9, 2019 [5 favorites]
It's so often the case, that what looks easy is difficult, and what looks difficult is easy. What's the opposite of a black triangle? I remember one coding job where we were able to add support for same-sex marriage certificates in a day (as opposed to the months that a neighbouring jurisdiction took), because we'd never hard-coded gender on any of the display fields. It was just easier to feed them from the database, and updating *that* took a day (including testing). Made us look pretty good, though.
posted by Mogur at 5:46 AM on June 9, 2019 [12 favorites]
posted by Mogur at 5:46 AM on June 9, 2019 [12 favorites]
I really like this metaphor. So much of great management is this architecture. Both at work and of a household. The invisible work that lets everything flow.
posted by eyeofthetiger at 5:46 AM on June 9, 2019 [5 favorites]
posted by eyeofthetiger at 5:46 AM on June 9, 2019 [5 favorites]
This is so true! There are so many errors you might make when attempting to draw your first triangle —
fail to get the triangle into the draw pipeline at all
omit to program a triangle fill (or generally fail to get the shader to draw something)
forget to swap the buffers so that the screen never updates
fumble the camera transform so that the camera is not looking at the triangle
fumble the camera scale so that the triangle is too tiny to see
mess up the view frustrum so that the triangle is culled even though it ought to be visible
put the near/far planes at the wrong distance so that the triangle is not drawn
get the triangle's orientation backwards so that it gets back-face culled
fill the triangle in the same colour as the background
Until you have drawn your first triangle, it is hard to figure out which of these errors you made. But once you can see one triangle, then errors become much easier to track down because you can see them.
posted by cyanistes at 5:52 AM on June 9, 2019 [12 favorites]
fail to get the triangle into the draw pipeline at all
omit to program a triangle fill (or generally fail to get the shader to draw something)
forget to swap the buffers so that the screen never updates
fumble the camera transform so that the camera is not looking at the triangle
fumble the camera scale so that the triangle is too tiny to see
mess up the view frustrum so that the triangle is culled even though it ought to be visible
put the near/far planes at the wrong distance so that the triangle is not drawn
get the triangle's orientation backwards so that it gets back-face culled
fill the triangle in the same colour as the background
Until you have drawn your first triangle, it is hard to figure out which of these errors you made. But once you can see one triangle, then errors become much easier to track down because you can see them.
posted by cyanistes at 5:52 AM on June 9, 2019 [12 favorites]
I once was subject to a development cycle where we had to give a demo every two weeks. How do you demonstrate half of a networking protocol?
(Of course the answer lies with the old saw "Any sufficiently advanced technology is indistinguishable from a rigged demo". I know for a fact that at least half of the team faked one of theirs)
posted by Tell Me No Lies at 7:21 AM on June 9, 2019 [6 favorites]
(Of course the answer lies with the old saw "Any sufficiently advanced technology is indistinguishable from a rigged demo". I know for a fact that at least half of the team faked one of theirs)
posted by Tell Me No Lies at 7:21 AM on June 9, 2019 [6 favorites]
What's the opposite of a black triangle?
CSS.
posted by RobotVoodooPower at 7:31 AM on June 9, 2019 [16 favorites]
CSS.
posted by RobotVoodooPower at 7:31 AM on June 9, 2019 [16 favorites]
Most of my fondly remembered programming triumphs have been those moments - probably because I was a low-level coder working below the API, and often on prototype hardware that was just coming to life. The fun we had with softies blaming hardies and hardies blaming softies over bugs...
Networking was particularly sweet, though. I remember doing two consecutive DIRs of a hard drive under DOS in pre-multitasking days, and seeing a file appear I'd copied via the network using another computer. A file magically appearing due to the code I'd written that did everything from fishing packets from the LAN hardware to recreating the DOS filing system calls - months and months of learning curve mountaineering to produce something that would look utterly mundane to almost anyone else.
Good times.
posted by Devonian at 7:46 AM on June 9, 2019 [9 favorites]
Networking was particularly sweet, though. I remember doing two consecutive DIRs of a hard drive under DOS in pre-multitasking days, and seeing a file appear I'd copied via the network using another computer. A file magically appearing due to the code I'd written that did everything from fishing packets from the LAN hardware to recreating the DOS filing system calls - months and months of learning curve mountaineering to produce something that would look utterly mundane to almost anyone else.
Good times.
posted by Devonian at 7:46 AM on June 9, 2019 [9 favorites]
I’m pretty sure I was a victim of this. I was doing work that was all black triangle work. I did a huge amount of background work setting up build and deploy pipelines that were previously manual processes. But it is was never showy. It “duplicated” things there were already functional.
Of course, it was faster, automated, and not prone to as much human error, but it wasn’t directly generating revenue. So, reduction in workforce it was.
posted by mfu at 8:28 AM on June 9, 2019 [4 favorites]
Of course, it was faster, automated, and not prone to as much human error, but it wasn’t directly generating revenue. So, reduction in workforce it was.
posted by mfu at 8:28 AM on June 9, 2019 [4 favorites]
Huh, we're doing the backend API so another agency's website and apps can connect to a third company's movie theater backend and to a fourth company's payment system for our client's new movie theater chain.
We basically never have anything to show at meetings with our very untechnical clients. We worked up a script which puts the API through its paces mostly to have something to point to as progress.
posted by signal at 9:06 AM on June 9, 2019 [1 favorite]
We basically never have anything to show at meetings with our very untechnical clients. We worked up a script which puts the API through its paces mostly to have something to point to as progress.
posted by signal at 9:06 AM on June 9, 2019 [1 favorite]
This phenomenon is part of why I've had trouble working on any personal programming project (video games, weird web toys) that take more than a couple days to bootstrap or go on for more than a week or two of development, and why even the ones that get that far basically always, always die at the first prototype. It's hard to explain a black triangle to other people, it's hard to show off, and I have a low threshold for frustration with programming and a strong desire to show creative progress of every day, every step, every New Thing. I buoy my own creative energies on the feedback that comes with showing stuff off.
And so even deciding to make a black triangle is hard; taking the long way around to building out the infrastructure to do a thing right instead of the shortcut to doing the thing now is, every time, me gambling with the possibility that I'll lose patience or lose heart before it gets built in the first place. This is probably worse working independently and as a hobby than it is working with a team and as a job—a lot of the support and incentives for doing a programming task the hard right way aren't there—but I think it's also more of a matter of personal predisposition and if I look back over the years at how I didn't turn into the videogame programmer I imagined I would be at 13, I think that's part of it. It's not a style of creative work that works well for me; I want something I can see at least surface-level results of every step of the way, even if a project isn't a literally straight line from start to finish. A painting might come along in pieces and passes and get early sketches and washes covered up, but all of that was there on the canvas along the way.
I think that's why PICO-8 ended up working well for me; it's so constrained in its toolset, so immediate in its ability to let me build something visible and audible and animated, that there aren't any real big black triangles involved with making a small game. I might spend a day or two gussying up a hacky engine for this or that core idea, but even with that I've already got some sprites on screen, some music looping in the background, some fast and dirty content to show off. A couple weeks is probably the longest by far I spent on one of those games, and it was one that I didn't finish because I got to exactly the point where all my quick and dirty prototype structures in the code weren't going to cut it anymore and needed to be torn out and replaced by a better, from-scratch infrastructure for the game. Which wouldn't even have been a huge task by game standards, but it was big enough. That black triangle showed up and I balked and that was that, maybe for the time being but probably forever.
posted by cortex at 9:33 AM on June 9, 2019 [7 favorites]
And so even deciding to make a black triangle is hard; taking the long way around to building out the infrastructure to do a thing right instead of the shortcut to doing the thing now is, every time, me gambling with the possibility that I'll lose patience or lose heart before it gets built in the first place. This is probably worse working independently and as a hobby than it is working with a team and as a job—a lot of the support and incentives for doing a programming task the hard right way aren't there—but I think it's also more of a matter of personal predisposition and if I look back over the years at how I didn't turn into the videogame programmer I imagined I would be at 13, I think that's part of it. It's not a style of creative work that works well for me; I want something I can see at least surface-level results of every step of the way, even if a project isn't a literally straight line from start to finish. A painting might come along in pieces and passes and get early sketches and washes covered up, but all of that was there on the canvas along the way.
I think that's why PICO-8 ended up working well for me; it's so constrained in its toolset, so immediate in its ability to let me build something visible and audible and animated, that there aren't any real big black triangles involved with making a small game. I might spend a day or two gussying up a hacky engine for this or that core idea, but even with that I've already got some sprites on screen, some music looping in the background, some fast and dirty content to show off. A couple weeks is probably the longest by far I spent on one of those games, and it was one that I didn't finish because I got to exactly the point where all my quick and dirty prototype structures in the code weren't going to cut it anymore and needed to be torn out and replaced by a better, from-scratch infrastructure for the game. Which wouldn't even have been a huge task by game standards, but it was big enough. That black triangle showed up and I balked and that was that, maybe for the time being but probably forever.
posted by cortex at 9:33 AM on June 9, 2019 [7 favorites]
Not to start a debate about "agile programming," but that is one of its features that seems the most cult-like and senseless. It is fine if you're doing web pages or something, where each intermediate step is a thing, but if you're doing lower level stuff, you can easily have tasks that take three months, and suddenly "just work." What good is half a kernel? This was an interesting post and a great metaphor which I am immediately adopting to the exclusion of all others.
I once was subject to a development cycle where we had to give a demo every two weeks. How do you demonstrate half of a networking protocol?
posted by Gilgamesh's Chauffeur at 12:15 PM on June 9, 2019 [5 favorites]
An Agile environment wouldn’t have a step where you stop in the middle of developing a kernel. Which is not to say that some nimrod in management wouldn’t demand to see a demo for things like half-done kernels, but nimrods in management are sadly outside any hope of being addressed by process.
posted by Autumnheart at 1:11 PM on June 9, 2019 [6 favorites]
posted by Autumnheart at 1:11 PM on June 9, 2019 [6 favorites]
I wish there were a politically correct way to show this to management. That’s all I got.
posted by Alterscape at 1:22 PM on June 9, 2019 [1 favorite]
posted by Alterscape at 1:22 PM on June 9, 2019 [1 favorite]
Where I work, the holy Grail of achievement right now isn't even creating things; it is deprecating things, shutting them down so we can free up money and resources for things we want to do. If I can demonstrate that we made some old legacy triangles disappear, people are delighted.
posted by davejay at 1:25 PM on June 9, 2019 [5 favorites]
posted by davejay at 1:25 PM on June 9, 2019 [5 favorites]
With a bit of effort, you can often break things down to smaller demonstrable units, to the benefit of the project. For instance, the black triangle was a subtle but key milestone. Still, at an earlier point one could have "demonstrated" the model code by showing, "678 unit tests passed, 0 failed, with 97% code coverage." Slicing the other way, you could deliver the model code without persistence in one cycle and with persistence in the next.
Either way, you are going to be doing something in those initial two weeks. If you can arrange for that to be a demonstrable, tested bit of code, you have almost certainly improved the modularity and quality of the overall system. If you can produce a requirements document signed off by the module's clients, that too is progress that an Agile process could recognize.
My experience with Agile is that the transition to it is very difficult. At some point you have to stop thinking of it as a set of processes and rituals you merely accommodate and start thinking of it as a set of tools that can help you do your job better and more reliability, while at the same time developing the team's strengths.
Even taking Linux as an example, you can see how it evolved from a terminal emulator to a toy operating system, to a somewhat realer operating system, to a real but very basic operating system as "toy" components are replaced, iteratively, by realer and realer implementations, and finally to a world-class operating system supporting a range of devices that had not even been dreamed of when the project began. You could even think of Agile as being a variety of formal processes intended to bring the successful techniques from the world of Open Source to everyday development.
I'm painfully aware that Agile is difficult to use in long-pole efforts. But careful thinking about how to break the work down can lead to benefits. For instance, if there is a fundamental problem with a design, wouldn't it be better to find out a third of the way in rather than after completing it? That alone could make the additional overhead worthwhile.
Successful projects are not usually ones that test how long developers can swim underwater. They are ones that iterate from "finished" to "finished" in small increments of useful functionality.
(Ten years ago I could not have imagined I would one day write something like this.)
posted by sjswitzer at 1:31 PM on June 9, 2019 [12 favorites]
Either way, you are going to be doing something in those initial two weeks. If you can arrange for that to be a demonstrable, tested bit of code, you have almost certainly improved the modularity and quality of the overall system. If you can produce a requirements document signed off by the module's clients, that too is progress that an Agile process could recognize.
My experience with Agile is that the transition to it is very difficult. At some point you have to stop thinking of it as a set of processes and rituals you merely accommodate and start thinking of it as a set of tools that can help you do your job better and more reliability, while at the same time developing the team's strengths.
Even taking Linux as an example, you can see how it evolved from a terminal emulator to a toy operating system, to a somewhat realer operating system, to a real but very basic operating system as "toy" components are replaced, iteratively, by realer and realer implementations, and finally to a world-class operating system supporting a range of devices that had not even been dreamed of when the project began. You could even think of Agile as being a variety of formal processes intended to bring the successful techniques from the world of Open Source to everyday development.
I'm painfully aware that Agile is difficult to use in long-pole efforts. But careful thinking about how to break the work down can lead to benefits. For instance, if there is a fundamental problem with a design, wouldn't it be better to find out a third of the way in rather than after completing it? That alone could make the additional overhead worthwhile.
Successful projects are not usually ones that test how long developers can swim underwater. They are ones that iterate from "finished" to "finished" in small increments of useful functionality.
(Ten years ago I could not have imagined I would one day write something like this.)
posted by sjswitzer at 1:31 PM on June 9, 2019 [12 favorites]
I love this analogy.
A counterexample (not a disagreement) is that, in a lot of design-related practices, black triangles can be a risky and unhelpful way to think. Which is: in some practices, you can't abstract an engine/pipeline if the engine itself might change and be contingent on the content.
For example: graphic, print, or web design. A design pitfall is to design something with example Lorem Ipsum content... and then, when you actually put the actual content in, you realize that the wrong things are being highlighted, or text takes up more/less space than expected, or just that the everything feels a little bit off.
The same goes for architectural and spatial design. You can't design a space without looking at some specifics; you can't design a building without thinking of the dimension of units, and you can't design a house without actually being careful about the dimensions of rooms, and thus the furniture within them. Drawing things at scale, with the specific objects and furniture necessary, is thus actually a generative task, not a constraining task.
In other words, abstractions can become constraining. Or: for some practices, the way that abstractions flow 'upwards' and 'downwards' in terms of scale are different.
posted by suedehead at 1:54 PM on June 9, 2019 [4 favorites]
A counterexample (not a disagreement) is that, in a lot of design-related practices, black triangles can be a risky and unhelpful way to think. Which is: in some practices, you can't abstract an engine/pipeline if the engine itself might change and be contingent on the content.
For example: graphic, print, or web design. A design pitfall is to design something with example Lorem Ipsum content... and then, when you actually put the actual content in, you realize that the wrong things are being highlighted, or text takes up more/less space than expected, or just that the everything feels a little bit off.
The same goes for architectural and spatial design. You can't design a space without looking at some specifics; you can't design a building without thinking of the dimension of units, and you can't design a house without actually being careful about the dimensions of rooms, and thus the furniture within them. Drawing things at scale, with the specific objects and furniture necessary, is thus actually a generative task, not a constraining task.
In other words, abstractions can become constraining. Or: for some practices, the way that abstractions flow 'upwards' and 'downwards' in terms of scale are different.
posted by suedehead at 1:54 PM on June 9, 2019 [4 favorites]
An Agile environment wouldn’t have a step where you stop in the middle of developing a kernel.
...other than every two weeks when you blow at least a day of development to produce a demonstration.
Truthfully I've never seen two implementations of "Agile" practices that were the same. This is supposedly a major feature, but it has taught me never to say what "Agile" is or isn't. Well, except that in my own personal experience most of "Agile" is "Unnecessary Overhead".
posted by Tell Me No Lies at 2:30 PM on June 9, 2019 [1 favorite]
...other than every two weeks when you blow at least a day of development to produce a demonstration.
Truthfully I've never seen two implementations of "Agile" practices that were the same. This is supposedly a major feature, but it has taught me never to say what "Agile" is or isn't. Well, except that in my own personal experience most of "Agile" is "Unnecessary Overhead".
posted by Tell Me No Lies at 2:30 PM on June 9, 2019 [1 favorite]
...other than every two weeks when you blow at least a day of development to produce a demonstration.Demonstration is not waste. Demonstration is not waste.
Demos are how you show yourself and others that you’re moving in a useful direction. They build trust with project participants who aren’t off noodling around in the black triangle pits, but are doing other important things like lining up customers and money or trying to plan when the previous version of a thing can be turned off. They also prove to you, the engineer, that you haven’t lost the plot entirely because you can communicate some form of value, maybe to an end user who’s testing your thing.
I have participated in numerous projects with people who convinced themselves they were “doing agile” even though they continued to produce nothing but black triangles all the way through. They could prove that their system produced a formally-specified result of some kind, but not that their work was useful or valuable to anyone in particular. The black triangle can easily become a form of self-justification, a way of lying to yourself about progress that’s meaningful only in a formal sense rather than a useful one.
posted by migurski at 3:08 PM on June 9, 2019 [11 favorites]
Not to start a debate about "agile programming," but that is one of its features that seems the most cult-like and senseless. It is fine if you're doing web pages or something, where each intermediate step is a thing, but if you're doing lower level stuff, you can easily have tasks that take three months, and suddenly "just work." What good is half a kernel?
It probably behooves everyone to recall the first version of Linux. Lars Wirzenius -- college friends with Linus and early kernel hacker-- has stories about the first, unpublished versions of Linux. They were just two threads printing AAA and BBB to the screen. And probably, before that, just AAA. It's very much an agile story of doing and publishing little bits of work, and even starts with a demo after two weeks.
And sure, I don't know what value Linux had on these first iterations, but it's a polar opposite of the OPs 'we have ten months to deliver a working product to Sony for the Playstation launch' Instead of 20 dudes hacking away for a month, it's one, working in solitude for a few weeks.
posted by pwnguin at 4:15 PM on June 9, 2019 [6 favorites]
It probably behooves everyone to recall the first version of Linux. Lars Wirzenius -- college friends with Linus and early kernel hacker-- has stories about the first, unpublished versions of Linux. They were just two threads printing AAA and BBB to the screen. And probably, before that, just AAA. It's very much an agile story of doing and publishing little bits of work, and even starts with a demo after two weeks.
And sure, I don't know what value Linux had on these first iterations, but it's a polar opposite of the OPs 'we have ten months to deliver a working product to Sony for the Playstation launch' Instead of 20 dudes hacking away for a month, it's one, working in solitude for a few weeks.
posted by pwnguin at 4:15 PM on June 9, 2019 [6 favorites]
Demonstration is not waste. Demonstration is not waste.
Demonstrating that we match the checksum algorithm for the input feed cache on the Santo HS-100 chip that is four layers below anything any other engineer, much less customer will see, when I’m the only person in the company who has any knowledge of the Santo chipset much less the Santo HS-100 is a waste of everyone’s time. Mine, and everyone else’s.
posted by Tell Me No Lies at 4:30 PM on June 9, 2019 [1 favorite]
Demonstrating that we match the checksum algorithm for the input feed cache on the Santo HS-100 chip that is four layers below anything any other engineer, much less customer will see, when I’m the only person in the company who has any knowledge of the Santo chipset much less the Santo HS-100 is a waste of everyone’s time. Mine, and everyone else’s.
posted by Tell Me No Lies at 4:30 PM on June 9, 2019 [1 favorite]
There’s totally still value in a demonstration.
In my current company, a senior, long-tenured engineer on a neighboring team is working themselves into a frenzy on a very deep technical problem that’s blowing a hole in the side of a larger project and pushing deadlines out by wild amounts (“6-9 months”). They are convinced it’s critical that this problem be solved and that the overall user-facing effort is dependent on its success. Meanwhile, a bunch of less-senior people elsewhere who have more experience in this problem area are pressing the emergency brake button with increasing levels of urgency trying to communicate that in fact this deep problem is no problem at all, doesn’t need solving, and has taken similar teams in other companies little time to circumvent once they’ve built tight feedback and demo loops into their process.
I trust you know what you’re doing but imagine demonstrating a half-working Santo HS-100 checksum algorithm and someone in the back of the room saying “wait – another team just learned that we need to go in a different direction on the product and don’t need this work done at all.” That’s potentially a lot of time and effort saved through early communication and demonstration.
posted by migurski at 5:23 PM on June 9, 2019 [6 favorites]
In my current company, a senior, long-tenured engineer on a neighboring team is working themselves into a frenzy on a very deep technical problem that’s blowing a hole in the side of a larger project and pushing deadlines out by wild amounts (“6-9 months”). They are convinced it’s critical that this problem be solved and that the overall user-facing effort is dependent on its success. Meanwhile, a bunch of less-senior people elsewhere who have more experience in this problem area are pressing the emergency brake button with increasing levels of urgency trying to communicate that in fact this deep problem is no problem at all, doesn’t need solving, and has taken similar teams in other companies little time to circumvent once they’ve built tight feedback and demo loops into their process.
I trust you know what you’re doing but imagine demonstrating a half-working Santo HS-100 checksum algorithm and someone in the back of the room saying “wait – another team just learned that we need to go in a different direction on the product and don’t need this work done at all.” That’s potentially a lot of time and effort saved through early communication and demonstration.
posted by migurski at 5:23 PM on June 9, 2019 [6 favorites]
Testing, and demonstration via testing, is fraught with difficulties. A big one is failing to normalize for equivalence classes in tests. That seems to me the brunt of the Santo HS-100 issue described just above.
In one system I worked on the web-invocation layer was "mocked" at an API layer that cared which sequence of API calls you made to construct a URL rather than the net effect. But these are meant to form an equivalence class. Should it matter whether you add a fragment first and then a query parameter, or vice versa? No, but the test cases nevertheless insisted and this made refactoring or simply adding new features extremely costly.
In another, the product's markup language had all the usual equivalence classes (e.g., didn't matter where line breaks occurred, as long as on whitespace). New changes that were required meant adding new attributes to the markup and thus changing line breaks in the markup language. The tests, however, were brittle in this regard and development spent half their time fixing brittle tests rather than developing new features. Whether any of it added to the quality of the product is at best debatable. (In the next release, that test suite was quietly abandoned, so.)
In the first case we found a better mocking framework, reduced the size of the test code significantly and, as a result, had tests that were far more robust against changes that were functionally equivalent (e.g., refactoring) and adding new features was straightforward. In the second case the situation was either hopeless or everyone was just too exasperated with the situation.
So, IMO, the complaint that a testing-based demo was not helpful is not determinative. There are better and worse ways to do it.
I will say, however, that in the case that failed there was a lot of emphasis on following a certain handed-down orthodoxy. Questioning it was not a very good use of your time or social capital. It was nevertheless amusing to see the utterly implicit "let us never speak of this again" attitude that accompanied the abandonment of that (very well-intended!) systematic testing effort. (Automated black-box testing, especially of user interface code, is particularly vulnerable to this problem. I regard it as an open question ripe for innovation. This is your billion-dollar opportunity. Drop everything and solve it!)
An explicit component of most Agile processes is a "retrospective" that looks into what worked well and what didn't in order to improve the process over time. Sadly, in the second case the process wasn't Agile and didn't have that phase. I can't even speculate whether it would have helped since the "process-first" mentality so overwhelmed honest self-examination. (In contrast, you might not be completely wrong to see the Agile retrospective as being a sort "Struggle Session" to identify and blame those who have failed the cause. Avoiding that failure mode is key to implementing Agile effectively.)
Testing has to be designed to be robust against refactoring-level changes and also easily adaptable to the sorts of incremental change that are, of course, expected. This is not easy to get right! But it is worth the effort: developing software without thorough test suites is terrifying and leads developers to make bad choices at every step.
posted by sjswitzer at 5:27 PM on June 9, 2019 [4 favorites]
In one system I worked on the web-invocation layer was "mocked" at an API layer that cared which sequence of API calls you made to construct a URL rather than the net effect. But these are meant to form an equivalence class. Should it matter whether you add a fragment first and then a query parameter, or vice versa? No, but the test cases nevertheless insisted and this made refactoring or simply adding new features extremely costly.
In another, the product's markup language had all the usual equivalence classes (e.g., didn't matter where line breaks occurred, as long as on whitespace). New changes that were required meant adding new attributes to the markup and thus changing line breaks in the markup language. The tests, however, were brittle in this regard and development spent half their time fixing brittle tests rather than developing new features. Whether any of it added to the quality of the product is at best debatable. (In the next release, that test suite was quietly abandoned, so.)
In the first case we found a better mocking framework, reduced the size of the test code significantly and, as a result, had tests that were far more robust against changes that were functionally equivalent (e.g., refactoring) and adding new features was straightforward. In the second case the situation was either hopeless or everyone was just too exasperated with the situation.
So, IMO, the complaint that a testing-based demo was not helpful is not determinative. There are better and worse ways to do it.
I will say, however, that in the case that failed there was a lot of emphasis on following a certain handed-down orthodoxy. Questioning it was not a very good use of your time or social capital. It was nevertheless amusing to see the utterly implicit "let us never speak of this again" attitude that accompanied the abandonment of that (very well-intended!) systematic testing effort. (Automated black-box testing, especially of user interface code, is particularly vulnerable to this problem. I regard it as an open question ripe for innovation. This is your billion-dollar opportunity. Drop everything and solve it!)
An explicit component of most Agile processes is a "retrospective" that looks into what worked well and what didn't in order to improve the process over time. Sadly, in the second case the process wasn't Agile and didn't have that phase. I can't even speculate whether it would have helped since the "process-first" mentality so overwhelmed honest self-examination. (In contrast, you might not be completely wrong to see the Agile retrospective as being a sort "Struggle Session" to identify and blame those who have failed the cause. Avoiding that failure mode is key to implementing Agile effectively.)
Testing has to be designed to be robust against refactoring-level changes and also easily adaptable to the sorts of incremental change that are, of course, expected. This is not easy to get right! But it is worth the effort: developing software without thorough test suites is terrifying and leads developers to make bad choices at every step.
posted by sjswitzer at 5:27 PM on June 9, 2019 [4 favorites]
I now pause to reflect, again, whether Agile processes are, at root, attempts to implement in the command economy of commercial organizations what works* organically in open source communities.
* Note that I am not holding up open source organizations as ideal social structures. Their failings are famous but there have been many encouraging recent developments.
posted by sjswitzer at 6:07 PM on June 9, 2019 [3 favorites]
* Note that I am not holding up open source organizations as ideal social structures. Their failings are famous but there have been many encouraging recent developments.
posted by sjswitzer at 6:07 PM on June 9, 2019 [3 favorites]
Hmmm...I guess don't neglect to loosely couple the test types to the types-under-test. Then yer always one facade away from an easy refactor. Automated UI tests? Yeah, that's a puzzler.
posted by j_curiouser at 9:07 PM on June 9, 2019
posted by j_curiouser at 9:07 PM on June 9, 2019
And yeah, +1 for the black triangle metaphor.
posted by j_curiouser at 9:13 PM on June 9, 2019
posted by j_curiouser at 9:13 PM on June 9, 2019
What's the opposite of a black triangle?
Well, from the article:
By the end of the day, we had complete models on the screen, manipulating them with the controllers. Within a week, we had an environment to move the model through.
I don't know what the snappy name is for that but it does demonstrate the other side. Going from a single black triangle to complete models on the screen in a day and maps in a week seems like incredible progress if you don't understand why the original black triangle is so important.
In a previous job I was involved in turning client data into useful insights. We would spend weeks working in parallel to brainstorm hypotheses and testing them on data subsets while another team was cleaning and mapping out dodgy client datasets. It looked like we'd done nothing until the analysis pipeline was up at which point we could produce interactive views of important insights from the data at a firehose-like rate. 90% of my job was figuring out which early insights we could pull out so that we could stage-manage the client experience and leave them feeling that we were making constant progress.
posted by atrazine at 6:14 AM on June 10, 2019 [4 favorites]
Well, from the article:
By the end of the day, we had complete models on the screen, manipulating them with the controllers. Within a week, we had an environment to move the model through.
I don't know what the snappy name is for that but it does demonstrate the other side. Going from a single black triangle to complete models on the screen in a day and maps in a week seems like incredible progress if you don't understand why the original black triangle is so important.
In a previous job I was involved in turning client data into useful insights. We would spend weeks working in parallel to brainstorm hypotheses and testing them on data subsets while another team was cleaning and mapping out dodgy client datasets. It looked like we'd done nothing until the analysis pipeline was up at which point we could produce interactive views of important insights from the data at a firehose-like rate. 90% of my job was figuring out which early insights we could pull out so that we could stage-manage the client experience and leave them feeling that we were making constant progress.
posted by atrazine at 6:14 AM on June 10, 2019 [4 favorites]
As an FYI, there are ways to show progress on design for things that can make sense when you don't have live code to show.
For example, I looked at a code stream that we had that made sense to be replaced with a pipeline of message handlers, each handler was designed to do one small thing and do it well. A handler could ignore a message, respond to a message, change/add to a message, short circuit other processing, etc. When the design was done, I set up the management as a human version of the pipeline, each with their job description in front of them. I then handed notecards with messages and watched everything magically work.
That was for the design. Once the design was done, it was then a matter of time for the implementation to happen.
posted by plinth at 10:01 AM on June 10, 2019 [6 favorites]
For example, I looked at a code stream that we had that made sense to be replaced with a pipeline of message handlers, each handler was designed to do one small thing and do it well. A handler could ignore a message, respond to a message, change/add to a message, short circuit other processing, etc. When the design was done, I set up the management as a human version of the pipeline, each with their job description in front of them. I then handed notecards with messages and watched everything magically work.
That was for the design. Once the design was done, it was then a matter of time for the implementation to happen.
posted by plinth at 10:01 AM on June 10, 2019 [6 favorites]
We've actually done wiki page "demos" in the past. And also has sprints where we decided "no, I don't think we can make a compelling demo of that, let's just skip it".
Basically, stories get demoed every two weeks... if there's something to demo. Sometimes there just isn't, and that's fine. And if it's NOT fine with your management, that's a problem with your management.
posted by tobascodagama at 10:36 AM on June 10, 2019 [1 favorite]
Basically, stories get demoed every two weeks... if there's something to demo. Sometimes there just isn't, and that's fine. And if it's NOT fine with your management, that's a problem with your management.
posted by tobascodagama at 10:36 AM on June 10, 2019 [1 favorite]
Meanwhile, a bunch of less-senior people elsewhere who have more experience in this problem area
We are talking about extremely different environments. The place with the two-week demo requirement was a startup with 10 engineers, only three of whom were doing platform work. Basically the only three people with the expertise to even comment on the problem sat within ten feet of one another and had lunch together most days.
posted by Tell Me No Lies at 2:34 AM on June 11, 2019
We are talking about extremely different environments. The place with the two-week demo requirement was a startup with 10 engineers, only three of whom were doing platform work. Basically the only three people with the expertise to even comment on the problem sat within ten feet of one another and had lunch together most days.
posted by Tell Me No Lies at 2:34 AM on June 11, 2019
"In other words, abstractions can become constraining. Or: for some practices, the way that abstractions flow 'upwards' and 'downwards' in terms of scale are different."
One analogy I've used for the different levels of abstraction not being symmetrical is thinking about percentages. Increase 100 by 20%, you've got 120; but decrease 120 by 20%, you've got 96.
posted by klangklangston at 6:04 PM on June 11, 2019 [1 favorite]
One analogy I've used for the different levels of abstraction not being symmetrical is thinking about percentages. Increase 100 by 20%, you've got 120; but decrease 120 by 20%, you've got 96.
posted by klangklangston at 6:04 PM on June 11, 2019 [1 favorite]
« Older Technology, Law and Political Economy for Humans | The ultra-rich have no redeeming social value. Newer »
This thread has been archived and is closed to new comments
posted by Autumnheart at 5:35 AM on June 9, 2019 [7 favorites]