The Beginning of the End of Big Government IT
December 8, 2015 10:42 AM   Subscribe

The state of California just announced that the new technology underpinning its Child Welfare System [pdf] won't be the usual "IT Solution" bought up in one big lump to follow a 4000-page specification. Instead, it's going to be built as a series of smaller modular projects driven by user needs, drawing on agile methodologies, a wider range of vendors, and, wherever possible, open standards and open source software. The decision, made in collaboration with Code for America and the federal government, sets an important precedent for how governments on all levels can get past the pitfalls of the standard procurement model.

Large institutional computer systems, especially in government, have long been their own punchline: expensive, bloated, malfunctioning, often designed around the needs of the small group of contractors qualified to bid for them, not their intended users. If visiting the DMV is bad for your mental health, working there and having to use its broken computer systems every day is worse. The list of recent Big IT failures covers not just Healthcare.gov's first iteration, or the broken health exchanges, but state welfare systems, city financial systems and regional emergency phone systems. The federal government has just frozen its chunk of payments towards Texas's new child support IT system until the contractors explain how it's going to get finished.

It's not easy to break the cycle of bad systems that emerge from bad procurement models. Choices are often constrained by legislation, influenced by lobbyists, or simply hemmed in by convention: no one ever got fired for buying IBM. It requires alternative approaches that work. Over the past five years, the UK's GDS has helped set an example of how to do things differently, and influenced the formation of similar in-house digital government teams around the world, including its American equivalent. Every time government chooses to do IT differently and succeeds, it redefines too-big-to-fail monolithic projects as risky choices, not comfortable or conventional ones. Or as CfA's (and MeFi's own) Dan Hon puts it: "It isn't hard to do this kind of thing. It isn't easy, either. It's just simply doable. The fact that you're not doing it is now a valid signal that you're not doing it for a reason."
posted by holgate (63 comments total) 87 users marked this as a favorite

 
What a fantastic post! I've been following this shift in US government IT practice in the past few years, I have a few friends involved in the work. It's quite inspirational. I'm hoping this new California project is the first in a line of successful projects in my state.

Government should provide useful services to citizens. And it should provide those services competently and cost effectivey. I have a lot of admiration for the software engineers and product designers and others rolling up their sleeves and making it happen.
posted by Nelson at 10:55 AM on December 8, 2015 [1 favorite]


Finally! I'm really glad to see tax dollars being used in something at least approaching sanity. In a way healthcare.gov was a blessing in disguise. Government IT projects have failed in similar ways for DECADES, but that was the first one that really got people to notice.
posted by sp160n at 10:57 AM on December 8, 2015 [3 favorites]


The vendor forum meeting in Sacramento on Friday was a procurement thrill ride. The state saw significantly more interest in this project than past ones, and I was particularly happy to see state experts in the actual business of child welfare like Kevin Gaines present to the group. All the right words were said: user research, user needs, agile iterative, etc. etc.
posted by migurski at 11:00 AM on December 8, 2015 [5 favorites]


Yes, this sounds promising. Standards are key, along with standards compliance test suites. Otherwise, the same old integration and vendor finger-pointing irritations and catastrophes will continue. Any significant vendor reaction yet?
posted by gregoreo at 11:08 AM on December 8, 2015


I'm still confused about the nature of the problem. Are large government IT projects no worse than large private sector IT projects, and we only hear about the government failures due to freedom of information laws? Or is there something unique about government that makes IT procurement impossible?
posted by miyabo at 11:11 AM on December 8, 2015 [4 favorites]


The required visibility of large government IT failures via FOIA is definitely an issue, yes.

There has also been a move away from knowing how to do digital things in government, replaced by handing large initiatives over to vendors and attempting to control those large initiatives with contractual levers. Imagine planning a technology project by spending two years writing down all the bad things that can happen! The federal version of this “big lobotomy” was well-described in a Washington Monthly article last year:
The third target of Gingrich’s attacks was the legislative support agencies. The Government Accountability Office, Congressional Budget Office, Congressional Research Service, and now-defunct Office of Technology Assessment operated on a bipartisan basis, offering measured reports on topics suggested by members themselves. Sometimes, these agencies act as helpful librarians, and sometimes they’re more like referees, carefully adjudicating among the competing quantitative claims of various members and outside groups.

Gingrich, perhaps not surprisingly, viewed them all as potential rivals to his singular narrative. He particularly despised the OTA, as did many other conservatives, despite its evident usefulness. For instance, Congress saved hundreds of millions of dollars by incorporating OTA recommendations when the Social Security Administration moved to replace its old-school mainframe computers with a new computer network in the mid-’90s. (One wonders how things might have been different if the OTA had been around when the health care insurance exchanges were being built.) But over the years, the OTA had also cast doubt on some conservative ideas—a mortal sin as far as Gingrich and his followers were concerned. For example, an OTA report raising serious questions about the feasibility of Reagan’s Star Wars project was later used by Democrats to help defund the program.
posted by migurski at 11:20 AM on December 8, 2015 [16 favorites]


Seems to me they're exchanging one set of headaches for a different set. Maybe the second set isn't as bad; that would be nice. But what I'd be most worried about is the big "I": interoperability.
posted by Chocolate Pickle at 11:27 AM on December 8, 2015 [3 favorites]


Man, I've done a lot of stuff for CfA, and generally really like them, but this feels like it's ridiculously far outside of their core mission.

I don't want this to be the road that they go down.

Governments need more (and better) in-house expertise, better use of shared services (through people like the GSA and things like 18F), and to continue to fund/support research organizations that can establish best-practices, and act in an unbiased advisory role (ie. MITRE).

And, oh gosh, they need to fix their hiring process. But that's not what I'm going to rant about here.

It's good that governments are getting out of the mindset of building monolithic enterprise IT services, but I'm also a bit worried that this one of those technological cargo cult situations, where some Silicon Valley techbros are encouraging the government to jump on the microservices bandwagon.

Others have said as much (and better!), but I've previously ranted on the blue that government IT has a fundamentally different set of requirements than anything that's come out of Silicon Valley recently. Go read it -- all of it still applies.

Microservices aren't perfect for everybody -- the hosting/maintenance costs can be quite high, and there are certain problems that are inherently huge, complex, and heavily relational. I work in education, and this is definitely a problem for us, because most of our data is very tightly-coupled by its very nature, and our customers (quite understandably) don't have the resources to operate and integrate dozens of separate IT systems, as tempting as the lure of modularity might be... It sounds really good in theory, but it's really tough to do in practice, especially if you have a limited capacity to make compromises in your system's functionality.

Similarly, Agile just doesn't make any sense for contracting, and especially doesn't make any sense for the sort of contracting that the government does. While those onerous contracting requirements are indeed a terrible match for software development, they do still serve their original purpose of preventing or mitigating fraud.

Don't hire startup guys to fix the government's IT problems. Hire a corporation that's successfully consolidated or integrated tons of disparate systems. It's less sexy, but it's a lot harder, and is a much more realistic analogy to the problems that the government solves -- dealing with a complicated legacy; a nebulous set of stakeholders; complicated and conflicting requirements that impede a "clean" implementation; etc...
posted by schmod at 11:32 AM on December 8, 2015 [30 favorites]


Welcome to modern Big Company methodologies! You may find them less inherently better than you expect.
posted by Artw at 11:36 AM on December 8, 2015 [6 favorites]


No one’s suggesting microservices, schmod.
posted by migurski at 11:38 AM on December 8, 2015 [7 favorites]


Oh, wow. I completely lost the original thread of my comment.

tl;dr; The government has problems it needs to solve. I'm deeply uncomfortable that CfA is being viewed as the solution. Even if the government solves its problems, it'll probably look a lot different than the stuff that the private sector does -- the requirements are different, and the governance/incentives/motives are very different*.

*I'm not talking about the "Government is inefficient because there's no profit motive" fallacy -- there are plenty of well-run nonprofits. However, the outcomes and metrics of success tend to be quite different in government organizations, and there's a much higher level of accountability, oversight, and redundancy.

"Government is inherently inefficient" isn't technically wrong, but it is an overly reductive way of saying "The checks and balances that we've (rightly) put in place to ensure an accountable and democratic government have some costs associated with them."

posted by schmod at 11:40 AM on December 8, 2015 [4 favorites]


It's hard to do an apples for apples comparison, but what if we really duplicated what works with private sector tech companies in the public sector? This would mean 100 DMV systems built on spec, 5 end up with a usable system, 99 go bankrupt, and one gets rich (on the public budget rather than using stock exchange money this time).
posted by idiopath at 11:43 AM on December 8, 2015 [4 favorites]


CfA is definitely not being proposed as the solution, either! We acted as a consultant to help introduce a methodology that provided the state with better options and lower risk, but the actual work is going to get done by vendors who go through a formal bidding process. The key difference is that instead of one big-ass project to be delivered in 2021 (really), there will be a series of smaller projects to be delivered starting next year. This allows the state and vendors to course-correct, respond to new needs, and learn from the process.
posted by migurski at 11:44 AM on December 8, 2015 [11 favorites]


Speaking as an employee of a state agency which is facing the rollout of a big-name ERP solution -- which is off to a decidedly rocky start (ahem) -- I applaud this. There is just nofuckingway it could cost taxpayers MORE to hire engineers to build sustems that actually serve organizational needs... jesus. Just, yeah.
posted by trunk muffins at 11:44 AM on December 8, 2015 [2 favorites]


Don't hire startup guys to fix the government's IT problems.

The success of a bunch of startup guys rescuing healthcare.gov is a direct counterpoint.

Of course a bunch of "techbros" selling "microservices" isn't some magic solution to all government IT problems. But government IT projects are way, way too far into multiyear, $XXm waterfall debacles. Bringing a little modern product design and engineering skills to the table is a good thing.
posted by Nelson at 11:50 AM on December 8, 2015 [5 favorites]


(I don't want to derail, but I've had nothing but competent, prompt service from my state's DMV. It never makes sense when I seem them held up as the paragon of shittiness.)
posted by paper chromatographologist at 11:53 AM on December 8, 2015 [10 favorites]


Are large government IT projects no worse than large private sector IT projects, and we only hear about the government failures due to freedom of information laws? Or is there something unique about government that makes IT procurement impossible?

My take is that many in the federal government who are in change of buying and managing IT services still operate with the waterfall methodology as their paradigm, even though it has been largely abandoned in the private sector (in Silicon Valley, anyway) as unwieldy and expensive. Part of the pitch that Code for America, 18F, et al have been giving is that the private sector has developed proven practices for building websites and other services quickly and cheaply, and that among these practices is using agile methodologies for software development. I can say from experience that there are many projects where a lot of grief could have been avoided had agile been used instead of waterfall, which offers no real way to correct course without spending a lot of extra time and money.

That said, I think ~schmod is right that you can't simply import the Silicon Valley ethos into the public sector, because they are focused on very different problems. "Move fast and break things" might be a good rule for selling shoes, but it's a terrible rule for distributing benefits fairly or providing information for a diverse population. Nevertheless, I'm convinced that the CfA approach will improve things on the margin, and may lead to real transformation in how government agencies do IT down the road.
posted by Cash4Lead at 11:55 AM on December 8, 2015 [6 favorites]


Finally! I'm really glad to see tax dollars being used in something at least approaching sanity. In a way healthcare.gov was a blessing in disguise. Government IT projects have failed in similar ways for DECADES, but that was the first one that really got people to notice.

healthcare.gov had a well-publicized bumpy launch, but it cannot be fairly characterized as to have failed. From here:

-8 million enrolled in the Marketplaces during 2014 open enrollment (Oct 2013 to April 2014).

-11.7 million are estimated to have enrolled in the Marketplaces during 2015 open enrollment (Nov 2014 to Feb 2015). This includes 4.5 million who re-enrolled from 2014. As of June 30 2015 9.9 million were still enrolled and had paid for a Marketplace plan. This is on par with HHS’s original projection of 9.1 million.

-As of March 2015 HHS reported a total of 16.4 covered due to the ACA between the Marketplace, Medicaid expansion, young adults staying on their parents plan, and other coverage provisions. This number was reported by HHS to have rose to 17.6 million according to September 2015 data.

-According to Gallup the uninsured rate was 11.9% for the 18 – 65 demographic in the 1st quarter of 2015, down from a high of 18% in 2013. By the second quarter of 2015 the uninsured rate fell to 11.4%.


Even software companies will buy off the shelf when the requirement is not in their strike zone. Building and certainly integrating software is certainly not in government agencies' strike zone. The issue is what happens when the buyer can't build it, and it also doesn't exist on the shelf. That means custom software, and custom software is expensive - regardless if you're building from scratch (which is frighteningly naive given how much legacy stuff still needs to run/be talked to ) or tailoring/modifying existing options.

Pro-tip: if the vendors/system integrators involved don't have a long list of success and detailed explanations addressing project failures (assuming any project failures exists), you either need to find a new vendor, or realize the problem is much harder than you thought, previously.

The success of a bunch of startup guys rescuing healthcare.gov is a direct counterpoint.

I've said repeatedly on here that this narrative is bullshit. I say this as someone that used to have a healthcare.gov email address - to my knowledge there was no one involved in the so-called "tech surge" from any company that had any fewer than 300 employees.
posted by NoRelationToLea at 11:57 AM on December 8, 2015 [4 favorites]


migurski: "No one’s suggesting microservices, schmod."

Right in the FPP:
Instead, it's going to be built as a series of smaller modular projects driven by user needs
This implies microservices to me, or something similar.

Like I said, the benefits of microservices are well-understood, but certain problem-spaces (like education, healthcare, and government) have had a ton of difficulty breaking out of the monolith, and I think that a lot of the criticisms of these industries are frankly insulting to the people who have been fighting from the inside to improve things.

Monoliths are bad, because they tend to lead to lock-in, are difficult to evolve, and can be difficult to integrate with.

Modular systems are generally easier to integrate with, because it inherently becomes a very important design criteria of those systems. [Anecdotally, I've worked with modular systems that exposed their modularity in a profoundly unhelpful way. Go find some Java code from 10 years ago if you're wondering what I'm talking about.]

Integrations are hard, even when they are done well. Standards help, but the standards also need to be well-written, accommodate all known use-cases, and be implemented consistently. Developing good standards is a really hard problem.

When I read things like this, I hear "If only they did things like us, they'd be so much better!" coming from people who don't fully understand the problem at hand. Outsider perspectives are often helpful, but I'm getting skeptical of the cheers to "Disrupt! Disrupt! Disrupt!"

Nelson: "The success of a bunch of startup guys rescuing healthcare.gov is a direct counterpoint."

I know quite a few people who worked on healthcare.gov. The narrative that got told in the media was only a very tiny fraction of the story. Depending on who you ask, those guys only got in the way.

I know that I'm probably coming off as defending the existing contracting system, and monolithic projects with 6-year timelines (Jesus Christ!). Maybe CfA did good work here (IMO, advocating for standards/open data/OSS is definitely within their mission), but I'm so, so, so jaded when I read things like this.
posted by schmod at 12:01 PM on December 8, 2015 [5 favorites]


holgate: ""It isn't hard to do this kind of thing. It isn't easy, either. It's just simply doable. The fact that you're not doing it is now a valid signal that you're not doing it for a reason.""

This is a fantastic quotation.
posted by boo_radley at 12:08 PM on December 8, 2015 [4 favorites]


Hi schmod, I am the mefi's own, former Editorial Director at Code for America, who has been working on this with the State of California, the Department of Social Services, their technology sister agency the Office of Systems Integration and Government Operations, where the state Department of Technology lives.

I'll take your comments and provide as much information as I'm able to:
Governments need more (and better) in-house expertise, better use of shared services (through people like the GSA and things like 18F), and to continue to fund/support research organizations that can establish best-practices, and act in an unbiased advisory role (ie. MITRE).
Absolutely agreed. Many of these shared services do not exist, and there's a lot of Not Invented Here. Part of the problem is that many agencies and departments are completely lacking in strong technical ability or service planning, having outsourced the bulk of that kind of work to third parties. The other part is that many agencies and departments also a) don't know what they need, and rely on a vendor to figure that out for them, or b) think they know what they need, and then frequently get exactly, but only, that.
And, oh gosh, they need to fix their hiring process. But that's not what I'm going to rant about here.
Absolutely agreed, too. Long-term, the state needs to have good technical and product capability in-house, and that's part of a strand of separate-but-related-and-required work that's going on at Government Operations.
It's good that governments are getting out of the mindset of building monolithic enterprise IT services, but I'm also a bit worried that this one of those technological cargo cult situations, where some Silicon Valley techbros are encouraging the government to jump on the microservices bandwagon.
It isn't, but I'm not sure there's much I can do to reassure you of no intention to cargo-cult at this point; the only thing I can really say is to wait and see what comes out in the RFP and to take a look at the services that are delivered. This is, as many people are fond of saying, hardly anything to do with technology, and a lot instead to do with people and process and how organizations work. And for the record, no-one is suggesting microservices, and if anyone had done in any of the rooms I'd been in over the last few weeks, I'd have politely shut them down. Please understand that the focus is on delivering *good technology that meets users' needs* and that is continuously improved.
Others have said as much (and better!), but I've previously ranted on the blue that government IT has a fundamentally different set of requirements than anything that's come out of Silicon Valley recently. Go read it -- all of it still applies.
But it doesn't, not entirely. The *technical* requirements aren't particularly out of the ordinary anymore, not compared to the stupendously large consumer services we have these days. The other requirements - the regulatory and bureaucratic requirements - are those that I think you call out in your previous comment. And those are the ones that I've spent as much as my time working on, talking with, persuading and meeting with both executive leadership, middle management, line managers and front-office workers that, amongst other things *they need the right tools to do their jobs* - which addresses your IE8 issue and overly draconian IT and security policies - and it means getting the policy people *and* the design/development/product people in the same room to ask: why is it that the current system is this way? Is there a better way? Because you're absolutely right that nothing significantly *better* will be delivered *without* those changes. And I had hoped that *those* types of changes had been highlighted in my two blog posts.
Microservices aren't perfect for everybody -- the hosting/maintenance costs can be quite high, and there are certain problems that are inherently huge, complex, and heavily relational. I work in education, and this is definitely a problem for us, because most of our data is very tightly-coupled by its very nature, and our customers (quite understandably) don't have the resources to operate and integrate dozens of separate IT systems, as tempting as the lure of modularity might be... It sounds really good in theory, but it's really tough to do in practice, especially if you have a limited capacity to make compromises in your system's functionality.
Again, no-one has proposed microservices and the blog posts (at least the ones that I wrote about the California Child Welfare System) do not anticipate microservices in any way. I think they're kind of dumb most of the time, and especially in the transition from a legacy system, a super dumb premature optimisation. If by "microservices" you instead mean orchestrating a platform, this is part of another, higher-level look at *what government needs to be able to do to govern well* and part of that is, in partnership with the vendor and development community, making sure that they're able to control and direct the evolution of a platform that enables them to deliver the services that people need without too much vendor lockin.
Similarly, Agile just doesn't make any sense for contracting, and especially doesn't make any sense for the sort of contracting that the government does. While those onerous contracting requirements are indeed a terrible match for software development, they do still serve their original purpose of preventing or mitigating fraud.
I'm not sure how to address this, other than there are situations where customers have needs and requirements that, with all the will in the world, are not absolute and where good off-the-shelf commodity solutions just don't exist. In such situations, being able to research, understand, build services against those needs and then iterate again without having to spend time documenting requirements ahead of time (which we know will change, or will be wrong!) is a bonus. There has been a significant amount of work done in the past few weeks with the state's lawyers and with advisors from 18F and the Federal government who *have* gone through successful procurement for agile development services where the state's lawyers have been satisfied that they can meet requirements for detecting, preventing and mitigating fraud.
Don't hire startup guys to fix the government's IT problems. Hire a corporation that's successfully consolidated or integrated tons of disparate systems. It's less sexy, but it's a lot harder, and is a much more realistic analogy to the problems that the government solves -- dealing with a complicated legacy; a nebulous set of stakeholders; complicated and conflicting requirements that impede a "clean" implementation; etc...
I don't think anyone is suggesting that *startups* be hired to "fix the government's IT problems". No one can fix the government's IT problems other than government itself; saying things like that perpetuates the myth that IT is a "thing" that can be fixed in the first place. The situation we're in is that governments need to *understand* and integrate systems and services that work together that get the job done, and they also need to get better at dealing with stakeholders and conflicting requirements. Again, this is why one of my early recommendations was that the state empower a single person who can be the product owner - who is trusted to make decisions that are in the best interests of the service, without micromanaging from stakeholders.

I hope you understand that Code for America isn't "doing" this. We've been working with the state to reform *how the state treats technology* and how the state works internally *so that the state gets better technology outcomes*.
posted by danhon at 12:13 PM on December 8, 2015 [48 favorites]


@schmod, re your last comment.

Again, I think there's a vast continuum in one person's understanding of "microservices" to another's, and hopefully when the vendor presentation is up you'll see how the state intends to move forward. I don't think it's what you think it's going to be, though, and I completely understand why you might *think* it's going to be cargo-culting stupendously-small-microservices direction.

As someone who has been in the room and has been working with California and its Federal partners on this, I can't emphasize enough that everyone understands that this is *hard*. The issue is that the payoff, if this is done *well* is order of magnitudes better than what the state would expect to achieve through a stereotypical monolithic procurement (which I'm fairly certain to stand behind, having read the last draft).

For this work to succeed, and for government to get better, a bridge needs to be forged between people who *have* delivered good services, quickly and can cut through molasses *and* the people who understand the current legacy systems. *Both* parties are needed: the new to push the old harder, in a stronger direction, and the old to explain why things were done a certain way (and why they might need to continue to be done that way, too).

I've tried very hard at least in the tone of the blog posts that I've put together to demonstrate that everyone involved understands that this is *hard* and requires a fundamentally different way of working, and all of the associated changes that entails.

Are you saying the state shouldn't try because it's hard? Or that not enough people have stood up and immolated themselves demonstrating to your satisfaction that it *is* going to be hard?
posted by danhon at 12:20 PM on December 8, 2015 [20 favorites]


I've been following some of the 18F developments, as I'd really hope that Canada would try something along these lines.

One thing that did trouble me a bit about a recent services tender round reported by 18F is that it appears that competitive bids are allowed below what could be legal minimum wage for the work required. That's not trimming fat; that's "sharing economy" exploitation.
posted by scruss at 12:39 PM on December 8, 2015 [2 favorites]


scruss: "That's not trimming fat; that's "sharing economy" exploitation."

AKA "trimming bone".
posted by boo_radley at 12:40 PM on December 8, 2015 [3 favorites]


Just had to use privately made HR software that was a maze of stupidity, so I tend to believe the problem lies in when this stuff was made. Things worked differently 30 years ago.
posted by destro at 12:44 PM on December 8, 2015


Unfortunately GDS now apparently imploding as the Civil Service reasserts its right to hand massive contracts to the Right Sort of Huge corp.
posted by bonaldi at 12:55 PM on December 8, 2015 [2 favorites]


You'll take my COBOL from my cold dead hands.
posted by Damienmce at 12:57 PM on December 8, 2015 [4 favorites]


Damienmce: "You'll take my COBOL from my cold dead hands."

Good news: You get to keep it.
Bad news: We plan on nudging it into the pit along with your cold dead self.
posted by boo_radley at 1:21 PM on December 8, 2015 [5 favorites]


Don't hire startup guys to fix the government's IT problems. Hire a corporation that's successfully consolidated or integrated tons of disparate systems.

From what I understand, these corporations are few and far between, if they exist at all.
posted by panama joe at 1:32 PM on December 8, 2015 [7 favorites]


you can't simply import the Silicon Valley ethos into the public sector

there was a good article on this posted previously:
Park and Dickerson have been steadily recruiting an elite digital corps—a startup team, essentially, built mainly from the ranks of top private-sector companies—and embedding them within the U.S. government. Their purpose is to remake the digital systems by which government operates, to implement the kind of efficiency and agility and effectiveness that define Silicon Valley’s biggest successes, across everything from the IRS to Immigration Services... whether a small number of technologists can actually bring about vast changes within the most massive, powerful, bureaucratic regime on earth.

[I]f Obama’s tech team can successfully rebuild the digital infrastructure of Washington—an outcome that is by no means certain yet—you might not only change its functionality. You might transform Americans’ attitudes about government too. And you might even boost their waning feelings of empowerment in an ideologically riven country of 320 million people.

Which is not to say that replacing Washington’s culture with that of Silicon Valley should be the goal. Some hybrid of tech people who can innovate with patience rather than aggression may be more effective. Dickerson notes that government tech contractors, even the most skillful ones, face the arduous challenge of trying to repair an aging digital system without compromising any essential services. The method for issuing Social Security checks, for instance, relies upon old mainframe servers running on the dying COBOL computer language... In this case, the West Coast mentality could be counterproductive. "There’s an attitude in the entrepreneurial private sector where we don’t care what came before us: We’re going to disrupt it," Dickerson explains. "But we are not going to disrupt Social Security. That’s a big reason why it’s so hard to make these changes, because you can’t interrupt the flow of operations."

[...]

There is no Moore’s Law for ­government—at least not yet. And another thing that ­McDonough, Park, and Dickerson must confront is that this government startup will never have the same lean, concentrated focus of a private-sector company... the tech teams, ranging in size from five people to 50, that will be installed within 25 government agencies over the course of the next 18 months. These teams will consult regularly with USDS for guidance and may utilize 18F for its services... you might consider Washington’s tech landscape, as it currently exists, as a kind of brown and barren field. And on that field, consider each agency as having a fenced-in plot of land. The USDS works now as landscape architects—the ones who design what kind of trees and plantings will go in each plot, and who will do the work. The people at 18F function like a nursery and ­contractors—they’ll provide the healthiest trees and do the plantings, either on their own or via someone they trust. They’ll even teach you how to be a good gardener. Meanwhile, the tech teams at agencies like Education and Veterans will take what USDS and 18F advise to make their plot flourish.

The overarching goal here is to get everything to grow together—very tall, very fast, inevitably joining up into a forest canopy so as to create a functional and interconnected system... The goal here is to reveal how U.S. citizens use government websites, and to spark healthy competition among agencies to create more popular services. In keeping with the tech corps’ guiding principles, everything is open source, so outsiders are free to adapt the program. And they do: A few weeks after the analytics website went live, Philadelphia used the program for its own analytics website, which the 18F team considered a measure of success. Thanks to their open-source code, they had improved government without doing any extra work.
an ancillary benefit i'd reiterate is how transformational it would be to "transform Americans' attitudes about government... in an ideologically riven[*] country of 320 million people."

oh and the CTO of CfA is mefi's own migurski!

also btw...
CivicTech :P

---
[*]"As difficult as it surely will be, there is no substitute for restoring some measure of public and elite respect for government's enormous role in making society richer, healthier, fairer, better educated, and safer. To do that requires encouraging public officials to refine and express that case, and rewarding them when they do so. And it requires designing policies not to hide the role of government, but to make it both visible and popular."
posted by kliuless at 1:49 PM on December 8, 2015 [6 favorites]


Shouldn't this process start with an understanding of the services the state of California is delivering before picking a supplier? I'm nervous that the "IT solution" will be in search of a problem.
posted by grefo at 2:32 PM on December 8, 2015 [2 favorites]


Not sure if these Silicon Valley types are either extremely naive or extremely cynical. I'm just imagining a bunch of techpersons dropped into an episode of Yes, Minster. I imagine they'll do what many have done before; realize it's a losing battle, collect their money and do a passable job, and quit.

Another problem is that tech has this vision of themselves being successful on merit, rather than being successful due to gross distortions in our economy and monetary policy, plus all of the ego fluffing going on as part of court politics. Yet they're taking these distortions as evidence of their own efficacy, and now they're trying to reshape the face of the government based on what's built largely on the very inefficiencies they're trying to correct. It's like they're trying to kill their own parents in a way, a moment destined to come.
posted by gehenna_lion at 3:17 PM on December 8, 2015 [2 favorites]


gehenna_lion, what you’re describing is very much the present vendor ecosystem. The change being proposed here shifts a degree of power and control back to the state department commissioning the work, where it might previously have been held by a traditional large-scale “beltway bandit” contractor. The intent is to avoid a situation where SV types bail out, by ensuring that the Department of Social Services develops the knowledge and experience to own its services.
posted by migurski at 3:31 PM on December 8, 2015 [5 favorites]


I'm just imagining a bunch of techpersons dropped into an episode of Yes, Minster. I imagine they'll do what many have done before; realize it's a losing battle, collect their money and do a passable job, and quit.

You do know that you don't need to imagine "a bunch of techpersons dropped into Yes, Minister" because that's the story of GDS, right? And yes, some of the more familiar names at the top of the hierarchy moved on this year, but the structures and culture remains, so it's not quite the triumph of Sir Humphrey that you'd like to believe from role-playing in your head with caricatures.

One of the slides that shows up in a lot of GDS presentations is of a high window looking out over a busy London street. Someone's put up a piece of paper with an opening cut out, an arrow and the label 'USERS'. When you're shaping government technology, your users are... everyone. At the same time, "techpersons" are not an elided category, they're people who do things in technology, and they use government services and they have friends and families who use government services and some of those people want to make those services better, not "tech better", but better for the people they're meant to serve. Is that really just a drama of false consciousness?
posted by holgate at 3:33 PM on December 8, 2015 [3 favorites]


At the same time, "techpersons" are not an elided category, they're people who do things in technology, and they use government services and they have friends and families who use government services and some of those people want to make those services better, not "tech better", but better for the people they're meant to serve. Is that really just a drama of false consciousness?

I guess I'm just having a little fun with dark side of Silicon Valley's reputation. Facebook makes its money off strip-mining and selling off personal information that they elicit by tying a service in with manipulative social psychology that they experiment with on their own users without their consent; Uber makes its money off exploiting a broken government; so on and so forth. And then to read tales of utopian visions and praises of glory about an industry that's made its money either exploiting people or chipping away at the social fabric that ties society together with the government.

So I thought it was funny that this same industry would go in and try to make government services better for people, when it's the same industry that's been at the forefront of helping to dismantle and abuse the government and social fabric for their own profit. Tech isn't exactly a pro-social industry, at least its biggest and most celebrated successes, and that's part of the reputation you're working with here. Wolves, hen house, that sort-of thing.
posted by gehenna_lion at 3:45 PM on December 8, 2015 [3 favorites]


That’s a great point, but I do think the dynamic here is a bit different. It’s not the tech industry coming into government, it’s the hard-won expertise designing and running digital services. We now take it for granted that all services must be digital, so we might as well figure out how to transfer the world’s best knowledge in running big ones.
posted by migurski at 3:56 PM on December 8, 2015 [5 favorites]


That’s a great point

You're too kind. I understand how it can help, but it'd be nice if people actually figured out how to run digital services well in the first place. Other than knowing the best ways to strip-mine and sell data or get people addicted to certain things and then soak money from them. Elegant, useful designs, beautifully-run and efficient systems that respond to people in helpful, rather than cynical and harmful, ways aren't exactly big sources of revenue in the technology world.

In fact, those pro-social designs actually harm making money. So it's like you'd have to do the exact opposite of what you do in the business world to make it beneficial for the public.
posted by gehenna_lion at 4:07 PM on December 8, 2015


Nevermind, Code for America is doing what I wish other organizations would. Good job, I'm being too cynical for my own good here.
posted by gehenna_lion at 4:27 PM on December 8, 2015 [3 favorites]


Similarly, Agile just doesn't make any sense for contracting

Of course it does. The consulting company I work for does nothing but "agile" projects.

especially doesn't make any sense for the sort of contracting that the government does.

A large portion of our work is with government.

What doesn't make any sense is the status quo. Where large tech companies sign enormous contracts on waterfall projects and then proceed to drain every penny out of the government that they can while never delivering anything of value, but staying exactly on the right side of the contract.
posted by markr at 4:43 PM on December 8, 2015 [9 favorites]


I like the idea of reforming government IT. One of the things preserving my sanity at my federal agency is the fact that our departmental IT people don't actually know how to integrate Macs into their trainwreck of a security system, so I get a pass on the card reader, for now. But I also work for an agency whose web filter blocks access to the new federal web design guidelines, but not ESPN. Our mission is complex and our stakeholder community is large, which underscores the need for effective information systems. I hope this is a big success.
posted by wintermind at 5:37 PM on December 8, 2015 [2 favorites]


Of course it does. The consulting company I work for does nothing but "agile" projects.

Arguably, Agile was invented by consulting firms to enhance their bottom line.
posted by pwnguin at 6:23 PM on December 8, 2015 [1 favorite]


The success of a bunch of startup guys rescuing healthcare.gov is a direct counterpoint.

A lot of what happened during the Healthcare.gov fix was suspending a *lot* of minor requirements, interoperability among dozens (hundreds?) of disparate systems and and not supporting the many, many edge cases inside the system, under Presidential directive, to focus on stabilizing and repairing the core product. This would *never* have happened with the existing companies and rules running that project. It took a major crisis to suspend those requirements, and that was integral to actually fixing the system. I don't want to take anything away from the people who worked on this problem, but it was not purely miracle working in the face of incompetent Government IT staff.
posted by cnc at 6:57 PM on December 8, 2015 [4 favorites]


Agile, yay! No need to document shit.
posted by RedShrek at 7:20 PM on December 8, 2015 [3 favorites]


cnc: "This would *never* have happened with the existing companies and rules running that project. It took a major crisis to suspend those requirements"

It's debatable if those requirements could have been suspended by anyone, outside of a crisis.

One of the key distinctions of government projects is that they (often necessarily) have a huge number of stakeholders, simply because it's not acceptable to eschew any portion of the population.
posted by schmod at 7:40 PM on December 8, 2015 [2 favorites]


[Can I just say how much I appreciate y danhon and migurski's participation in this thread? I don't think I've ever been so happy to be proven wrong. Those comments are a masterclass in calming down a confused ally. I agree on virtually all of the points that they brought up (oh, and I also signed up for my local CfA-affiliated meetup next month)]

One thing I will stick by, though -- the government should be borrowing practices, expertise, designers, and engineers from industry. I'm far more skeptical of any actual value that a C-suite "Ideas guy" is going to bring to the table.
posted by schmod at 7:47 PM on December 8, 2015 [10 favorites]


I’m happy to have Metafilter to chat about this too, so thanks schmod! Hope you enjoy the meetup.
One of the key distinctions of government projects is that they (often necessarily) have a huge number of stakeholders, simply because it's not acceptable to eschew any portion of the population.
Very much yes, or as Russell Davies formerly of the GDS put it: public servants don’t pivot.

There are other ways to shrink a problem though, and that’s where agile planning comes in. Right now, these efforts balloon in scope and budget because they’re planned like prenuptial agreements, with all imaginable adverse outcomes listed and documented in copious detail ahead of time. A single omnibus contract makes doom-prepping necessary. If you switch to a process that uses multiple contracts, you get natural breakpoints along the way, adjust the work as you go, and still serve 100% of the intended population with a slowly-expanding scope. If a vendor screws up, you’re not out $XXX million. In return, the state has to employ its own technical project management, user research, and subject matter experts to guide the work and keep records (that’s one place where documentation comes from, RedShrek).

In this case, the state is starting with two modules: a data management platform and an intake module for responding to allegations of child abuse. They’re described in slides 15-19 of Friday’s presentation linked from the New System website.
posted by migurski at 9:55 PM on December 8, 2015 [7 favorites]


Agile, yay! No need to document shit.
I'm well aware of how Agile can become a cargo cult, how it can be done wrong, how it can be a cover for flailing or the same old CYA extraction of money. But the core idea here, not of Agile but of being agile, is just to do things in small pieces and to think about what you're doing. That's perfectly compatible with documentation and both short- and long-term planning.

It's just that as Fred Brooks put it, "For most human makers of things, the incompleteness and inconsistencies of our ideas become clear only during implementation." We often need to start making the solution to discover what making it means. Iteration can still mean documentation. You just have to make it a project management priority. Documentation of a system as it's being built, rather than all up front before it exists or after the entire monolith is completed and no one remembers what happened last year, seems like a fine approach to me.
posted by daveliepmann at 10:59 PM on December 8, 2015 [12 favorites]


That’s a really good Fred Brooks quote.
posted by migurski at 11:06 PM on December 8, 2015 [1 favorite]


This is why Finland is able to implement the basic income experiment (in an age where so many government are left powerless with even smallest of changes in the way society works) - "Instead of speculating on the impact of proposed policies – such as basic income and environmental policies – Finland will now experiment, measure and scale."

Finland is introducing experimentation to politics on both national and city level
“It’s bizarre that the rest of the society works with testing, prototyping and then scaling, but not governance. It makes politics very theoretical, slow and to rely on guesses as opposed evidence,” explains researcher Mikko Annala of Demos Helsinki. He was part of the team that designed of the experimentation model.

“There’s a lot going on in government innovation right now, with initiatives such as the ‘Nudge unit’[*] in UK and the Mindlab in Denmark, but we wanted to take this a step further, with large experiments and scaling up to the policy level,” Annala explains. “What the typical government innovations units lack is a feedback loop to policy. That is different with the Design for Government initiative. Now the experiments are designed to scale from the start.”
agile gov't :P
posted by kliuless at 11:56 PM on December 8, 2015 [5 favorites]


I work for a Management Information Systems team for a consultancy who works with both the UK and US governments, so Holgate, and all commentators, I hope you won't mind if I steal basically this entire post and your comments practically word for word and use it as an in house article on corporate strategy.





*Note, I will claim it all as my own "research" and you will not be recompensed in any way and I will get a big promotion and a corner office and will have saved IT consulting and also christmas.
posted by Just this guy, y'know at 2:11 AM on December 9, 2015 [7 favorites]


This is fantastic, and just so much more sane.

Incidentally, is 18F mostly comprised of people that can commit startup-like hours (without startup growth-at-all-costs thinking, of course) currently? I love what 18F is doing and enjoyed the one CFA project meeting I went to, but I have a toddler now and can't work more than eight hours a day. The whole "sensible technology in government" thing does seem really stormy right now, so I'm kinda hoping it survives and stabilizes in a few years and can accept help from people with other commitments.
posted by ignignokt at 7:51 AM on December 9, 2015


18F has a couple day-in-the-life posts that might shed some light, ignignokt. The word from friends there is that it’s a sane ~40hrs/week situation. At the SF office, the lights automatically dim at 5:30 to remind everyone to get out.
posted by migurski at 9:28 AM on December 9, 2015 [5 favorites]


My view of these problems from my position as an employee at an FFRDC is that efforts like this are very welcome, but ultimately doomed until more can be done to narrow the pay gap between GS pay grades and what folks can earn in the private sector. 18F has more flexibility than many other agencies, and opening offices outside of DC helps avoid the cost of living problem, but even accounting for the fact that some people want the security of a government job and the good feels that come with public service, the pay gap is still way too large for a vast majority of government workers. Over time, this leads to more turnover and ultimately a lack of institutional knowledge, which is very hard to overcome when the institution you're dealing with is the federal government.

We shouldn't be counting on people to be patriotic enough to take much lower pay for the same job, and in my experience, it simply doesn't work.
posted by tonycpsu at 1:30 PM on December 9, 2015


Scruss, I think I have some backchannel information for your comment about the recent micro-purchasing experiment that 18F ran.

*Apparently* - and I'm not completely sure where I heard this from, but it may well have been from the $1 bidder's blog, the bidder in question described their work as "something they would have done anyway as part of the open source community" and wanted to "give back" to government. Which, you know, all well and good but illustrates the difficulty between opening up "how government does things" and enabling people to contribute (which is, net-net, a good thing, I guess?) to "society" but at the same time does completely open up a whole can of worms relating to privilege and being able to under-bid. If this is something that the government *should* be paying for, then shouldn't there be a minimum bid, for example?

From my understanding of this situation, the experiment wasn't *intended* to be a sharing-economy-style-exploitation, but instead a genuine (if perhaps naive) experiment to see whether a reverse-auction bid for small bits of functionality could work in the first place. Whether or not one might have reasonably anticipated a troll-ish $1 bid is left as an exercise for the reader.
posted by danhon at 10:04 PM on December 9, 2015


Grefo wrote:
Shouldn't this process start with an understanding of the services the state of California is delivering before picking a supplier? I'm nervous that the "IT solution" will be in search of a problem.
Yes - that's what's happening (and has been happening for, in different capacities the last two to seven years). The procurement that was *going* to happen had been running for the last two years in a more-or-less traditional waterfall, long-list of requirements type process that probably would have anticipated a single bidder. Please understand that a lot of work has gone into understanding a) what services the state is currently required to deliver; b) what services the state is currently able to deliver; c) how to evaluate different suppliers.

To be clear: no supplier has been picked. The RFPs for the smaller "modules" haven't been issued. If anything, the state is getting clearer on making sure it understands what its users need and that it's better at specifying and picking suppliers who are capable of meeting those needs.
posted by danhon at 10:08 PM on December 9, 2015


Gehanna wrote:
Not sure if these Silicon Valley types are either extremely naive or extremely cynical. I'm just imagining a bunch of techpersons dropped into an episode of Yes, Minster. I imagine they'll do what many have done before; realize it's a losing battle, collect their money and do a passable job, and quit.
So, from my point of view, the Silicon Valley types - the bundle of techpersons, if you will - aren't the people who're going to fix this. They're only the people who will help deliver better technology in the first place. The only way in which the outcome is really going to change is if there's genuine transformation (sorry, bit of a buzzword) from both the bottom-up *and* the top down. I'm happy to say (and feel free to say that I'm naive!) that both the bottom-up staff *and* executive leadership at the cabinet secretary level are pushing for this change. Sure, not *everyone* is. But most people can see which way the wind is blowing and are making decisions as to whether they want to be actively obstructive, passive or actively helpful.

I've never worked in government before, but after having met a lot of people who *do* work in government, I can safely say that Not All Government Workers want business as usual, and Some Of Them Are Even In Leadership. In this particular case, I would prefer to believe (again, possibly being naive) that there's been a perfect storm combination of right thing, right people, right time.
posted by danhon at 10:12 PM on December 9, 2015 [1 favorite]


So cnc wrote, about the Healthcare.gov "recovery" effort, that:
It took a major crisis to suspend those requirements, and that was integral to actually fixing the system.
and I think that honestly hits the nail on the head.

Imagine, though, the following hypothetical conversation where one might say that if nothing about the present situation were changed, which would include certain requirements and policy and business as usual then a failure of such proportion as Healthcare.gov's initial release is more likely than not.

In other words: there are people out there who understand that it's the environment, system and policy in which these technology procurements operate that is in large part responsible for their high likelihood of failure. When you have leadership who're willing to look at *how things are done*, *whether those things need to be changed* and that *nothing that isn't illegal is off the table* and the freedom to follow best practices to improve outcomes, then I think you have a shot at success.
posted by danhon at 10:17 PM on December 9, 2015 [1 favorite]


Schmod wrote that:
One of the key distinctions of government projects is that they (often necessarily) have a huge number of stakeholders, simply because it's not acceptable to eschew any portion of the population.
... which is why one of my recommendations was that the state needs to have people who can fill the role of a Product or Service Owner. A single, directly responsible individual who is empowered to make decisions in the context of the product or service and is able to prioritize work. Because a committee isn't going to do that, and you'll never get anything good.

Of course then, the job is to make sure that the product owners have the trust of the stakeholders :)

(and the key to *that*, is that stakeholders are only *really* going to trust you if you've demonstrated to them that you can deliver)
posted by danhon at 10:19 PM on December 9, 2015 [1 favorite]


Schmod also wrote that
the government should be borrowing practices, expertise, designers, and engineers from industry. I'm far more skeptical of any actual value that a C-suite "Ideas guy" is going to bring to the table.
... with which I agree completely, and even to the extent that some of these roles need to come back to government so government can actually be good at technology at the first place. Because in this day and age, if you don't understand and control technology that's vital to doing your job, then you're not able to do your job properly -- in this case, child welfare.

This is why in my second blog post, I included a photo of the project team. They're the people who're going to make this work, not some C-suite Innovation Consultant who's got some new ideas about Agile and Iterative and shit. That project team, with the support, skills and capabilities that it needs, is going to be the key to delivering better tech.
posted by danhon at 10:23 PM on December 9, 2015 [1 favorite]


danhon: “For this work to succeed, and for government to get better, a bridge needs to be forged between people who *have* delivered good services, quickly and can cut through molasses *and* the people who understand the current legacy systems. ”
I think that part of the problem is that outfits that have that kind of capability get frozen out at the RFQ stage. My partner and I are exactly that kind of team, but as the old saying goes, "Nobody ever got fired for buying IBM." So even on small-potatoes contracts and with us getting extra points for being small, local, and a woman-owned business the work always goes to a big outfit because of the inherent conservatism of state procurement regulations.
posted by ob1quixote at 12:22 AM on December 10, 2015 [1 favorite]


ob1quixote wrote:
I think that part of the problem is that outfits that have that kind of capability get frozen out at the RFQ stage.
Absolutely. I think the state understands that they need to be a) clear on the change in vendor requirements and capabilities and b) change their procurement policy and process so that vendors who satisfy those requirements and capabilities actually get in.

So, uh, my understanding is that the state understands that if the RFQs prevent qualified, capable vendors from taking part, they've shot themselves in the foot.
posted by danhon at 4:41 PM on December 10, 2015 [1 favorite]


Update: California is holding its second vendor forum today at 2pm PST, this time online-only as a webinar, where they'll also be taking questions.

via the website for the New System.
posted by danhon at 1:50 PM on December 18, 2015


hey, i just picked up the silo effect by gillian tett from the library and thought of this thread; check out the intro on mayor bloomberg's 'skunkworks' and the unlikely genesis of (precrime) 'predictive analytics' in policing! the individual stories/case studies are really interesting in the challenges and benefits of breaking down silos, esp for gov't, but also more generally from the anthropological perspective it provides on human behaviour and organisation :P
posted by kliuless at 6:48 AM on January 4, 2016 [2 favorites]


« Older Professor Kanye and the Business of Yeezus   |   I Was Internet Famous Newer »


This thread has been archived and is closed to new comments