Did agile processes contribute to HealthCare.gov's problems?
The well-documented critiques of the front-end and back-end software on HealthCare.gov beg the question of whether this critical site was properly designed — or, even worse, designed at all.
It has been reported that the front end was designed with an agile process; unfortunately, most agile processes reject and discourage “big design up front.” In a nutshell, many agile processes — and especially extreme programming — reject the big design phase as part and parcel of rejecting the waterfall methodology. Agile processes follow more of an “organic” software development, where developers start coding the smallest increment possible and “grow” the working software up, little by little, with constant customer feedback. These agile methodologies call for “user stories” to design each small increment of the system being developed. To be fair, agile can work for some software projects, but I assert that it is the kiss of death for projects with many moving parts, multiple organizations and complex interactions.
Personally, I am a fan of design and system architecture, and I have witnessed many successful projects that resulted from good design. Furthermore, I flatly reject the notion that user stories can suffice for large IT projects like HealthCare.gov that require scalability, data integration, numerous system interfaces and other complexities. These need to be designed up front or the results can be disastrous, as the HealthCare.gov launch demonstrates. Where is the documentation for this system? I just submitted a FOIA request for it and will report on the results once I get a response.
The assertion by federal CTO Todd Park that volume was the only reason for the failures is a weak excuse at best, because designing for scalability (or better yet, designing for elasticity) should have been an obvious upfront requirement. For example, if HealthCare.gov uses a synchronous interface, as reported by Paul Smith, then it was not architected properly. Given that developers had three years to prepare for the Oct. 1 launch, improper design is inexcusable.
The Health and Human Services Department should come clean on these deficiencies for the sake of the rest of the federal agencies that could learn from its mistakes. Those who never admit their failures will never learn from them. Other federal agencies need to learn from HealthCare.gov. Those painful lessons on system design (or the lack thereof), system architecture (or the lack thereof) and elasticity (or the lack thereof) might then not be repeated.