What the heck could be going on? My friend stated the obvious: “It's clear that they're getting more traffic than they can handle. The question is why they can't handle the traffic they're getting.” Load problems could explain servers hanging in California and New York … but the drop-downs? The standard explanation for this is “high load,” but high server loads don’t cause your security dropboxes to empty out.

“The drop-down thing is mystifying,” he told me. If federal exchanges decided to populate the security question fields by calling up a list of possible questions from another server -- one that didn’t have a lot of capacity -- then that might be causing the sign-up process to stall at that step. For an application that expects a lot of traffic, this is a very bad idea.

“Just cache them on the front ends, for heaven's sake, so you only need to ask once,” he said. “A database call to get questions shouldn't be in the critical serving path. If you're hitting the database just to load the security questions, then just serving individual pages is going to be expensive.”

The various glitches, he pointed out, “could very easily be because deadline pressure caused them to take some shortcuts that impacted their ability to scale.”

Such as?

“The aforementioned let's-hit-the-database-for-security-questions thing.”

Why would they use such a seemingly obvious poor design?

“It can be easier to make a call to another server to get something when you need it than to implement a cache that you prepopulate either from static files or from the database on startup. Making a call to another server is also something you'd naturally think to do if you hadn't had to focus on scalability before. The security question page is probably not the thing you're most concerned about, so you give it to the new hire to do as their starter project. They don't know what they're doing, so they implement it the straightforward way … and since you're under unbelievable deadline pressure to get something working now nobody reviews it in detail.”

Obviously, we don’t know if this theory is correct -- but it does fit the particulars.

Government programmers are subject to the same development pressures as the rest of us. Via Untangling Obamacare's Web Glitches - Bloomberg.