MOAR STRUTS FOR SPACE X ROCKETS
July 22, 2015 6:34 AM   Subscribe

 
Really?

Seems pretty straightforward to me. Critical component fails due to manufacturing weakness/lack of testing. Space X isn't giving enormous detail on this, but enough to be sure of the basics, assuming there isn't some big gob of evidence otherwise that's yet to be discovered or revealed. So far, it looks like a relatively simple fix - no big redesign needed - and while it might indicate that Space X's component qualification and system testing regime isn't up to scratch it shouldn't be a show-stopper. This is the sort of thing that happens when you're doing this sort of thing.

I think without having much more detail of failure modes and component specs, MS Paint and Mr Manley are perfectly up to the task of illustrating the course of events and the reasons behind them, which are straightforward material science and physics.
posted by Devonian at 7:01 AM on July 22, 2015 [4 favorites]


Space X's component qualification and system testing regime isn't up to scratch

There is no company in the history of anything whose testing regime is up to scratch. In my experience, test programs tend to go like this:

-development parts built
-test is run and failed
-office looks like coop full of headless chickens
-redesign implemented
-"oh crap new testing will cost $XX million"
-hand waving
-redesign approved for production
posted by backseatpilot at 7:12 AM on July 22, 2015 [7 favorites]


I seemed to have watched a very different video than you did, odinsdream. The cheesy graphics (which were disclaimed right up front) aside, I thought it was clear and informative, and the only part that I think goes beyond typical understanding of the physics going on is the fact that during higher acceleration, the helium tank buoyancy increases.

And I think he explained that well, too.
posted by chimaera at 7:29 AM on July 22, 2015 [2 favorites]


But really, the supplier knows what this will be used for, and they have a product that fails at 20% of spec? They better have good insurance.
posted by sammyo at 7:49 AM on July 22, 2015 [1 favorite]


Spacefight is hard, MS Paint is easy.
posted by fairmettle at 7:55 AM on July 22, 2015


He did this fantastic video explanation of recent near Earth asteroid discoveries. Turn up your resolution to eleven and full screen.
posted by bonobothegreat at 8:04 AM on July 22, 2015 [2 favorites]


backstreetpilot - yes, well, reality is no respecter of Gantt charts (considers rant about devops, decides against).

However, if a statistically significant proportion of a bought-in component fails at 20 percent of spec, I think that whatever processes allowed these components to get to flight is flawed. Given the small numbers and short in-use life of the struts, especially.

Quick! To the retrospectoscope!
posted by Devonian at 8:08 AM on July 22, 2015 [1 favorite]


But really, the supplier knows what this will be used for, and they have a product that fails at 20% of spec?

*Sometimes* fails at 20% of spec. This seems to be a pretty common failure mode for aerospace items. A component has a 5% defect rate, due to something unusual in the manufacturing steps that only happens occasionally. Like one worker on the crew taking a shortcut. No one finds it in spot testing, but after it fails, they test every part and find the real defect rate.
posted by smackfu at 8:15 AM on July 22, 2015


Why wouldn't the the helium tanks be positioned further up to reduce stress? Or was he just exaggerating to make the explanation clearer?
posted by ChurchHatesTucker at 8:27 AM on July 22, 2015


I found the bit about how they tracked the problem down using all the accelerometers they had attached to the rocket fascinating. It sounds like pretty much the same technique that they used to figure out exactly where an earthquake happened, except that it's happening on a rocket, where the amount of noise that had nothing to do with the failure must've made picking out the actual what-went-wrong a challenge. (Unlike an earthquake, where there's a big, clear spike on the seismograph to look at.)
posted by clawsoon at 8:29 AM on July 22, 2015 [2 favorites]


This seems to be a pretty common failure mode for aerospace items.

That "sometimes" value for flight critical components is once every 10^9 flight hours (possibly even higher for space vehicles, I'm not sure). Of course, aircraft generally fail more gracefully than spacecraft.

Why wouldn't the the helium tanks be positioned further up to reduce stress?

If they stay at the bottom of the tank, they stay immersed in LOX until the tank is basically empty. That way, they stay cooler for longer; if the helium tanks are exposed their temperatures rise and the helium starts expanding, which will cause the helium tanks to rupture.
posted by backseatpilot at 8:29 AM on July 22, 2015 [2 favorites]


I love that kerbalspaceprogram is a tag for this
posted by MysticMCJ at 8:31 AM on July 22, 2015


No doubt there will be a conspiracy theory about this because it shows a round grey object impacting the rocket in the simulation video at the end.
posted by boilermonster at 8:43 AM on July 22, 2015


He did this fantastic video explanation of recent near Earth asteroid discoveries.

Also this recent 360 video (click and drag to look around, at least on desktop) visualizing near-Earth asteroid positions.
posted by figurant at 8:49 AM on July 22, 2015


Scott Manley's video is excellent at presenting the important facts about what happened to the SpaceX's Falcon 9 during the CRS-7 launch in a manner that anyone can understand. The simplified drawings make it much easier to focus on the important parts and get a clear picture of what happened.

This is a video from inside the Falcon 9 second stage oxygen tank. Those black cylinders at the bottom and the one at the top-right are the helium tanks.

The strut that failed is a common component on Falcon 9 both in its first and second stages. SpaceX has flown literally thousands of them without problem. In fact SpaceX didn't believe the strut to have failed at first. They had to test thousands of the struts they had in stock before one failed. The failure rate on these struts is very, very small; we're talking a 1/100th of 1 percent.

These struts are manufactured in bulk. Testing, by either the buyer or the manufacturer, of each individual strut is not common. Instead samples from each batch are tested to verify the entire batch meets the spec. Faulting SpaceX for not previously testing every strut is to fault the industry standard. The industry won't change its standard, but SpaceX will.

And while each tank is tested on the ground, the problem here is that you can't test the exact conditions of a tank in-flight when it's on the ground. In-flight the tank is experiencing a few extra g's because of the vehicle's acceleration which puts more load on the struts keeping the helium tanks in place. Those extra couple of g's made the difference between a working oxygen tank on the ground and a failure in-flight.
posted by ruthsarian at 8:52 AM on July 22, 2015 [4 favorites]


Instead samples from each batch are tested to verify the entire batch meets the spec

The real trouble is that you can't use destructive tests to test every strut.
posted by smackfu at 8:54 AM on July 22, 2015 [2 favorites]


@ChurchHatesTucker - he briefly mentioned that, i think. the rocket is accelerating while the motors burn. and while it's accelerating the liquid oxygen will be pushed to the bottom of the container. so while the rockets are burning the liquid oxygen is "used up" from the top down.

so... if you put the helium lower down, it stays in the liquid oxygen longer. and liquid oxygen is cold. so what he guessed (somewhere pretty near the start of the video) was that the helium is kept down in the cold so that it can contain more gas and/or have a lower pressure (and so lower weight tanks) for the gas it needs to contain.
posted by andrewcooke at 9:05 AM on July 22, 2015


I love that kerbalspaceprogram is a tag for this

Just to remind people who haven't pieced it together, Scott Manley is the guy who's done an extensive series of YouTube video doing amazing things in Kerbal Space Program, and has actual relevant education on this subject. I heard somewhere that he's an actual rocket scientist, although maybe that's me misremembering.
posted by JHarris at 9:06 AM on July 22, 2015


The industry won't change its standard, but SpaceX will.

surely spacex and industry work the same - they use statistical sampling until they find something that seems to have outlier failures, and then test that.

they don't own common sense.
posted by andrewcooke at 9:09 AM on July 22, 2015 [1 favorite]


The real trouble is that you can't use destructive tests to test every strut.

Well, you could...

But I think the lesson here is, sometimes shit's just going to blow up. They might be able to push the defect rate for the struts and other components down even further than its already pretty impressively low rate, but I doubt it could ever be zero. And in spacecraft, any minor defect tends to end up like this.

I'm more surprised that there wasn't a mechanism in place to rescue the actual payload capsule since it survived the event and was happily transmitting "all is well, ground is getting closer" until it crashed. If they'd been able to tell it to deploy its parachutes, it would have been perfectly fine.
posted by Naberius at 9:11 AM on July 22, 2015 [1 favorite]


I heard somewhere that he's an actual rocket scientist, although maybe that's me misremembering.

He has an undergraduate in astrophysics and a masters in computational physics.
posted by a lungful of dragon at 9:11 AM on July 22, 2015 [1 favorite]


And he admits at the end of the video (which I've now seen through to the very very end) that he's not! not a rocket scientist!
posted by JHarris at 9:36 AM on July 22, 2015


Related: 'The six words you never say at NASA' (xkcd link, of course.)
posted by Zack_Replica at 9:48 AM on July 22, 2015 [1 favorite]


Regarding the capsule, I think they're trying to figure out a way to have chutes that could open in order to save the cargo, in case something like this happens again.
posted by fifteen schnitzengruben is my limit at 9:56 AM on July 22, 2015


Why wouldn't the the helium tanks be positioned further up to reduce stress?

If they stay at the bottom of the tank, they stay immersed in LOX until the tank is basically empty. That way, they stay cooler for longer; if the helium tanks are exposed their temperatures rise and the helium starts expanding, which will cause the helium tanks to rupture.
Anyway, it would only make things worse.

Buoyancy doesn't depend on how deep the tanks are, as long as they're completely submerged. But suppose they start out submerged and later emerge. First they'll experience buoyancy forces in one direction and later they'll experience their weight in the other.
posted by sjswitzer at 10:04 AM on July 22, 2015 [1 favorite]


ahem.

MOAR STRUTS!

(I cannot believe nobody made that joke yet!)

Zeroth: Don't need fancy graphics, we used to do this stuff with overhead slides, kids!

First: This is the *suspected* cause, based on deep telemetry analysis. What they really need to confirm it is to find the actual failed strut to confirm this.

Second: What this means is they may have what NASA calls a Criticality 1 issue -- a part that *cannot* be allowed to fail, period. If one strut failing causes a LOV, they either need to ensure that the part *cannot* fail, or they need to make sure that the part is redundant.

Redundant adds mass, which means less mass to orbit, of course. But "cannot fail" is a *very* hard standard to reach. You think "Oh, we can just test every one to 300% load, and use only the ones that pass" but then you find that one of them, which would have handled the load initially, now fails because of the stress from the test, and you have a LOV.

In fact, NASA considers "cannot fail" an almost impossibly high standard to reach -- six sigma is a minimum -- and usually uses redundancy to handle criticality 1 components.

If they'd been able to tell it to deploy its parachutes, it would have been perfectly fine.

Maybe. It was moving at very high velocity, there is some question if it would have just shredded the parachutes. But, hey, a *chance* of a successful deploy beats the hell out of *no* chance of a successful deploy, which is what they had with the software flying on the Dragon. So, yeah, they've already said that's going into the very next flight.

Indeed, I think they need to get the full escape system flying ASAP -- it would have been a big confidence boost for NASA if the Dragon had detected the failure, boosted away on the escape system, and came down safely. But they have the landing chutes qualified now, so they can get that software implemented quickly, they've only just started pad tests on the boosters of the LES -- successful so far, but that's 1 of 1 test.
posted by eriko at 10:06 AM on July 22, 2015 [2 favorites]


HULLO! I'M SCOTT MANLEY AND YOU'RE READING THIS IN MY VOICE!
posted by eriko at 10:07 AM on July 22, 2015 [9 favorites]


Missed a chance at the MOAR STRUTS tag, too
posted by eriko at 10:08 AM on July 22, 2015


Scott Manley is pretty damned awesome. Not only wouldn't I have got so far in Kerbal Space Program, but I probably wouldn't have bought the thing if his tutorials weren't so useful and easy to understand.

And yes, all orbital maneuvres are narrated, in my head, by Scott Manley's voice.
posted by Zack_Replica at 10:20 AM on July 22, 2015 [1 favorite]


I think they're trying to figure out a way to have chutes that could open in order to save the cargo

Ah! I had interpreted what I saw regarding that as there were chutes and they just didn't have a means to deploy them outside their expected operation. If there actually weren't any chutes, then that goes a lot farther toward explaining why there was no way to open them.

So can someone who actually knows clarify? Eriko, it sounds like you're saying the capsule does have them.
posted by Naberius at 10:23 AM on July 22, 2015


The testing doesn't have to be destructive. Things don't 'just fail', they fail because there's something different about them from working parts, and if you understand what that is then you can work up a check for it. Not always, but a strut won't have many failure modes when properly installed and used. Aeronautical engineering is good at this, and good at testing when it knows there's something that needs to be tested, so I have a high confidence that this particular problem will be suitably overcome. It may even be as simple as putting in moar struts, although you never want more weight than absolutely necessary. Or any less.
posted by Devonian at 10:35 AM on July 22, 2015 [1 favorite]


So can someone who actually knows clarify? Eriko, it sounds like you're saying the capsule does have them.

Yes, other than CRS-7, all the CRS missions have had the Dragon splash down at end of mission and had parachutes for this. What they didn't have was the launch escape software installed because the LES isn't built yet, so the parachutes didn't even try to deploy after the booster failure.
posted by eriko at 10:37 AM on July 22, 2015 [1 favorite]


Helium! (Not relevant, it just amuses me.)
posted by CheeseDigestsAll at 10:39 AM on July 22, 2015 [1 favorite]


MOAR STRUTS!

(I cannot believe nobody made that joke yet!)


Why would anybody make a joke that's already been made in the title ?
posted by Pendragon at 10:43 AM on July 22, 2015


Why would anybody make a joke that's already been made in the title ?

Umm, nothing to see here, move along.
posted by eriko at 10:46 AM on July 22, 2015 [1 favorite]


The testing doesn't have to be destructive.

Test items are simply not flown, full stop. We run equipment that costs upwards of six figures through batteries of environmental and EMI testing and despite the cost those items are never installed in aircraft. They get put in integration labs when we're done with them. Production parts are put through an environmental stress screen, but that's just to weed out infant mortality and it doesn't significantly reduce the life of the parts.

Failure's a complicated topic, it's really not a binary fail/won't fail. You can obviously overload a strut and snap it in half, and that's definitely failure. However, that's just looking at static systems; a rocket not only moves, but shakes and vibrates. That vibration causes cyclic loading that may be much lower than the ultimate strength of the material but be even more damaging than the static load. Cyclic loading opens up stress concentrations and causes cracks to grow, which can cause more complicated failures.

I come from a slightly different background (we actually expect our aircraft to fly more than once), so we look at failure differently. Failure for me isn't "part falls off and everyone dies", it's much more conservative than that - more like "crack that we don't know is there presumably grows to a certain length". We design everything with safety margins as generous as the plane will allow, but static overload is a really uncommon failure mode; cyclic loading is much more frequently the culprit. A static load test won't uncover these kinds of failures.

Obviously I don't have any more information on this accident non-nominal flight than anyone else not working at SpaceX, but as a betting man I'd put my money on fatigue-related failure caused either by a manufacturing or assembly defect. Machining and welding can cause microflaws that in the right vibration environment can quickly runaway into full-on fractures, or maybe holes were cut incorrectly, bolts overtightened, etc. These aren't things that are easily tested for, and the tests that do exist often require using up a significant portion of the part's life to uncover flaws. When you're optimizing for weight and cutting safety margins to the bone because of that, you can't waste that life on testing that may reject one part in ten thousand. And now all your accepted parts are that much more likely to fail.
posted by backseatpilot at 12:12 PM on July 22, 2015 [3 favorites]


Also worth watching is Scott Manley's Kerbal Space Program-based simulation of Falcon 9's oddball vertical barge landing. It's about 11 minutes long and features plenty of explanation on the how and why of this method of recovery. His first couple of attempts turned our more or less the same as SpaceX's attempts, while is last landing technically fails due to him landing an overfueled rocket in the game.
posted by Sunburnt at 1:05 PM on July 22, 2015 [1 favorite]


« Older 13. Snakes   |   Hell, there's an energy drink straight up called... Newer »


This thread has been archived and is closed to new comments