Diss SpaceX
Any SpaceX Engineers want to share some process advice with “old space”?
I work for what y’all would probably consider an “old space” entity and lately have been trying to figure out how to improve and sculpt our development and verification processes. Obviously SpaceX has been doing innovations in this regard, and in my opinion, is the distinguishing factor in their effectiveness. We have brilliant GN&C people, but if it takes 8 hours to run a sim when the dragon engineers have flight hardware on their desk, one of those teams is going to be able to test and improve ideas faster.
So here are some things I’ve sort of learned that I think spaceX is capitalizing on that others aren’t as much, fill in the blanks if you can.
Using proven COTS products when possible
When a proven COTS product is not available, build it in-house to reduce the chance of a garbage contractor burning you (we suffer from this a lot)
A focus on modern software development practices and applying that attitude to the vehicle. SpaceX benefits from being in the field of launch vehicles, which can be tested more readily in the actual operating environment than say a Martian orbiter/lander. I’d say they probably focus more on getting compile time and sim time tightened up as much as possible as well but I don’t know that for a fact.
Generally having less assessment time and more actually just make a decision and build it time to get it to the environment quicker. Again maybe not possible to extend as extremely to things outside of LEO but we’ll see. I know they couldn’t do this as much with dragon, which is closer to what I would want to emulate at my employer as opposed to starship.
What else am I missing? I don’t think there’s any great reason other entities can’t operate as quickly as SpaceX does, I really think eventually the methods used will spread and gain traction. Let’s speed that up a bit ;). If it makes you feel any better about spilling my employer is in no way competing with SpaceX, we only ever collaborate. Help us help you!
I'm in "old space" right now, and I'm fairly young.
I've found so many things I'd like to change coming from an agile background. I'll keep my list short, though:
GD&T needs to go. If the prototype part works, it works. That's it. You can load test it, but if there's any reason you're not automatically assembling it after machining, you're doing something wrong.
Stop using drawings for assemblies and internally machined parts (NX Drafting). I've spent 90%+ of my time making a drawing correct when the model and notes themselves were perfect. It's just stupid. Document your work and design intent - but don't have such specific requirements for drawings that make you churn out 10x fewer components.
Stop holding "delegation" meetings. If you see something that needs to be done, do it. If you need someone to do something, add it to their backlog and talk to them about it. We don't need a 3 hour weekly meeting to discuss who is going to work on what for these few specific components.
Improve prototype manufacturing flow. If I want to test an idea, let me make a cheap (maybe scaled down) version of it without having to go through the typical release process (which takes months and multiple peer reviews).
Test complete assemblies more often. Continue to test individual parts, but if you put it all together and test it, you can pretty quickly find a bunch of issues - especially stuff like interferences and material property issues.
EDIT: edited point 1 to better express my frustration with GD&T for prototype parts.
GD&T needs to go. If the part works, it works. That's it. You'll load test it, but if there's any reason you're not automatically assembling it after machining, you're doing something wrong.
I hate doing GD&T as much as the next guy but this is absurd. Manufacturing processess are not infinitely precise or repeatable. Finding out the part doesn't fit isn't suddenly ok just because you test fit it immediately after machining. There are very few instances where a machined part can be immediately assembled without secondary operations anyway (is machining suddenly the only manufacturing process in existence?). And what about replacement parts? Without GD&T (or something like it), there is no guarantee the part will fit in every assembly.
You're basically advocating for reverting to a world of before interchangeable parts.
I can't talk about space but for comercial nuclear its very similar phenomenon from what I can tell.
Extreme reluctance to try new technologies. So much so people have to retire before upgrading.
Kids show up straight out of college and have to learn 1970s technologies or software. Like they could eliminate an entire wing of the building with an Audrino and 3 weeks programing but aren't allowed
Specs are treated like the Bible. If you don't know where it came from and often you don't you aren't allowed to change it. These were often just picked from a long list of suitable solutions or numbers in the 70s because the loudest person wanted it. Now risk adverse management and regulatiots won't change.
Implementing anything substantial requires you to convince your manager who then has to convince their manager on up to like 4 levels. The 4th level came from energy sales and has no respect for technical input or technical decision making. Once your 4th level management is approved they have to personally take the case to the regulator at some risk. The government regulator then takes 2 years.
Spacex apparently has few of these limitations. I don't think any amount of lower level process improvement will help until the whole system is overhauled.
Asimov was right, nuclear power will become a religion. And nuclear engineers the priests.
Jesus. Reading that I could actually feel the weight of that kind of bureaucratic inertia.
o generalize, the important difference between old space and new space is that new space companies are run by founders who stand to do very well only if their company is successful, and old space companies are run by seasoned executives who get paid a lot of money regardless.
That sets up very different incentives between the two kinds of companies. In new space, employees get rewarded by doing things that push the company towards the overall space-related goals. In old space, employees are typically rewarded by fitting into the existing company structure and conforming with the company culture.
Being an innovator is a great thing at a new space company and it can get you fired from an old space company.
I talk about this more in this video, where I talk about Blue Origin and Rocket Lab.
This isn't unique to the space world; this is why so many of the software innovations come from small startups rather than the tech giants.
Being an innovator is a great thing at a new space company and it can get you fired from an old space company.
I dunno about space but my experience in software was the exact opposite. I worked for a startup that was three years old and like a bright eyed fool I assumed that everybody was looking for ways to improve our core product. However a lot of people had carved out their own little fiefdoms in the codebase and it turned out to be a cardinal sin to try to bypass any one of the fiefs. The company was only three years old and perfectly demonstrated Conway's law, clearly deliminated with the four different programming languages that were being used. After they fired me as a personal project I started rebuilding everything on my own as a way to recover my sanity and from the rough draft skeleton I built out in a few weeks I could see that it was possible to replace a bloated product that required about 12 GB of memory to run with something performing faster on only 50 MB of memory.
Jaded and more cynical I moved on to the opposite of a startup, an established firm with huge legacy systems. And there I found that as long as I didn't break things, people were perfectly happy to pay me to experiment and re-imagine. It wasn't a panacea, all the knowledge silos and technical debt you would expect were present. But they weren't angry at me for trying to untangle the Gordian Knots, they were quite happy with me for trying. They didn't have the enormous sticks up their rear ends that the people at the startups did and it was okay to say that things weren't done perfectly the first time and needed to be redone based on the lessons learned.
Eric Berger wrote nicely about this in his book - "Musk is able to instantly make a competent, informed decision on any design direction in the company."
“Liftoff” really is an insightful book highlighting the DNA of SpaceX. I got the impression that having the Money guy and the Engineering guy happened to be the same person made a huge difference.
You don't have Elon Musk as CEO to keep the company culture alive. That is the primary problem with old school aerospace.
The companies started and managed by Jack Northrop, Donald Douglas, Bill Boeing et. al. did amazing things when the founders were in charge. When the founders passed on and the board of directors took over, the culture died as well.
Is it though? NASA JPL does plenty of incredible things and there is no shining CEO. Lots of other really solid space projects that push the envelope, I think it has more to do with process and culture rather than CEOs, a culture of excellence can exist without these guys I think. You could have an incredible CEO take over Lockheed but their institutional culture is going to be very difficult to steer in a different direction at this point. I wonder if their initial success is more a function of their size and risk profile as opposed to their CEO. I’m not saying Elon isn’t a huge contributor to the success of SpaceX, but I think if he was gone SpaceX would continue to do pretty great work, but who knows.
Jack Northrop, Donald Douglas, Bill Boeing
I'm surprised Northrop wasn't named Nack
I think you're skimming the surface, but if it were that easy, more companies would be successfully emulating it.
The real aspects required for success are deeper, and therefore harder to implement in existing companies. I think there's an argument to be made that old space may never be able to successfully emulate SpaceX because it requires everyone to be bought into specific cultural principals from the start. If they aren't, it's nearly impossible to root-out old cultural holdouts.
None of this is new, but I think it really comes down to things like:
Ownership at the individual level
Empowering those closest to the work to drive change
Ability to differentiate between "good enough" and "perfect" and willingness to accept the former
Acceptance of failure while maintaining accountability
Clear understanding of risk posture on each project throughout the organization
Use of project constraints to drive innovation - budget and schedule are knobs you turn to ride the line between "too scrappy" and "good enough"
Proper motive - there's a big difference between the decisions a company whose motive is to innovate towards a higher-level goal will make versus the decisions a company whose goal is to simply stay in business or turn profit.
Not a SpaceX engineer…
Set a goal. Every decision and meeting should be centered around if it accelerate or slows down the process to meet that goal. Everything else is just window dressing.
-wasted time in meeting means less time working which slows everything down
-waiting on suppliers slows down procurement. So in source and go vertical
only outsource for expertise never for cost savings. It’s never as cheap as it sounds, and you loose control (read speed)
-software is the cheapest big purchase you can make. Dicking around looking for a solid works license is a waste of so much time it hurts.
the finance guys exist to find a way to fund stuff. Engineers should never have to justify to a non-engineer why they need X. And cost should be a minor part of the conversation.
speed towards the goal is the only measure of success. Not weight savings, not man hours expended, not revenue forecasts. Just progress towards the goal.
Now pick a goal that inspires engineers to want to work on the thing and let them go nuts.
This 100%. You absolutely nailed it.
only outsource for expertise never for cost savings. It’s never as cheap as it sounds, and you loose control (read speed)
It amazes me how much the company I work at ("old space") outsources basic machining tasks. We have all the equipment in our shop, but we outsource so much of it - it's nuts!
software is the cheapest big purchase you can make. Dicking around looking for a solid works license is a waste of so much time it hurts.
A good IT department is critical, but so is using a superior CAD package: NX and TeamCenter.
Watch this interview with NASA scientist Dan Rasky, where he talks about his experiences at SpaceX.
If your company isn't willing to work at something for a while, realize it isn't going to work that way and then throw the entire concept away and go in an entirely new direction, regardless of time and money spent, this development style isn't for you.
If your company isn't willing to blow things up, regularly.. this development style isn't for you.
If your company is beholden to bean counters for every design decision.. this development style isn't for you.
In all honesty, I think the reason SpaceX seems to outclass damn near everyone is this:
The sunk cost fallacy doesn't exist.
Elon is hands on at every major development decision and generally understands what he is being told.
The only guiding principle is goal accomplishment. It doesn't matter how a system is "supposed" to work or how things have been done in the past. Can you show him the math on a better way? They'll try it.
Get better leaders.
Great leaders don't just motivate you, hell they might never even pat you on the back. But they give you the authority, responsibility, and resources you need to excel, and most importantly, they focus. They eliminate un-necessary work, they remove questionable requirements, and they streamline goals and objectives to only what's absolutely needed to succeed.
I am just listening to Liftoff by Eric Berger. He reports that Elon attributed much of SpaceX's speed to the fact that Elon was both chief engineer and CFO, so there were no bean counters in the approval loop.
I'm not a SpaceX Engineer, or any kind of Engineer, but my advice to "old space" is "just try to make a reusable rocket, or any kind of innovation".
Old space is just stuck with the idea of building whatever they get told to build, without concern about reusability, or cost, or even if it ever flies.
I'm sure ULA, Ariane, Roscosmos and even BO have some brilliant engineers who can figure out the solutions to the "hard, hard, hard" problems -- but their talents are being wasted. There needs to be a complete change in the top-level management who either don't think reusability is important, or who just dont care as long as they get paid.
I'm not an insider (in SpaceX in particular or space in general) but I design and run complex computer-driven systems and have thirty years' experience doing that, so I find this sort of thing to be fascinating.
Two things I think I've picked up from the outside:
They use COTS, but they're not trusting anymore. After they lost CRS-7 on 28 June 2015 because a COTS strut failed, they've taken to doing a lot more validation of the manufacturer's claims.
I understand that they have an incredible in-house software system they share with Tesla. It really helps them track where every piece in every vehicle came from, so that they can know trends and recognize problems in sourcing early.
Yeah I think it's healthy to be skeptical of COTS. It should be used but it should be thoroughly tested to standards as well.
I'm an engineer at SpaceX. I'd recommend you read through this blog post at StackOverflow - it's an interview with an engineer in Software Delivery Engineering talking about how we develop flight software, and she goes into a lot of detail on process and off-the-shelf tech usage.
I'm in the same department, but Erin understands what we do way better than I do, and explains it better than I ever could. This is part of a whole series of blog posts on SO that go into software at SpaceX, but I think what we do is really underappreciated at a lot of places; everybody's interested in the actual software that goes on vehicles, but the software that helps write that software is, in many ways, a more complex problem.
Great contribution thank you! This is exactly the kind of rabbit hole I was hoping to find.
I noticed TDD being mentioned, is that something done for most SpaceX projects or more specifically for human rated systems? I’m also curious how far y’all go with unit testing and what your strategy is. In my experience with human vehicles, there were formal design, code, and unit test reviews with 100% coverage required then other V&V activities afterwards of course. Now I work on more R&D sort of work, where we’re flying technology demonstration work without humans on board. Our CI, MR, and unit test process is pretty open ended right now, and I’m trying to find the right balance of testing and dev ops practices without taking too many resources from development and iteration. This is part of why I’m trying to sus out what processes produce large returns on investment, as there’s no way we can be as rigorous as previous projects.
I get the feeling that having the unit test focus being on lower rates of coverage, but stress testing more at a functional level, would be a more useful for us. I guess I’m mentioning this because I’m curious what your opinion is on the return on investment for various types of software delivery and testing practices are. You’ve got a perspective from a pretty different world that could help us a lot.
This thread is that greatest thing I've ever seen in Reddit.
Adding to that:
"Everyone is the lead engineer". That means everyone is allowed to know the high level vision and is empowered to question things, design changes, etc.
Beware of the design of your thing reflecting structure of your organization (company, lab, agency, etc.). Shuttle with its multiple but not redundant hydrazine, hydrogen, oxygen and whatever loops is a prime example of that ailment. In-space propulsion and APU groups were separate so they designed their systems completely separately wasting mass and at the same time missing redundancy opportunities.
if it takes 8 hours to run a sim
In brief, get a better simulator.
By education, I'm a mathematician, but in a very checkered career, I did a fair amount of simulation, so I have a clue about what a simulator can be expected to do. Two or maybe three orders of magnitude in one-tenth to one-hundredth of real time is a really high-end product. These people are getting six orders of magnitude in real time, and cheaply enough that you can put one on every engineer's desk. To get one on my desk back when I was doing a lot of simulations, I probably wouldn't have killed anybody, but I would have committed a lot of illegal and immoral acts.
n5321 | 2025年7月7日 23:46