Keynote: Teaching risk to rocket scientists at NASA, it’s a blast!

Dr Jeevan Perera’s title may be senior engineer, and he is a rocket scientist, but he headed up the risk programme for several key NASA manned space programmes and his job for the past two decades has been managing the risk of putting people and large hunks of very expensive kit up into space, safely, and getting them back safely too. To do that, he needs the best risk management available.

It’s easier to teach risk management to rocket scientists than it is to teach rocket science to risk managers. That’s the core principal Perera has brought to the organisation.

NASA is facing dramatic changes, but it still needs to be the best. What are the technologies and processes needed to manage manned space missions when the budgets are not as infinite as space?

How does Perera manage restructured missions and partnerships with commercial space launch companies? What are the risk processes and tools used. What risk mitigation and communication happens when programmes are underway and stakeholders are multiple. What are the lessons learned from space age risk mitigation that others can take back to earth?

TXF: You've been head of risk for NASA for quite a while now. It sounds like a cool job. How did you get it in the first instance?

Jeevan Perera (JP): I've been with NASA since the late 1980s and managed risks for many manned programme since around 2000. NASA is a matrix organisation and sits within the government. And back then we had the space shuttle and space station and some other manned activities. All of the manned spaceflight is managed here in Houston at the Johnson Space Center. Even though it's all NASA, I was actually in a line of organisation and the engineering groups supporting those big programme.

But then they pooled their expertise in different science or engineering or other disciplines. I spent the first 10 years working in in the engineering directorate. My friends told me to look for a transfer into one of the program offices, which were smaller and leaner in terms of management structure, but took on the responsibility to manage key manned programs.

I talked to the space station programme office and looked at the different groups within it. And the one position that sounded interesting was a risk manager. I can't say that I knew a lot about it then, but it did sound challenging.

Within a week of being named as the International Space Station (ISS) risk manager, I had a meeting with the programme manager. One of his first words were that he had tried risk-management before and really hadn't seen the return on the investment, but it was required that he have a risk process.

TXF: So no great enthusiasm for risk management from your former bosses decades ago?

JP: The programme manager said: ‘All I want you to do is just manage it the way the guy before you managed it.’ Which was very minimally and just on paper. And so without the background or the expertise to argue with him, I did that for several months.

But I thought it would be a good opportunity to really dive into the details of how other people did risk management, so that at least I would be more knowledgeable than I was walking through the door. And fortunately or unfortunately, about six months into it, this was early in the ISS program (which started in 1998), they were having significant cost overruns and schedule delays.

Since it's a government programme, there was a lot of oversight from Congress and then also NASA headquarters into the programme management of the ISS. They had different review teams that came in to figure out why it was costing so much and why there were schedule delays. One of the areas that they highlighted for improvement was the fact that risk management was not really being followed and was only paper-process.

The programme manager then supported rolling out a more effective approach to risk management along with other management practice changes. I took that early time of understanding risk and risk management practices. And then I spent about a month travelling through the country looking at large government and corporate programs to understand what their best practices were, what affected and improved their risk processes.

I realised early on that, like anything, it had to fit in the processes, culture and organisational structure of NASA. I spent time talking from the ISS programme manager down to understand what they would find beneficial. How would a process improve the way that they managed their job functions. I followed from internal and external and then rolled out a phased approach that had the core elements in it. As people got more comfortable with the process I added more and more layers and then figured out what worked and what didn't and made adjustments as we put a more robust system together. That was my start 20 years ago.

And that risk management subsequently was rolled out to some of the other manned programme and to the Space Shuttle before it was retired.

TXF: What do you think the difference nowadays is between risk at NASA and risk for any other corporates based on your analysis?

JP: The core processes and techniques are very similar to any domain that I've seen, whether it's large engineering development or government versus the corporate side. There is some uniqueness, especially in the assessment tools from the different domains but most of the elements to identify, analyse and manage the risks are very similar.

TXF: What are you working on now and at what point does risk management get involved in the process?

JP: Besides the ISS, I was the risk manager for other programmes. About five years ago, I decided I wanted to get back into the engineering development that focused on a risk reduction area for exploration missions.

The human body has adverse effects from microgravity or zero gravity: muscle loss, bone density loss, other physiological and even psychological effects for a long duration in a microgravity environment. I've been working on engineering systems that focus on the human body to mitigate those risks, mainly in the area of exercise, because of that lack of gravity on the body. There's extensive time that's spent by crew members on maintaining that aerobic and strength conditioning, which also has some psychological benefits from isolation. I've been focused on that human physiology side of it.

TXF: You’ve revamped the whole of the agency's risk management processes over the past years. How has it changed now?

JP: There was quite a bit of change in the processes and techniques early on as everyone got comfortable. One of the key tenets was that everyone within their organisation was responsible for risk management. It's not an independent task that’s done by an independent risk team. We’ve tried to ensure that both the identification and the actual risk management are done by the subject matter experts.

We spent a lot of time developing the techniques and the actual processes, the mechanics, to make it simple and digestible enough. If it was overly complex, it couldn’t be implemented by everyone without a lot of training. That focus on the adoption and culture change was a lot of time that we spent early on to ensure that it was adopted not just from the top down, but throughout the organisation.

We also wanted to have a risk focal, or mini-risk manager, resident within each group – say in propulsion or avionics or any of the subsystem areas. We thought it would be easier to teach risk management, for instance, to a rocket scientist than teach rocket science to the risk practitioner.

For example, we take that rocket scientist assigned as a risk focal and we teach them in more depth than the rest of the organisation on the elements of effective risk management. That way, when issues come up, the team can leverage someone that they know and trust and have worked with over the years for assistance either in the identification analysis or the actual management of the risk.

TXF: When we spoke seven years ago, you were talking about ownership of risk. Is that the same sort of terminology you’d use now, and is there any sort of liability for that risk to those risk owners if something does go wrong? 

JP: Risk focals provide support to all the risk owners in their organisation. Risk owners are assigned based on who has the best knowledge, insight and leverage to ensure proper mitigation effects. That individual is ultimately responsible for bringing in all the stakeholders that are affected by the risk or can help effectuate mitigation. If a person identified the risk, they are usually assigned as the initial risk owner.  Risk owner maybe changed as we understand and characterise the risk better and determine all stakeholders.

TXF: Can that generate perverse incentives to not notice something, if it becomes something that has to be dealt with?

JP: It does bring up an interesting issue and we’ve had to work hard to have the right incentives and disincentives for proper behaviour related to risk. With those carrots and sticks, people try to game the system either to avoid work or give extra recognition or whatever the case may be. A good portion of the early work was to look at it from a game theory perspective to try to get the best behaviour out of that organisation.

TXF: Are incentives financial or more general? 

JP: It could be financial, it could be awards or indirect ones. If you have an individual who identifies a risk early and deals with it, they know that it would be less resource intensive than having to deal with a bigger problem later.

TXF:  What it comes down to is not having a checkbox approach to risk?

JP: Yes, that's the one thing that we've tried to avoid because, it would generally be just a very minimal amount of risk management if everyone was just doing it to check the box.

TXF: You don't take any credit market risks, but there's still a commercial element, particularly, as you mentioned, when budgets are tight. What are the challenges on that side?

JP: Yes, since we are a government agency, we don't have any credit or market risk but on the financial end, obviously, we use tools to look at the budgets and the actual costs, the planned costs versus actual costs to figure out where there is deviation and potential problems so attention can be made to those areas that have critical impacts. When we get allocated assets from Congress it’s very rare that we can go back and ask for more.

Most of the work is actually done by contractors and standard government contractor mechanisms apply for most of the work – whether that be Boeing or Lockheed or some other major defence or aerospace contractor. There’s a contractual side to make sure that the right contract mechanism is there and that they have the skills and the reputation to be able to complete the work.

TXF: Do you have anything to do with that sort of on the risk side in terms of the contractors?

JP: In the procurement phase of a project, before it's actually developed, there's a current strategy of offering out what we want done and the timeline requirements.

We try to ensure that a risk-based approach is there in the procurement strategy and also that we leverage, or that we assign, the requirement that the contractor have to follow an integrated management risk process so that it's not just NASA doing the risk management. We want to ensure that there's an integrated risk management process.

Basically, when it comes to risk there is no NASA or contractor system, it's one system.

TXF: Now that NASA is collaborating more with the private sector, how does that impinge on your approach to risk management?

JP: The traditional approach was that NASA would say ‘we need a space shuttle built and these are the requirements. This is a rough budget and this is a rough schedule.’ And then they would get bids. And once awarded, we would own the vehicle, pay all the costs upfront.

There’s been a commercial shift over the past 10 plus years with SpaceX and others. Since the Space Shuttle retired the approach was to try to leverage contractor services for our transportation needs. So instead of NASA saying, ‘we want to build a space shuttle’, NASA said, ‘we want access to space for our astronauts.’

And it was up to the private sector to see how they did that on their own without us funding all of their development or owning the vehicle. The SpaceX Dragon capsule was used for ferrying supplies to the space station and back. And that's not an vehicle we own or one we designed or helped design. All we did through this program is to say to SpaceX that if we selected them as one of our vendors, they are going to be more like a taxi service. They own the vehicle, they’re the driver. We're just going to say when we need that service. But it’s not like Uber or Lyft with plenty of other potential customers to drive profitability and access to space right now is still limited!

We also had to provide some development money, because if they did it all on their own, it probably wouldn't have been viable commercially. It probably would not be feasible to develop a system that has very limited needs and very limited opportunities to recoup their investment.

For the cargo or the transportation of, logistics up and down, we were selected three companies, SpaceX, Sierra Nevada which has a Dream Chaser vehicle that can carry cargo up and down. And then the third one is Orbital Sciences’ Cygnus. And so all three were funded with seed money and then guaranteed so many flights over the next 10 years to allow them to develop and operate. But again, we don't own the vehicles. All we tell them as and when we need their services.

They have freedom to market their services to other commercial interests. It's not that they only have to provide it to us. Hopefully with the opportunity to market and sell those services to other organisations, even to foreign governments, we could share the costs.

On the crew side, there were only two that NASA selected to the operational phase and that is SpaceX and Boeing. The strategy was that the commercial side will be much better or more efficient, more reliable and able to operate transportation systems [than the government], like the commercialisation of aircraft and commercial services in the 1920s and 1930s.

TXF: Coming back to your risk reduction project for space exploration missions, what sort of results are you having?

We’ve focused on a lot of human testing on the ground and tests with simulated missions and actual mission data to figure out what countermeasures are the most effective at reducing human risk impacts to our crews. We’ve, for the most part, just been in lower earth orbit which is 250 miles above earth and we’ve had Moon missions, but the data is somewhat limited. If we are talking about going into Mars, travel time would be back and forth about two years just for a 30 day stay on Mars, whereas our lunar mission was just about a week, more than a week. So there's lots of research and improvements that we need to do to ensure that humans safely get out there and can work once they get there and safely return. A lot of testing is needed at different levels to really understand what the human risks are and how best to mitigate them.

TXF: When we last spoke you were in a 1950s federal building. Is it more space age now?

JP: NASA was formed 1958. And there are still several buildings from that era. Now that I came back into the engineering group, I have a lab where we develop a lot of these prototype systems for testing and evaluation. And I found it really rewarding to try to get a lot of interns, university entrance, engineering entrance into the lab. We're competing against companies like Google or Apple for students. We've tried to establish that same creative environment at least in our lab. Generally interiors still look like what they might have looked in the 1950s and 1960s though.

TXF: What are you most excited about going forward?

JP: We shifted over to this paradigm on the commercial side for access, taking the taxi/rental car model versus owning the vehicle design and owning and operating like we have done in the past. The main reason has been to drive cost down and improve reliability and safety, and we’ve focused more of our attention on exploration. That means going back to the moon maybe in about five to 10 years and permanently setting up habitation there and then trying out systems to be able to mount a Mars mission, which is probably 10 plus years from now.

Originally I thought it would be in our national governmental program, like the ISS, which includes 16 different nations and a more rapid development. But on a purely private side companies like SpaceX or Blue Origin, and other different commercial ventures that are purely private to get access to space for humans. They're also looking at sending humans and equipment to the lunar surface again purely on a commercial side, and SpaceX has been talking about a spacecraft that contains humans with a view to colonising Mars in the next 40-50 years. Over time we’ll get there but it may be more of a private/government cooperation than a purely government driven program to send humans to Mars for the first time.

There have been setbacks and delays. The crew capability is still not completed for either SpaceX or for Boeing which are the systems NASA will use to get our astronauts into orbit. Currently, we are relying on the Russian Soyuz to get our crew up and down to ISS ever since we retired the shuttle in 2011. Originally the plan was to have the government-owned Orion taking over operations by about 2013.

We were supposed to have operational capability probably at least two years ago. The first crewed flights for the proposed SpaceX system and the Boeing system should occur early 2020 and start to be fully operational for another year or two. That is a long delay between when the US had capability to enter into orbit and probably 10 plus years of no access for humans for US astronauts using US assets. It hasn’t gone as smoothly as expected but in the long run the private sector should improve reliability, efficiency and cost. Overall, I’m optimistic.

TXF: If you had a single lesson to pass on to corporates that your experiences as head of risk for NASA gives you, what would it be?

JP: Leverage the organisation as a whole to get risk management effectively embedded within it. And part of that is to ensure and develop the right organisational culture. And it's a very difficult task to be able to do that. The focus has to be on leveraging your human capital to get our risk management implemented effectively. A lot of the root causes of risk are human-centred failures and process failures which create a huge risk, more than the technical side.

TXF: And we shouldn't be worried about aliens?

JP: Aliens are the least of our problems!

The comments/views are solely those of the author and do not necessarily represent NASA’s positions, strategies, plans or opinions. 

Meet Dr Perera at TXF Americas in Miami on 19-20 February where he is a keynote speaker.