twiga
Topic Author
Posts: 96
Joined: Wed Mar 13, 2013 11:24 pm

### Computing A/C Safety Metrics - Comb. Redundancies

How do you compute airplane safety metrics at the overall airplane level by combining the various levels or layers of redundancies. It is my understanding from the B 787 battery issues that Boeing designed the battery for the 1:10^7 (1:10 million hrs) event and this is for the entire chain of events leading to catastrophic failure at the airplane level. Questions:

1) Is 1:10^7 hrs standard across the industry (Boeing and Airbus) for all major items that have layers of redundancies that can end up as catastrophic, for example two engines out on two engine aircraft?
2) What is definition of catastrophic? Just loss of airplane or airplane and lives? Does FAA or regulators have anything to say about what metric to use?
3) For loss of two engines how do they determine the reliability for one engine and its metric for failure in flight (assuming its properly maintained to manufacturers standard).
4) With three independent hydraulic systems how do they combine these and come up with 1:10^7 if this is the right number?
5) Landing gear would have layers of redundancy for deployment but not if it failed from an exessively hard landing. In the latter I suppose its possible to get a metric for average and hard g-force landings and apply the appropriate factor of safety to say the worse 10%.

Anyway here are a few questions and I apologize if this has been covered before.

roseflyer
Posts: 9605
Joined: Fri Feb 13, 2004 9:34 am

### RE: Computing A/C Safety Metrics - Comb. Redundancies

Far 25.1309 defines almost everything you are looking at. Here’s the link to it: http://rgl.faa.gov/Regulatory_and_Gu...025.1309-1A/\$FILE/AC25.1309-1A.pdf

1.) Is 1:10^7 hrs standard across the industry (Boeing and Airbus) for all major items that have layers of redundancies that can end up as catastrophic, for example two engines out on two engine aircraft?

10^7 is the standard for hazardous events. Catastrophic events are 10^-9, Major are 10^-5 and Minor are 10^-3.

2) What is definition of catastrophic? Just loss of airplane or airplane and lives? Does FAA or regulators have anything to say about what metric to use?

In ability to continue safe flight and landing without exceptional pilot skill.

3) For loss of two engines how do they determine the reliability for one engine and its metric for failure in flight (assuming its properly maintained to manufacturers standard).

Engines are separate and it is based on the ETOPS program. I don’t remember the numbers off hand, but you can find them in the FARs.

4) With three independent hydraulic systems how do they combine these and come up with 1:10^7 if this is the right number?

Fault tree analysis are used for multiple failures. Reliability of the system is one factor. Number of systems and exposure time to latent failure are other factors.

5) Landing gear would have layers of redundancy for deployment but not if it failed from an exessively hard landing. In the latter I suppose its possible to get a metric for average and hard g-force landings and apply the appropriate factor of safety to say the worse 10%.

Landing gear has a redundant system for deployment, but no backup system for retraction. Any excessive hard landing must be evaluated before next flight. There are inspection requirements in the AMM.
If you have never designed an airplane part before, let the real designers do the work!

roseflyer
Posts: 9605
Joined: Fri Feb 13, 2004 9:34 am

### RE: Computing A/C Safety Metrics - Comb. Redundancies

I am very knowledgeable on the subject so let me know if you have any other questions. Almost everything is in the federal air regulation that I quoted so it applies to all airplanes certified by the FAA. I'm general there are 4 different criteria and different levels of redundancy and maintenance requirements are based on the effect. The majority of catastrophic failure modes are various flight control failures, but there are some spread across a variety of areas. In the certification the manufacture has to demonstrate compliance for every system to the reliability rates in the FARs. However there are always some failure conditions that were never thought of (fuel tank flammability after TWA 800) for example so things revolve over time.
If you have never designed an airplane part before, let the real designers do the work!

twiga
Topic Author
Posts: 96
Joined: Wed Mar 13, 2013 11:24 pm

### RE: Computing A/C Safety Metrics - Comb. Redundancies

 Quoting Roseflyer (Reply 2):I am very knowledgeable on the subject so let me know if you have any other questions. Almost everything is in the federal air regulation that I quoted so it applies to all airplanes certified by the FAA.

Thanks alot you have provided a lot of info to digest - so will do that first and then might have to come back for clarification/ interpretation. But one quick question (and don't answer if there are any conflicts for you) because this came up a number of times on another thread when discussing Boeings battery fix. In your opinion for their final fix thats combined metric for (1) bat mods and (2) containment vessel would they be looking at 1:10^7 - hazardous, or 1:10^9 - catostraphic. On the other thread many had 1:10^7 as catostraphic. And I think this is assuming an unenclosed runnaway battery could potentially bring the airplane down - hence the grounding.

roseflyer
Posts: 9605
Joined: Fri Feb 13, 2004 9:34 am

### RE: Computing A/C Safety Metrics - Comb. Redundancies

I am not going to comment on the battery for the 787. Either hazardous or catastrophic failure could result in an airplane being out of compliance with the FAR. You can look at 25.1309 and subsequent revisions and amendments. The link I have quoted does not have the advisory circular which outlines hazardous since it didn't use to be one of the top level event criteria.

I can't comment on the 787 battery because that is not my specialty. There are authorized representatives working on behalf of the regulators that make those determinations. All I will say is that the probability Is for whatever the top level event is and fault trees can be rather intricate.
If you have never designed an airplane part before, let the real designers do the work!

roseflyer
Posts: 9605
Joined: Fri Feb 13, 2004 9:34 am

### RE: Computing A/C Safety Metrics - Comb. Redundancies

To help you out more, here is AC 1309 which is the latest harmonized standard and details of hazardous conditions in it.

http://www.faa.gov/documentLibrary/m...isory_Circular/AC%2023.1309-1E.pdf

[Edited 2013-04-30 21:05:27]
If you have never designed an airplane part before, let the real designers do the work!

twiga
Topic Author
Posts: 96
Joined: Wed Mar 13, 2013 11:24 pm

### RE: Computing A/C Safety Metrics - Comb. Redundancies

 Quoting Roseflyer (Reply 4):I am not going to comment on the battery for the 787. Either hazardous or catastrophic failure could result in an airplane being out of compliance with the FAR.

Yes - I can see why. There is alot of room for interpretation in AC 25.1309-1A, and it could go in several directions. The experts probably had quite a bit of back and forth over the last few months. Thanks for the latest AC 1309 will review tomorrow.

[Edited 2013-04-30 22:14:35]

roseflyer
Posts: 9605
Joined: Fri Feb 13, 2004 9:34 am

### RE: Computing A/C Safety Metrics - Comb. Redundancies

 Quoting twiga (Reply 6):Yes - I can see why. There is alot of room for interpretation in AC 25.1309-1A, and it could go in several directions. The experts probably had quite a bit of back and forth over the last few months. Thanks for the latest AC 1309 will review tomorrow.

It takes a lot of expertise to apply the FAR and AC to airplane design. That’s why the certification process is so complex and robust. It takes very intelligent and experienced engineers to do this analysis. When I read in the news or comments that Boeing should have known what they are doing or they were trying to trick/deceive or blast the manufacturers for delays because the executives should have “known better”, I just smile at the ignorance. The process is so complex and robust that safety is better than ever. The consequence is an extremely lengthy development process, and any one component not performing as expected or failing it’s qualification program can delay an entire program 6 months.
If you have never designed an airplane part before, let the real designers do the work!

cornutt
Posts: 333
Joined: Sun Jan 20, 2013 6:57 am

### RE: Computing A/C Safety Metrics - Comb. Redundancies

 Quoting twiga (Reply 6):There is alot of room for interpretation in AC 25.1309-1A, and it could go in several directions.

As you've probably noticed, that AC is really old now; it was issued in 1988. The whole field of hazard analysis has made a lot of progress since then, and I don't understand why the FAA doesn't issue an updated version. Roseflyer can comment more on this, but a lot of the time we rely on AC 23.1309 for better definitions of terms, even though that AC applies mostly to GA aircraft.

roseflyer
Posts: 9605
Joined: Fri Feb 13, 2004 9:34 am

### RE: Computing A/C Safety Metrics - Comb. Redundancies

The AC was intended to harmonize the EASA and FAA requirements which is why it was created. AC23.1309 provides additional guidance and the manufacturers have their own internal processes depending on the level of delegated authority.
If you have never designed an airplane part before, let the real designers do the work!

twiga
Topic Author
Posts: 96
Joined: Wed Mar 13, 2013 11:24 pm

### RE: Computing A/C Safety Metrics - Comb. Redundancies

 Quoting cornutt (Reply 8):As you've probably noticed, that AC is really old now; it was issued in 1988. The whole field of hazard analysis has made a lot of progress since then, and I don't understand why the FAA doesn't issue an updated version. Roseflyer can comment more on this, but a lot of the time we rely on AC 23.1309 for better definitions of terms, even though that AC applies mostly to GA aircraft.
 Quoting Roseflyer (Reply 9):The AC was intended to harmonize the EASA and FAA requirements which is why it was created. AC23.1309 provides additional guidance and the manufacturers have their own internal processes depending on the level of delegated authority.

Thanks cornutt. Roseflyer kindly provided links to both AC 25.1309-1A and to the new one AC 23.1309-1E. I went through AC 25.1309 and made notes and now I've been through AC 23.1309. Had to make a lot of revisions to my notes to try and marry the two. Its a real mish mash because they have overlapping definitions and there are inconsistancies in wording. And its hard to get your head around some of it and "what is current today". I think their main problem occured when they decided to squeeze in "hazardous" between major and catastrophic. I agree they should start from scratch and clean it up and get the Europeans on board. Yes these are just brief guides for the manufacturers they probably have developed 3 inch thick manuals to work from - I can see this stuff getting very complicated and time consuming.

What I'm planning on doing is making an attempt at condensing the failure conditions for (2)- major, (3)-hazardous, and (4)-catostrophic, in a one page brief summary "laymens" version. And anyone who wants details can deal with the 19 page and 56 page documents and good luck.

roseflyer
Posts: 9605
Joined: Fri Feb 13, 2004 9:34 am

### RE: Computing A/C Safety Metrics - Comb. Redundancies

 Quoting twiga (Reply 10): Yes these are just brief guides for the manufacturers they probably have developed 3 inch thick manuals to work from - I can see this stuff getting very complicated and time consuming.

Each system safety assessment that contains all the fault trees and functional hazard assessment for any given system tends to be more than 3 inches and there are a hundred assessments done.
If you have never designed an airplane part before, let the real designers do the work!

twiga
Topic Author
Posts: 96
Joined: Wed Mar 13, 2013 11:24 pm

### RE: Computing A/C Safety Metrics - Comb. Redundancies

This has been brought over from FAA Approves Boeing 787 Battery System Changes because it fits with safety metrics.

 Quoting zeke (Reply 133): From a pure maths point of view, the probability of a specific event is actually zero. The reason being the aircraft as a system is a continuous random variable with an infinite number of outcomes, thus it does not have discrete countable outcomes. We measure (i.e. continuous distribution) rather than count (i.e. discrete distribution) aircraft in service events and reliability. We can come up with a probability for a range of measurements on a continuous distribution, however not a singular value as we have an infinite number of outcomes. In a similar fashion I have seen people "abuse" stats when talking about engine reliability, engines are also a continuous distribution, decreasing/increasing the number of engines on an aircraft does not actually decrease/increase the rate of engine failures for an airframe. We never know when an engine will fail, as a system it has an infinite number of outcomes. If we did know when one would fail, it would be changed before hand and thus the discrete event never realised.

This also does not make much sense. If you had "four engines" on an airplane versus "one engine" on an airplane the probability of "any one of the 4 engines" failing is 4 times greater than it is for the one engine on the single engine airplane. Your last sentence - When discussing statistics you are never dealing with a single event. You are dealing with metrics derived from sufficient data points to do normal distribution, frequency analysis, probability distribution etc. The experts that develop the metrics for engine reliability likely have 3 inch thick binders on methodology, statistical modeling and have the data flowing in from a number of sources - airplanes in service, continious 24/7/365 testing data from RR, GE, etc.

Quote:"To put this in the battery context, having two batteries in the aircraft does not double the chance of failure, as both batteries may go their entire service life with failing. When a battery will fail is unknown."

This does not make any sense. Two batteries versus one battery doubles the probability that any "one of the two batteries" could fail before the single battery. Look at it this way, if you took a healthy person and you took 1,000 healthy people what would be the probability of that single person dying before "any one of " at random of the 1,000 people more or less? It would be less by 1/1,000. What would be the probability of that single person dying before a particular "one of " the 1,000 people? It would be even odds. Isn't that how insurance companies make money?

Quote:"Pure maths seems illogical at times, however this containment fix is actually mathematically sound. It should not factor into ETOPS at all".

You are right 'pure math' is too theoretical and doesn't appear to have any practical application here other than to cloud and fog the issues at hand. Since when do mathematicians design pressure vessels or give advice on something they know nothing about? The design involves engineers that are familiar, with thermodynamics (mechanical and chemical engineers) material engineering and structural engineering.

The purpose for posting this in the first place was to gain some small understanding in how these metrics for safety might work. I qualified myself that I'm not an expert in this field and was seeking help in furthering this along.

I must admit I didn't expect a long qualitative circular discussion with undefined phraseology, that neither I or 99.0% of the posters could understand. It also suprised me that someone with a math and supposed statistics background stayed away from numbers and the quantitative, which is after all the language of these sciences. If you are capable and are still interested please comment specifically on the issue and methodology described for the metrics of two engines out.

The question is how do you compute safety at the overall airplane level of 1: 10^7 (1:10 million) from combining the various levels or layers of redundancies? Sometimes things can be better clarified if you hang numbers on them, however there is always the danger of being wrong (I was never good at statistics), but others that know better can always jump in and correct.

Now lets assume (a) is the first level/ layer of redundancy and (b) is the second level of redundancy and the overall combination safety metric at the airplane level is (c) for catastrophic failure**. So ( a) x (b) = (c). We always know (c) = 1:10^7 (1:10 million hrs) and we always know either one of (a) or (b). Note**: Since origionally posting according to AC 23.1309-1E - (3) Hazardous failure condition is 1:10^7 hrs (1:10 million) and (4) Catastrophic failure condition is 1:10^9 hrs (1: 1 billion) not sure which applies to two engines out but will use (3) for discussion purposes and for simplicity and methodology.
_____________________________________________________________________________________________
Two engines out - assume (a) = (b) for reliability of engines
--(a)---one engine out------------------------------1: 3200 hrs (approx) (by deduction)
--(b)---two engines out-----------------------------1: 3200 hrs (approx) (by deduction)
--(c)---Safety metric overall airplane level-----1: 10^7 hrs (1:10 million) (known) (assume Hazardous failure condition)
_____________________________________________________________________________________________
Assume the the number of hrs to failure metric for one engine is derived statistically by all means available and in this case has to be greater than 3,200 hrs to meet overall safety metric requirement at airplane level of 1: 10^7 hrs
This assumes engines are regularly inspected and maintained as per specifications.
____________________________________________________________________________________________
Definition of Hazardous Failure Conditions
- Extremely remote failure conditions. Those failures not anticipated to occur to each airplane during its total life, but may occur a few times when considering the total operational life of all airplanes of this type. (Note: This could easily involve over 3,000 aircraft over a period of 30 years)
- Failure conditions that would reduce the capability of the airplane or the ability of the crew to cope with adverse operating conditions to the extent there would be the following (a)-large reductions of safety margines of functional capabilities. (b)-physical stress and higher work load such that the flight crew can not be relied upon to perform their tasks accurately or completely or (c)-serious or fatal injury to an occupant other than the flight crew.

zeke
Posts: 11322
Joined: Thu Dec 14, 2006 1:42 pm

### RE: Computing A/C Safety Metrics - Comb. Redundancies

 Quoting twiga (Reply 12): If you had "four engines" on an airplane versus "one engine" on an airplane the probability of "any one of the 4 engines" failing is 4 times greater than it is for the one engine on the single engine airplane.

I did mention this pure maths stuff does not make sense. To start getting your brain thinking differently, let me ask you a question. Do you replace the tires on your car only after a blowout, or do you do it when you notice the wear is excessive ? Have you ever had a car that the entire time you had it it never had a blowout ?

I am sure you are familiar with dice, if you roll a normal die on solid surface, you have an inbuilt bias knowing that one of the facets will make contact with the surface, and one will be at the top. There is only 6 possible outcomes. Roll the same die on a very uneven surface like carpet, and it may or may not end up on an edge. The bias that there will be at least a singular outcome is no longer true, the probability of rolling a 1 is not longer 1/6 it is 1/infinity. If you did experiment of 10,000 rolls, you would come up with a distribution of how many times a 1 would come up for that type of carpet. You could then measure, i.e. 1500/10000 probability of getting a 1. Do the same for 2, and you may end up with 4000/10000. On a flat surface the probability of rolling a 1 and a 2 in any order is 1/12, on the carpet it is 3/50.

You have an inbuilt bias in your thinking, that bias is that an engine will fail. The majority of modern turbine engines in service will go their entire service life without failing. The service life is something you also need to keep in the back of your mind, we retire parts from aircraft before they become a risk. If you take that population of engines that did not fail their entire service life, their probability of failure is zero (none failed). If you have 4 of those engines, 4 times zero is still zero, which is not 4 times greater. If you have one engine failure, and then have 3 engines form the pool of engines that does not fail, the probability of failure on the second engine is zero.

Aircraft are designed to stringent standards, have complex overlapping maintenance procedures, and have health monitoring. If we detect something is wrong, it is fixed before it ever before close to failure most of the time. if something does fail, we investigate it, and promulgate a fix to prevent it from subsequently causing a threat.

If we see signs an engine has a trend which is know towards failure, we do some maintenance, or pull it out of service. We stack the deck, so the probability of failure is towards the pool of engines that does not fail. We are getting better and better at stacking that deck, despite more aircraft flying, and flying more sectors/hours, we have fewer engine failures.

 Quoting twiga (Reply 12): You are dealing with metrics derived from sufficient data points to do normal distribution, frequency analysis, probability distribution etc. The experts that develop the metrics for engine reliability likely have 3 inch thick binders on methodology, statistical modeling and have the data flowing in from a number of sources - airplanes in service, continious 24/7/365 testing data from RR, GE, etc.

Actually the amount of data they have is enormous, I am sure just the trends derived from that data for various parameters would fill a USB stick. They use that data in the FADEC software so the engines do not fail. We download engine data for every flight, for every engine, and the aircraft as well. That data is stored in a database, and some pretty smart analysis is done on it.

 Quoting twiga (Reply 12):This does not make any sense. Two batteries versus one battery doubles the probability that any "one of the two batteries" could fail before the single battery.

As above, batteries should go their entire life without failing, if their monitored performance is off, or the maintenance schedule calls for it, they get pulled out of service and replaced before failing. We do not use parts to the point of failure.

 Quoting twiga (Reply 12): Since when do mathematicians design pressure vessels or give advice on something they know nothing about? The design involves engineers that are familiar, with thermodynamics (mechanical and chemical engineers) material engineering and structural engineering.

Engineers are applied mathematicians, you should see how much maths is involved in an engineering degree.

 Quoting twiga (Reply 12):The question is how do you compute safety at the overall airplane level of 1: 10^7 (1:10 million) from combining the various levels or layers of redundancies?

For a start, we do not assume an aircraft is certified in isolation, we draw on the last century of combined knowledge of aircraft design and accidents to develop the regulations the aircraft is certified, maintained, and operated. That is basically how an aircraft can be ETOPS certified out of the box.

Manufacturers and the regulator bring to the table its in service experience on similar aircraft.
Human rights lawyers are "ambulance chasers of the very worst kind.'" - Sky News

twiga
Topic Author
Posts: 96
Joined: Wed Mar 13, 2013 11:24 pm

### RE: Computing A/C Safety Metrics - Comb. Redundancies

 Quoting zeke (Reply 13):I did mention this pure maths stuff does not make sense.

Zeke thanks for coming back and responding in a sincere fashion. Will give a fuller response to your post later. I guess it was the pure math stuff that threw me off. Most engineers are more familiar with calculus and applied math - even though a very small percentage such as "the R and D types" will get invoved with pure math. The majority of us, at least I am, are comfortable with our feet firmly planted on the ground and lofty ideals cloud and fog our vision from our earthy pursuits. Having said this, I think we can get on the same page and have a good mature discussion on this subject. And from your background there are still a number of things I would like to quiz you on. I am relatively new on this forum and have a keen interest in learning more. My apologies for being a little blunt.

roseflyer
Posts: 9605
Joined: Fri Feb 13, 2004 9:34 am

### RE: Computing A/C Safety Metrics - Comb. Redundancies

 Quoting zeke (Reply 13):You have an inbuilt bias in your thinking, that bias is that an engine will fail. The majority of modern turbine engines in service will go their entire service life without failing.

I am really struggling to understand how your theoretical math applies to the reliability requirements in the FARs.

When doing fault tree analysis you do assume the part will fail. The analysis uses the reliability of a component mean time between failure. The number is either based on in service experience or the endurance qualification testing. A critical component is going to be tested to 4 times airplane life. It doesn't matter how long a specific part lasts on wing.

What you are talking about sounds more like the continuous monitoring that operators do as a part of their reliability program and ETOPS program. Original certification is based on predicted MTBF, exposure time, redundancy, and event frequency. The match between FAR25.1309 and an ETOPS program are very different.
If you have never designed an airplane part before, let the real designers do the work!

zeke
Posts: 11322
Joined: Thu Dec 14, 2006 1:42 pm

### RE: Computing A/C Safety Metrics - Comb. Redundancies

There are pre-certification design requirements, and post certification requirements. The 787 being certified, is now subject to the post certification requirements, not the design requirements. The post certification requirements are commonly referred to as continuing airworthiness, the aim is to maintain the aircraft airworthy in accordance with its type certificate. The pre-certification requirements you mention are all true, and they form part of the TCDS (under the certification basis heading), so does the continuing airworthiness (under the Certification Maintenance Requirements (CMRs), service information etc), they go hand in hand, one is like the absolute limit, the other is the roadmap established also pre-certification to ensure these absolute limits are never realized. They continue to evolve through amendments to the SRM, AMM, ADs, SBs etc.

While the 787 would have had to demonstrate that it met or exceeded the FARs in terms of those probabilities, or had an alternative means of compliance. The continuing maintenance procedures would be to make sure components are removed well before their MTBF, safe life etc. And the design is such that it is fail safe, and damage tolerant. The battery containment is an example of damage tolerant design, even if the battery were to fail, its failure is contained.

These is the excerpt from the 787 TCDS for continuing maintenance.

"To maintain compliance with Type Certification requirements of the 787 airplane, each operator must incorporate into their airline’s FAA-approved maintenance program the applicable items from the following FAA-approved documents (as cited in Section 9 of the 787 Maintenance Planning Data, Boeing Document D011Z009-03):

D011Z009-03-01, 787 Airworthiness Limitations (AWLs). Contains required structural inspections and the retirement times for structural safe-life and life-limited parts. Also contains required retirement times for systems life-limited parts
and other systems limitations.

D011Z009-03-02, 787 Airworthiness Limitations (AWLs) – Line Number Specific. Existing structures AWLs that were impacted by airplane production nonconformances may result in airplane specific revised inspection requirements and/or inspection intervals.

D011Z009-03-03, 787 Certification Maintenance Requirements (CMRs). Required periodic tasks to specific Systems installations.

D011Z009-03-04, 787 Special Compliance Items (SCIs) /Airworthiness Limitations. This document lists and provides instructions for Airworthiness Limitation Instructions (ALIs) and Critical Design Configuration Control Limitations
(CDCCLs) required to comply with 14 CFR Part 25.981"

These are the bare minimum requirements to maintain airworthiness, operators can elect to do more than this, an example would be perform an engine core wash.

The sort of process that is followed following an in service defect is outlined in this presentation

http://www.ntsb.gov/news/events/2011...ions/5.1%20FAA-Panel%205-Final.pdf
Human rights lawyers are "ambulance chasers of the very worst kind.'" - Sky News

roseflyer
Posts: 9605
Joined: Fri Feb 13, 2004 9:34 am

### RE: Computing A/C Safety Metrics - Comb. Redundancies

 Quoting zeke (Reply 16):There are pre-certification design requirements, and post certification requirements. The 787 being certified, is now subject to the post certification requirements, not the design requirements.

The battery modification must comply with a certification plan that demonstrates compliance to the original Type certification requirements. The existing 787s that have had their airworthiness certificate issued are covered under continued airworthiness instructions and a modification would be mandated by AD, but all new 787s that have not been delivered must comply with the original requirements dictated in FAR 25.1309 and are certified with certification plans that demonstrate compliance with all FARs as required under a new type design. CMRs, ADs, and AWLs will often be line number limited for those airplanes with modifications under the airworthiness certificate or via continued airworthiness instructions via service bulletin.
If you have never designed an airplane part before, let the real designers do the work!

twiga
Topic Author
Posts: 96
Joined: Wed Mar 13, 2013 11:24 pm

### RE: Computing A/C Safety Metrics - Comb. Redundancies

 Quoting zeke (Reply 13):Quoting twiga (Reply 12): If you had "four engines" on an airplane versus "one engine" on an airplane the probability of "any one of the 4 engines" failing is 4 times greater than it is for the one engine on the single engine airplane.
 Quoting zeke (Reply 13):You have an inbuilt bias in your thinking, that bias is that an engine will fail.

Where is the "bias"? Lets deal with reality - engines on an airframe at random on a fleet of aircraft, will and do fail even if they have been properly maintained to specifications, replacing parts as required up to including engine replacement. In fact, and this is pure speculation, because I don't know the number of trans-atlantic crossings per day, but I would guess that there would be at least one or two if not more single engine failures per month averaged out over the year for all aircraft types. This is reality - engines do fail, so if their is a bias it must have something to do with "pure math" or the wrong interpretation of some principle of pure math, where you end up with these famous "zeros". And lets be clear, nobody in the context of this discussion is that naive to be discussing the failure of a "specific engine" in a fleet of aircraft when studying the metrics in general.

Whether we are discussing and comparing the rate of failure probability question of "one engine versus 4 engines" or the failure of "one battery versus 2 batteries" is the same issue and is very relavent to these discussions. I don't see the relavency of your dice or carpet experiments other than complicating or clouding the simple issue which is relavent to our discussion. We simply don't need a myraid of other questions with slightly different nuances. Otherwise we are going to end up in a never-ending circuitous discussion like a dog chasing its tail, which of course it never catches. Having said that lets resolve this simple issue for once and for all. I will re-frame the "one versus four engine" question because it is analogous the "one versus two battery" question and it is also analogous to the " one versus 1,000 people" question. Their are two outcomes to this basic question and it depends on how the question is framed. Case (1) is relavent, and Case (2) is naive and irrelevent to these discussions.

Case (1) - If you compared the failure rates of a "single engine on single engined airplanes" with the failure rates of "any one of the 4 engines at random" on 4 engined airplanes, the probability of failure of the single engine would be 1/4 that of "any one of the 4 engines at random" on the 4 engined airplanes. Put another way the probability of failure of "any one of the 4 engines at random" is 4 times greater than that of the single engine.

Case (2) - If you compared the failure rates of a "single engine on single engined airplanes" with the failure rates of "one specific engine of the 4 engines " on 4 engined airplanes, the probability of failure of the single engine, would be even or the same as for "one specific engine of the 4 engines" on the 4 engined airplanes.

Similarily if you took a single healthy person and you took 1,000 healthy people what would be the probability of that single person dying before "ANY ONE OF " at random of the 1,000 people more or less? It would be less by 1/1,000. What would be the probability of that single person dying before a particular "ONE OF" the 1,000 people? It would be even odds. Please note the "nuance" in wording and I've added "at random" and "particular" to fu . And the former is how insurance companies make their money. So lets frame the question on the context of what we are trying to accomplish.

 Quoting zeke (Reply 13):Aircraft are designed to stringent standards, have complex overlapping maintenance procedures, and have health monitoring. If we detect something is wrong, it is fixed before it ever before close to failure most of the time. if something does fail, we investigate it, and promulgate a fix to prevent it from subsequently causing a threat. If we see signs an engine has a trend which is know towards failure, we do some maintenance, or pull it out of service. We stack the deck, so the probability of failure is towards the pool of engines that does not fail. We are getting better and better at stacking that deck, despite more aircraft flying, and flying more sectors/hours, we have fewer engine failures.

This makes alot more sense,

 Quoting zeke (Reply 13):Engineers are applied mathematicians, you should see how much maths is involved in an engineering degree.

You don't have to explain this to me I've been there. Enjoyed the Applied and Calculus but struggled with the "Pure" so this ruled out an R and D career.

 Quoting zeke (Reply 16):The battery containment is an example of damage tolerant design, even if the battery were to fail, its failure is contained.

Agree, because its an end of the line feature thats stand alone and is not dependent on layers of redundancies like it was part of a "system". Its no different than designing everyday things like buildings, bridges etc. The usual practice is to design with a FS (factor of safety) of 2. Apparently Boeing designed this containment vessel with a FS of 3. Its design is safer than any of the sky scrapers in New York City and the buildings you occupy 24/7/365, and then the containment vessel will only be called upon very infrequently we hope.

In post #12 table for two engines out I assumed a "Hazardous" failure condition at the airplane level which is 1:10^7 hrs (1:10 million) with the computed metric of approx. 1:3,200 hrs for one engine out. If I had used a "Catastrophic" failure condition at the airplane level of 1:10^9 hrs (1: 1 billion) the computed metric for one engine out would be approx. 1:32,000 hrs. (This number seems awfully high for the reliability of an engine - it would be equivalent to driving your car for at least 1.3 million miles) so for now I will stay with Hazardous and 1:3,200 hrs for one engine out. Based on this a typical commercial pilot with 10,000 hrs flying time on average could have experienced "one engine out" about 3 times. Obviously if you considered a single pilot this number could be misleading, but if you sampled 100 pilots there would be some mean statistic.

Questions
What is your experience with "one engine out"? What about other pilots that you know? Have you seen any metrics for "one engine out" for atlantic or pacific crossings - weekly, monthly or yearly? When you do/ did flight training in simulators did they just tell you, we are going to check you out for one engine out failure or did they give you any idea as to how often this might happen to you? Recognizing that the FAA - CA is only a guideline and manufacturers may construct to higher standards, it seems certain aircraft types might be more risky to fly than others, also it seems that new aircraft types roughly between 2 and 10 years old might be the safest to fly - reasons same as cars new models have quirks for the first couple of years and older models are less reliable. The question is do pilots get higher pay based on higher risk aircraft, and does the pilot association mine the data bases to establish the metrics for risk parameters to justify extra compensation for its pilots or are the pilots just happy to have a job? The reason I am asking is because other occupations get higher pay for taking risks. And please don't answer any of these questions if they might cause a conflict for you. Anyone else with information on these questions please jump in.

Finally zeke please burn that confounded Pure Math book of yours and replace it with something more useful like an Applied Math book.

roseflyer
Posts: 9605
Joined: Fri Feb 13, 2004 9:34 am

### RE: Computing A/C Safety Metrics - Comb. Redundancies

 Quoting twiga (Reply 18): Have you seen any metrics for "one engine out" for atlantic or pacific crossings - weekly, monthly or yearly?

ETOPS certification forces airlines to monitor their inflight shutdown rate. They calculate a rolling 12 month average for each fleet. The requirement is equivalent to an in-flight shutdown rate of one every 50,000 hours. Most of the requirements are in Flight Hours and not cycles. Also, most airlines are far better than that rate. Airlines with robust maintenance programs usually average one in every 200,000 – 800,000 flight hours. The reliability rates are so high that for most airlines a shutdown is more of an anomaly. However at the very large ETOPS operators like UA, DL and AA, you’ll see a shutdown every month or so across their fleets.

Some airlines without robust maintenance programs using older used fleets are likely to have much higher shutdown rates. Allegiant is an example of an airline that took a lot of work to get ETOPS approval on their 757 fleet. After a series of in-flight shutdowns, Air France lost its ETOPS 180 certification on its 777s and had it downgraded to 120 minutes in 2007. Flyglobespan lost their ETOPS certification on their entire 757 fleet in 2007 and never got it back. The 1 in every 50,000 flight hours does creep up on some airlines from time to time

[Edited 2013-05-06 08:20:17]
If you have never designed an airplane part before, let the real designers do the work!

twiga
Topic Author
Posts: 96
Joined: Wed Mar 13, 2013 11:24 pm

### RE: Computing A/C Safety Metrics - Comb. Redundancies

 Quoting twiga (Reply 18):In post #12 table for two engines out I assumed a "Hazardous" failure condition at the airplane level which is 1:10^7 hrs (1:10 million) with the computed metric of approx. 1:3,200 hrs for one engine out. If I had used a "Catastrophic" failure condition at the airplane level of 1:10^9 hrs (1: 1 billion) the computed metric for one engine out would be approx. 1:32,000 hrs. (This number seems awfully high for the reliability of an engine - it would be equivalent to driving your car for at least 1.3 million miles) so for now I will stay with Hazardous and 1:3,200 hrs for one engine out.
 Quoting Roseflyer (Reply 19):ETOPS certification forces airlines to monitor their inflight shutdown rate. They calculate a rolling 12 month average for each fleet. The requirement is equivalent to an in-flight shutdown rate of one every 50,000 hours. Most of the requirements are in Flight Hours and not cycles. Also, most airlines are far better than that rate. Airlines with robust maintenance programs usually average one in every 200,000 – 800,000 flight hours. The reliability rates are so high that for most airlines a shutdown is more of an anomaly. However at the very large ETOPS operators like UA, DL and AA, you’ll see a shutdown every month or so across their fleets.

Thanks for the info. To hear that airlines with robust maintenance programs have an engine in-flight shut down rate of one in every 200,000 - 800,000 flight hrs is incredible - little did I know! Well with this info. two engines out in flight is definitely in that 1: 10^9 hr (1 in 1 billion) Catastrophic failure condition. Not anticipated to occur during the entire operating life of all airplanes of one type. With expected loss of life and normally loss of airplane. You can certainly see this happening mid-atlantic or mid-pacific in choppy waters or over the artic. On the continental US there would likely be many airports within 1/2 to 1 hr for one engine landings or some possibility of RAT deployment landings for two engines out.

I think the 1:3,200 hrs would be more applicable to when I crossed the Atlantic in 1963 on a Lockheed Superconstalation with 4 rotary piston engines and purchasing a \$10 insurance policy (\$100 today) was the norm. I guess the relatively high accident rate of piston engined a/c including the Stratocruiser was always fresh in our minds. It appears modern turbine engines are at least 10 to 20 times safer and 5 times quieter.

I expect the phenominal flight hour record has a lot to do with the automated real time monitoring Rolls Royce and GE do. Does it ever get to the situation that they see something that would say require an engine shut down in flight before it could catch fire or disintegrate, and they forward a real time message to the flight crew to do a shut down. It is recognized the flight crew would have some on board warnings but the manufacturers would have more including better analytical procedures.

roseflyer
Posts: 9605
Joined: Fri Feb 13, 2004 9:34 am

### RE: Computing A/C Safety Metrics - Comb. Redundancies

 Quoting twiga (Reply 20):Does it ever get to the situation that they see something that would say require an engine shut down in flight before it could catch fire or disintegrate, and they forward a real time message to the flight crew to do a shut down. It is recognized the flight crew would have some on board warnings but the manufacturers would have more including better analytical procedures.

It is definitely possible to detect problems before a catastrophic engine failure. Vibrations, oil temp, exhaust gas temp, etc are all monitored. Rarely does an engine actually catch fire. They are usually shut down as a precaution before that happens.
If you have never designed an airplane part before, let the real designers do the work!

twiga
Topic Author
Posts: 96
Joined: Wed Mar 13, 2013 11:24 pm

### RE: Computing A/C Safety Metrics - Comb. Redundancies

 Quoting Roseflyer (Reply 19):ETOPS certification forces airlines to monitor their inflight shutdown rate. They calculate a rolling 12 month average for each fleet. The requirement is equivalent to an in-flight shutdown rate of one every 50,000 hours.

Couple of questions. Assuming in-flight two engines out is in the "Catastophic" failure category of 1:10^9 hrs (1:1 billion) and each one the two engines is considered equal in reliability performance to the other. That is engine (a) is equal to engine (b) then to meet 1:10^9 hrs, engine (a) and (b) = square root of 10^9 = approx 32,000 hrs. Why doesn't this match with the rate of 50,000 hrs? Does some other adjustment or multiplier come into play? I assume all new commercial aircraft are constructed to meet ETOPS-60 and therefore the 50,000 hrs would apply as a minimum and for higher ETOPS ratings, obviously regulation prescribed in-flight performance records would need to be proven. For ETOPS-180 would the minimum be 150,000 hrs (3x50,000) or much more complicated than this? Could you point me to which FAR to look at.

roseflyer
Posts: 9605
Joined: Fri Feb 13, 2004 9:34 am

### RE: Computing A/C Safety Metrics - Comb. Redundancies

I'm not an ETOPS expert. However I will point out that your calculation is missing a few pieces. First off, cascaded failures need to be addressed. ETOPS maintenance programs limit them, but engines are not entirely independent. Fuel starvation, contamination and leaks affect both.

Also, exposure time is a factor. How many hours is the airplane exposed to engine out operations. If it is one hour (ETOPS 60), then the multiplier is 1, but if it is 2 hours (ETOPS 120), you have to double your engine reliability rates to get the same top level event probability rate.

I don't know the FARs for ETOPS. Maybe someone else knows more about that.
If you have never designed an airplane part before, let the real designers do the work!

twiga
Topic Author
Posts: 96
Joined: Wed Mar 13, 2013 11:24 pm

### RE: Computing A/C Safety Metrics - Comb. Redundancies

 Quoting Roseflyer (Reply 23):I'm not an ETOPS expert. However I will point out that your calculation is missing a few pieces. First off, cascaded failures need to be addressed. ETOPS maintenance programs limit them, but engines are not entirely independent. Fuel starvation, contamination and leaks affect both.

Thanks. To clarify, I think what you are saying by cascading failures - is that one-engine-out, is not just based on the reliability of the engine itself, but is dependent on other systems such as fuel delivery, electrical supply etc. than could also cause engine out. And each system, for example fuel delivery, has several layers of redundancies such as say 3 pumps for fuel delivery and similarily the electrical system would have its layers of redundancies. If I understand things correctly if the reliability of the engine metric was 1:10^9 each of these engine dependent systems, fuel, electrical, etc would also need to have the same 1:10^9 metric based on the combination of their redundancies. Because if anyone of these systems failed, a reliable engine in of itself, would no longer meet the overall one-engine-out metric.

zeke
Posts: 11322
Joined: Thu Dec 14, 2006 1:42 pm

### RE: Computing A/C Safety Metrics - Comb. Redundancies

 Quoting twiga (Reply 18): Where is the "bias"? Lets deal with reality - engines on an airframe at random on a fleet of aircraft, will and do fail even if they have been properly maintained to specifications, replacing parts as required up to including engine replacement

The point of continuing airworthiness programs is that parts should be removed from service before we know they will fail. Any part may fail at any time at random, however the process we have in place should in an ideal world remove parts/aircraft/engines from service well before they are expected to fail.

Airlines these days tend to look at engine failures as being fairly improbable events, they are not expected to occur during the total operational life of a random single airplane of a particular type, but may occur during the total operational life of all airplanes of a particular type. There are rules and definitions as to what is and is not considered to be part of the engine, and what is and is not considered a failure.

 Quoting twiga (Reply 18):This is reality - engines do fail, so if their is a bias it must have something to do with "pure math" or the wrong interpretation of some principle of pure math, where you end up with these famous "zeros".

"The probability that a continuous random variable will assume a particular value is zero."

from http://stattrek.com/probability-distributions/discrete-continuous.aspx

The continuous distribution we use when looking at this sort of data is known as a Weibull distribution.

 Quoting twiga (Reply 18):Case (1) - If you compared the failure rates of a "single engine on single engined airplanes" with the failure rates of "any one of the 4 engines at random" on 4 engined airplanes, the probability of failure of the single engine would be 1/4 that of "any one of the 4 engines at random" on the 4 engined airplanes. Put another way the probability of failure of "any one of the 4 engines at random" is 4 times greater than that of the single engine. Case (2) - If you compared the failure rates of a "single engine on single engined airplanes" with the failure rates of "one specific engine of the 4 engines " on 4 engined airplanes, the probability of failure of the single engine, would be even or the same as for "one specific engine of the 4 engines" on the 4 engined airplanes.

This is abuse of the stats. Engine failure rates are measured across a fleet/family of engines, this measurements does not know what they are installed on. All we are interested in is the number of hours and cycles the engine fleet/family has done.

I consider the "bible" for this sort of work is a USAF document called the "Weibull Analysis Handbook", it helps us express different failure modes that appear in service. What we saw soon after the A380 EIS with the RR oil pipe issue was a manufacturing issue, this sort of problem, like the 787 battery would at this stage be considers "infant mortality" failure mode. We have other failure modes caused by wear, and totally random failures. Looking at the curve produced by plotting in service data, we can determine by the slope of the Weibull distribution the dominate mode of failure, and make the necessary changes to make failure improbable again.

If we had engines installed on a quad or twin, the failure rate of that type of engine is the same on both, say it 1x10^5. The probability however of a twin having a dual engine failure is less than quad, it would be 1x10^5x1x10^-5=1x10^-10. The risk of a dual engine failure on a quad however is equivalent to 3! (factorial) combinations of double engine failures, 6x10^-10.

However a dual engine failure on a twin is considered to have severe consequences, a quad, it is considered serious when deriving the hazard ratio for a failure mode. If you are interested have a look at the Continued Airworthiness Assessment Methodologies Advisory Circular, AC39-8.

The above information is for post certification.

Have a look at this http://www.soloy.com/files/Products/Documents/Dual%20Pac.pdf it will give you an idea of how it is done at the design stage.
Human rights lawyers are "ambulance chasers of the very worst kind.'" - Sky News

mrocktor
Posts: 1391
Joined: Mon Jan 24, 2005 12:57 am

### RE: Computing A/C Safety Metrics - Comb. Redundancies

 Quoting cornutt (Reply 8):The whole field of hazard analysis has made a lot of progress since then, and I don't understand why the FAA doesn't issue an updated version.

This is true, and this is why the whole industry uses the draft AC/AMC 25.1309 known as the "Arsenal" version (this is from 2002, so already old too). This is a harmonized version between FAA and EASA (at the time JAA) that for some reason never made it to official guidance material.

Note that the AC 23.1309-1E cited in this thread applies to general aviation and commuter aircraft, not to transport category aicraft.

twiga
Topic Author
Posts: 96
Joined: Wed Mar 13, 2013 11:24 pm

### RE: Computing A/C Safety Metrics - Comb. Redundancies

 Quoting zeke (Reply 25):Quoting twiga (Reply 18):Case (1) - If you compared the failure rates of a "single engine on single engined airplanes" with the failure rates of "any one of the 4 engines at random" on 4 engined airplanes, the probability of failure of the single engine would be 1/4 that of "any one of the 4 engines at random" on the 4 engined airplanes. Put another way the probability of failure of "any one of the 4 engines at random" is 4 times greater than that of the single engine. Case (2) - If you compared the failure rates of a "single engine on single engined airplanes" with the failure rates of "one specific engine of the 4 engines " on 4 engined airplanes, the probability of failure of the single engine, would be even or the same as for "one specific engine of the 4 engines" on the 4 engined airplanes. This is abuse of the stats. Engine failure rates are measured across a fleet/family of engines, this measurements does not know what they are installed on. All we are interested in is the number of hours and cycles the engine fleet/family has done.

You are quite right - because in the two cases, which I was trying to keep simple, I inadvertently used the plural "single engined airplanes" and "on the 4 engined airplanes" this was not my intent, I will reword my intent below as Case (1a) and Case (2a). You are right where I used the plural because this implies failures in the engine fleet and measurements across the fleet are required. And sophisticated methods for analysis as you mentioned.

Case (1a) - If you compared the failure rate of a "single engine on a single engined airplane" with the failure rate of "any one of the 4 engines at random" on a 4 engined airplane, the probability of failure of the single engine would be 1/4 that of "any one of the 4 engines at random" on the 4 engined airplane. Put another way the probability of failure of "any one of the 4 engines at random" is 4 times greater than that of the single engine.

Case (2a) - If you compared the failure rate of a "single engine on a single engined airplane" with the failure rate of "one specific engine of the 4 engines " on a 4 engined airplane, the probability of failure of the single engine, would be even or the same as for "one specific engine of the 4 engines" on the 4 engined airplane.

 Quoting zeke (Reply 25):If we had engines installed on a quad or twin, the failure rate of that type of engine is the same on both, say it 1x10^5. The probability however of a twin having a dual engine failure is less than quad, it would be 1x10^5x1x10^-5=1x10^-10. The risk of a dual engine failure on a quad however is equivalent to 3! (factorial) combinations of double engine failures, 6x10^-10.

Agreed. I think we are on the same page now. Thanks for the reference info.

[Edited 2013-05-10 22:24:29]

### Who is online

Users browsing this forum: teachpdx, trav110 and 1 guest

### Popular Searches On Airliners.net

Top Photos of Last:   24 Hours  •  48 Hours  •  7 Days  •  30 Days  •  180 Days  •  365 Days  •  All Time

Military Aircraft Every type from fighters to helicopters from air forces around the globe

Classic Airliners Props and jets from the good old days

Flight Decks Views from inside the cockpit

Aircraft Cabins Passenger cabin shots showing seat arrangements as well as cargo aircraft interior

Cargo Aircraft Pictures of great freighter aircraft

Government Aircraft Aircraft flying government officials

Helicopters Our large helicopter section. Both military and civil versions

Blimps / Airships Everything from the Goodyear blimp to the Zeppelin

Night Photos Beautiful shots taken while the sun is below the horizon

Accidents Accident, incident and crash related photos

Air to Air Photos taken by airborne photographers of airborne aircraft

Special Paint Schemes Aircraft painted in beautiful and original liveries

Airport Overviews Airport overviews from the air or ground

Tails and Winglets Tail and Winglet closeups with beautiful airline logos