fxra
Posts: 600
Joined: Fri Jul 16, 1999 1:03 am

RE: ORD And MDW Ground Stop Due To ATC Fire

Sun Sep 28, 2014 6:10 pm

Quoting jetwet1 (Reply 96):
Did he cut the only feeds coming out of the radar system ? I would have to think a modern computer system would not find the job of collecting radar info and presenting it in the correct manner a hard task, with the right network you could then run that airspace from anywhere, or am I missing something ?

Problem 1: if you take the crazy guy out of the equation, it's still a bad policy to keep your back up server sitting right beside the primary. They really should have them off site (just the servers, maybe at another ATC facility nearby like C90), but at the very least in a separated room. The FAA requires the air carriers to have a much higher lever of redundancy, I don't think keeping the servers separate and hardlined would be unrealistic.

Second, running a sector at another facility is more a person problem then a hardware problem. Controllers train for months to learn one sector. I read an article a few months ago about a C90 (I think, of KZAU) controller who said they're only allowed to be qualified on a limited number of sectors (like 3 or 4). You can't just take a KZID controller and assign him KZAU airspace he's unfamiliar with on a days notice. Hence, the previous comment about asking the controlled to go to other centers to work.

We've had good weather this weekend, imagine a severe weather event on top of this snarling the open airspace. Fingers crossed the weather luck holds.... at least through when i work tomorrow.
Visualize Whirled Peas
 
cpl22586
Posts: 63
Joined: Thu Jun 28, 2012 1:39 am

RE: ORD And MDW Ground Stop Due To ATC Fire

Sun Sep 28, 2014 6:36 pm

The latest update from yesterday evening from the FAA.


"Press Release – FAA Statement-10:45 PM Update on Chicago Air Traffic Facility

For Immediate Release

September 27, 2014
Contact: Tony Molinaro/Elizabeth Cory
Phone: (847) 294-7427/(847) 294-7849


--------------------------------------------------------------------------------

The Federal Aviation Administration continued to increase the air traffic flow into and out of the Chicago-area airports today, following a criminal incident and fire yesterday at a high-altitude radar facility in Aurora, IL. Air traffic controllers at other FAA facilities safely managed about 60 percent of the flights as compared to last Saturday at O'Hare and over 75 percent of the flights at Midway. The FAA expects a substantial increase in operational capabilities by Monday.

The FAA is working closely with the airlines to manage the traffic flow. The agency also has been able to steadily increase air traffic and reduce delays by improving direct communication between the FAA facilities that are now managing air traffic in the Chicago area, and by developing new methods for automatically filing and transferring airline flight plan information. Air traffic controllers initially had to file flight plans manually after yesterday's fire compromised some communications systems. The FAA is using all other tools at its disposal to reduce disruptions as much as possible.

After inspecting the damaged equipment at the Chicago En Route Center, the FAA has decided to completely replace the central communications network in a different part of the same building, to restore the system as quickly as possible. The FAA is assembling the new components at a remote site, and they will begin to arrive at the center tomorrow. Technicians will work on the installation around the clock, to set up the new equipment, connect it to several undamaged systems, and complete testing. Cleaning crews are also continuing to work around the clock to ensure a safe work environment.

The FAA is developing a timeline for transferring service back to Chicago Center, but in the meantime is working to safely accommodate flights in and out of the Chicago area through its other facilities, significantly reducing impacts on travelers. "

https://www.faa.gov/news/press_releases/news_story.cfm?newsId=17235

Also from talking to one of my friends in Indy Center they took up to the EON VOR.

[Edited 2014-09-28 11:39:43]
 
freakyrat
Posts: 1809
Joined: Fri Aug 22, 2008 1:04 pm

RE: ORD And MDW Ground Stop Due To ATC Fire

Sun Sep 28, 2014 9:03 pm

NATCA Press Release just minutes ago. I know my former home town controllers at SBN are doing a tremendous job.

http://www.natca.org/news.aspx?zone=Top%20News&nID=8617#n8617
 
cpl22586
Posts: 63
Joined: Thu Jun 28, 2012 1:39 am

RE: ORD And MDW Ground Stop Due To ATC Fire

Mon Sep 29, 2014 3:26 am

Quoting freakyrat (Reply 102):
NATCA Press Release just minutes ago. I know my former home town controllers at SBN are doing a tremendous job.http://www.natca.org/news.aspx?zone=Top%20News&nID=8617#n8617

Really a great article. Like all the details of how they handled the traffic.

Latest FAA release from early today regarding the restorations.

https://www.faa.gov/news/press_releases/news_story.cfm?newsId=17254
 
Marcus
Posts: 1665
Joined: Fri Apr 20, 2001 5:08 am

RE: ORD And MDW Ground Stop Due To ATC Fire

Mon Sep 29, 2014 3:27 pm

Flew from DFW to ORD last Friday evening and the flight was delayed for a bit over 3 hours, considering by that time some people next to me had been in the airport for over 12 hours I was not going to complain!; the original plane was supposed to be a 737 but this was changed to a very old 767 with old overhead bins and CRT screens in the aisles as IFE, needless to say it was a PACKED flight.

What caught my attention was befure departure the captain mentioned our cruising altitute would be 37K feet and we would drop to 10K feet near Chicago, and that this would add some time to scheduled flight time; following the in flight map I saw that once we flew from MO to IL we descended to 25K feet and stayed there for a few minutes, then to 20K and so and son until we reached 10K feet in the middle of the state of IL. Does anyone know why did we have to fly the last 30 minutes of the flight at 10K feet and then do the final descent to the airport?
Kids!....we are going to the happiest place on earth...TIJUANA! signed: Krusty the Clown
 
ckfred
Posts: 5178
Joined: Wed Apr 25, 2001 12:50 pm

RE: ORD And MDW Ground Stop Due To ATC Fire

Mon Sep 29, 2014 3:28 pm

According to the last press release from the FAA, Chicago Center should be back to normal by October 13, two weeks from tomorrow.

Now, for some observations. It seems to me that security isn't an issue. It's not like someone just wondered into the Aurora facility. The suspect was someone who has pretty much access to the entire facility. If the nature of his job required him to have access to the entire facility, then it seems to me that problem isn't security, as much as it is that contractor employees need to be better vetted for working in such sensitive locations.

I know the FAA does psychological screening for people who enter the controller training program. I know both pilots and F/As who fly or used to fly for AA, and there is a psychological screening program as part of the application and training process. So, if Harris Corp. and other FAA contractors aren't required to screen their employees working on site the way the FAA and the airlines do, that is a serious mistake on the FAA's part.

In the post 9/11 world, most companies in the financial sector have back-up systems and facilities located some distance form their primary computer systems and facilities. A lot of the brokerage houses and banks moved their primary facilitiese out of Manhatten to locations throughout New Jersey, and a number of back-up systems are located some distance from the Tri-State area. It's also common for have IT employees scattered throughout the country, simply because of severe weather hits one area, it won't lead to all employees being stuck home, or the office being flooded, without power, or snowbound.

Now, it probably doesn't make sense to scatter controllers who work in one center or Tracon, but back-up systems shouldn't be in the same building, and certainly not in the same room as the primary systems.
 
Goldenshield
Posts: 5019
Joined: Sun Jan 14, 2001 3:45 pm

RE: ORD And MDW Ground Stop Due To ATC Fire

Mon Sep 29, 2014 3:49 pm

Quoting Marcus (Reply 104):
Does anyone know why did we have to fly the last 30 minutes of the flight at 10K feet and then do the final descent to the airport?

Discussed earlier.

Quoting Goldenshield (Reply 24):
They are shifting to Tower Enroute Control for ORD/MDW. It's not very nice, but at least it gets planes moving.
Quoting airstatdfw (Reply 27):
The TRACONs will work the aircraft with in their airspace to the next TRACON. You have to fly at a low level to stay inside the TRACON airspace. if going west stay in Chicago TRACON airspace until you are in Rockford airspace and they could hand you off to ZMP center, or north to MKE to be handed off to ZMP center, etc.
Two all beef patties, special sauce, lettuce, cheese, pickles, onions on a sesame seed bun.
 
User avatar
northwestEWR
Posts: 1974
Joined: Sat Oct 09, 2004 12:45 pm

RE: ORD And MDW Ground Stop Due To ATC Fire

Tue Sep 30, 2014 5:58 am

Can anyone comment on how things are running passenger-wise at ORD/MDW? Are things running somewhat normally? Are the masses of passengers that ended up stranded gone?
Northwest Airlines - Now You're Flying Smart
 
N766UA
Posts: 8270
Joined: Thu Jul 29, 1999 3:50 am

RE: ORD And MDW Ground Stop Due To ATC Fire

Tue Sep 30, 2014 12:47 pm

Quoting jetwet1 (Reply 96):
or am I missing something ?

You're missing something.

Quoting ckfred (Reply 105):
as much as it is that contractor employees need to be better vetted for working in such sensitive locations.

I highly doubt any pre-employment screening would have picked this guy out for a potential firebug. You just can't cover 100% of all eventualities! Unfortunately, the nature of the system and the facilities is that this kind of thing is possible.

Looks like the ZAU guys will be camping out in adjacent centers for a couple weeks, anyway. Things should get better and better every day, then hopefully they can go home in a couple weeks!
 
N766UA
Posts: 8270
Joined: Thu Jul 29, 1999 3:50 am

RE: ORD And MDW Ground Stop Due To ATC Fire

Tue Sep 30, 2014 1:19 pm

Quoting ckfred (Reply 105):
Now, it probably doesn't make sense to scatter controllers who work in one center or Tracon, but back-up systems shouldn't be in the same building, and certainly not in the same room as the primary systems.

Rabble Rabble Rabble Rabble!

All things considered, I think this whole thing went pretty well, and 2 weeks is a new york minute for government work...
 
trnswrld
Posts: 1370
Joined: Sat May 22, 1999 2:19 am

RE: ORD And MDW Ground Stop Due To ATC Fire

Tue Sep 30, 2014 1:25 pm

Latest update says all the new equipment has arrived at ZAU. Harris Corporation says everything should be back up and running by the 13th. Obviously its still a bit early to tell if thats going to actually happen or not, but things are moving along.
As stated above, Chicago Center employees have been scattered to adjacent centers as well as underlying approach controls, and ORD tower. The only ZAU controllers that will be working traffic are at their own sectors that have been set up at the adjacent centers. Otherwise, ZAU controllers that have been relocated to approach controls and towers have been passing flight plan information. Thats helpful because if a controller had to do manual handoffs at the sector while working traffic things would be overwhelming. Manually passing flight plans takes time and during that process controllers are "on the line" which takes away from communicating with pilots.
As for the time being they have been able to get enough controllers to volunteer to go to these facilities and no one has been forced to go yet.
Sounds like all considering things are working pretty well.
 
ckfred
Posts: 5178
Joined: Wed Apr 25, 2001 12:50 pm

RE: ORD And MDW Ground Stop Due To ATC Fire

Tue Sep 30, 2014 4:35 pm

Quoting N766UA (Reply 108):
I highly doubt any pre-employment screening would have picked this guy out for a potential firebug. You just can't cover 100% of all eventualities! Unfortunately, the nature of the system and the facilities is that this kind of thing is possible.

True, but clearly, but according to the guy's Facebook page, he had come to the conclusion that government employees were lazy people who weren't capable of learning anything. It seems to me that someone who thinks he's smarter than everyone else and gets incredible annoyed by those who won't work as hard is someone who shouldn't be in a sensitive area such as an air traffic control center. Are there methods for determining whether someone will get so annoyed with co-workers that he will try and damage his workplace and/or cause harm to himself and others? That's a question for psychologists to answer.

Quoting N766UA (Reply 109):
Rabble Rabble Rabble Rabble!

All things considered, I think this whole thing went pretty well, and 2 weeks is a new york minute for government work...

You don't work in IT, do you? My wife does. It doesn't matter whether its a government agency, a large bank/brokerage house, or any other large business. You keep primary and back-up systems/hardware separated, whether it's in separate rooms, buildings, states, or even countries.

We are talking about Illinois. What if a tornado had come through and leveled the building? The FAA wouldn't just be trying to replace a lot of hardware while cleaning drying out other systems. It would be trying to find some long-term, albeit interim solutions while building a new facility.
 
trnswrld
Posts: 1370
Joined: Sat May 22, 1999 2:19 am

RE: ORD And MDW Ground Stop Due To ATC Fire

Tue Sep 30, 2014 4:55 pm

People keep mentioning this guy being upset with government employees or saying they are lazy. Where are you guys seeing this? I read all the news reports and even the entire report about this guy and only thing I see being mentioned about facebook is his post he made that morning saying he is sorry to his family and hopes they get over him soon, and he is gonna smoke a blunt and take out ZAU".

So what am I missing?!?
 
User avatar
tjwgrr
Posts: 2474
Joined: Fri Mar 24, 2000 4:09 am

RE: ORD And MDW Ground Stop Due To ATC Fire

Fri Oct 03, 2014 10:35 am

This is an interesting article from Bloomberg regarding the on-going incident at ZAU / Chicago Center. What I found disturbing is how carefully planned out this attack was- not only to cripple the primary system, but also the back-up systems. This was not a case of just someone suddenly going off at work.

Contact Lost With Planes One by One as FAA Fire Spread - Bloomberg:

http://www.bloomberg.com/news/2014-1...one-as-faa-center-fire-spread.html
Direct KNOBS, maintain 2700' until established on the localizer, cleared ILS runway 26 left approach.
 
N766UA
Posts: 8270
Joined: Thu Jul 29, 1999 3:50 am

RE: ORD And MDW Ground Stop Due To ATC Fire

Fri Oct 03, 2014 11:18 am

Quoting ckfred (Reply 111):
You don't work in IT, do you? My wife does.

Hmm, well anyway, for those who work in and/or actually understand air traffic, again I say things went very well, considering, and excellent work. There are a LOT of people who stepped up during this crisis, and they should be appreciated.



[Edited 2014-10-03 04:57:08]
 
NBGSkyGod
Posts: 865
Joined: Sun May 30, 2004 7:30 am

RE: ORD And MDW Ground Stop Due To ATC Fire

Fri Oct 03, 2014 11:24 am

Quoting ckfred (Reply 111):
You don't work in IT, do you? My wife does. It doesn't matter whether its a government agency, a large bank/brokerage house, or any other large business. You keep primary and back-up systems/hardware separated, whether it's in separate rooms, buildings, states, or even countries.

The reason for the co-located back-up and primary systems are due to the need to ensure that both are functioning properly if the need to switch occurs, and the switch must be seamless. A bank can be okay with a few minutes/hours of downtime between switching, ATC cannot. If the the primary goes away the tech can be working on the primary then, if needed, also provide support for the standby without having to drive across town to fix that too (there are enough components that require that as it is).

Quoting ckfred (Reply 111):
We are talking about Illinois. What if a tornado had come through and leveled the building? The FAA wouldn't just be trying to replace a lot of hardware while cleaning drying out other systems. It would be trying to find some long-term, albeit interim solutions while building a new facility.

There are other contingencies for a center going ATC-Zero for long periods of time, as we have seen. This is not some grab-ass "out-of-the-box" thinking, with other centers covering, it is a documented plan. Granted it is not practiced and so there was some lead time to get it up and running, and it is not the best way to handle things, but it is working. As for a total destruction of the facility, there are plans for that too.
Pilots are idiots, who at any given moment will attempt to kill themselves or others.
 
N766UA
Posts: 8270
Joined: Thu Jul 29, 1999 3:50 am

RE: ORD And MDW Ground Stop Due To ATC Fire

Fri Oct 03, 2014 11:54 am

Quoting NBGSkyGod (Reply 115):
This is not some grab-ass "out-of-the-box" thinking, with other centers covering, it is a documented plan.

Good luck trying to get people to understand that.

[Edited 2014-10-03 05:29:44]
 
Goldenshield
Posts: 5019
Joined: Sun Jan 14, 2001 3:45 pm

RE: ORD And MDW Ground Stop Due To ATC Fire

Fri Oct 03, 2014 5:36 pm

Quoting NBGSkyGod (Reply 115):
his is not some grab-ass "out-of-the-box" thinking, with other centers covering, it is a documented plan.

The running of flights via Tower Enroute until they could get the high altitude sectors up and running again was "out-of-the-box" thinking.
Two all beef patties, special sauce, lettuce, cheese, pickles, onions on a sesame seed bun.
 
pedaltodaflo
Posts: 3
Joined: Mon Oct 06, 2014 6:09 pm

RE: ORD And MDW Ground Stop Due To ATC Fire

Mon Oct 06, 2014 6:12 pm

So the culprit was a Harris employee. Does anyone know if Harris' FAA contract would be at risk here or if they have any liability? Thanks
 
bond007
Posts: 4428
Joined: Mon Mar 14, 2005 2:07 am

RE: ORD And MDW Ground Stop Due To ATC Fire

Tue Oct 07, 2014 2:21 pm

Quoting NBGSkyGod (Reply 115):
The reason for the co-located back-up and primary systems are due to the need to ensure that both are functioning properly if the need to switch occurs, and the switch must be seamless. A bank can be okay with a few minutes/hours of downtime between switching, ATC cannot. If the the primary goes away the tech can be working on the primary then, if needed, also provide support for the standby without having to drive across town to fix that too (there are enough components that require that as it is).

Failover systems, with backup servers at opposite ends of the USA are a 'norm' for many Fortune 500 companies. Arguing that a salary and availability of a local support tech is the reason for having them in the same location is a very poor one - you get more support techs if you need to. That's how companies work. I was recently working with a Fortune 50 company which did a test failover onto another set of servers mid-morning, and it was seamless to all users.

Most large companies, especially manufacturing, cannot have much system down time. The last manufacturing company I worked with measured their backlog in 1 day, for each hour the system was unavailable. It appears their disaster revovery plans are much more robust than the ATC systems ...although I'm not entirely surprised.

Jimbo
I'd rather be on the ground wishing I was in the air, than in the air wishing I was on the ground!

Popular Searches On Airliners.net

Top Photos of Last:   24 Hours  •  48 Hours  •  7 Days  •  30 Days  •  180 Days  •  365 Days  •  All Time

Military Aircraft Every type from fighters to helicopters from air forces around the globe

Classic Airliners Props and jets from the good old days

Flight Decks Views from inside the cockpit

Aircraft Cabins Passenger cabin shots showing seat arrangements as well as cargo aircraft interior

Cargo Aircraft Pictures of great freighter aircraft

Government Aircraft Aircraft flying government officials

Helicopters Our large helicopter section. Both military and civil versions

Blimps / Airships Everything from the Goodyear blimp to the Zeppelin

Night Photos Beautiful shots taken while the sun is below the horizon

Accidents Accident, incident and crash related photos

Air to Air Photos taken by airborne photographers of airborne aircraft

Special Paint Schemes Aircraft painted in beautiful and original liveries

Airport Overviews Airport overviews from the air or ground

Tails and Winglets Tail and Winglet closeups with beautiful airline logos