As a formal Amtrak employee, I can add to this.
First, Amtrak gets bad press. When people think Amtrak, they think train wrecks. Why? Because the only, and I mean ONLY, thing you ever hear about Amtrak nationwide is when there is an accident. You never hear about increased ridership, or new service, or faster service.
As for the severity of derailments, that's a different story. Granted, there have been some issues that are the fault of the railroads in general, like when trains rear-end each other. But quite a few through the years have been caused by motorist or vandals leaving equipment on the tracks or vandalizing the tracks.
Another problem is that almost all of the Nations rail system is private. Amtrak has contracts with freight railroads to get the "right of way" on freight tracks. Yes, Amtrak IS supposed to get the right of way, but freight lines will put their intermodal service ahead of Amtrak 90% of the time. The only reason Amtrak is allowed on their tracks to begin with is because Amtrak is subsidized by the government. Otherwise, Amtrak really can cut into the freight railroad's bottom line.
So far, there are only a few areas where passenger service is really being pushed. California, Washginton State, North Carolina, Virginia, and the Northeast are all making their efforts. And as soon as the Charlotte-Greensboro-Raleigh-Richmond-DC corridor is finished and upgraded for high speed, you'll see a great increase in rail traffic.
But as for freight railroads restarting passenger service, it'll never happen. Like I said, passenger service has to be priced fairly low to compete with flying, especially with the time you lose traveling by train. It's just a big money looser for the freights, which is why the states and federal gvts picked it up. Also, why would the freight litter their tracks with passenger trains when they could run more profitable trains?
The passenger rail system in this country is realyl degrading, but depending on how the southeast handles their part, considering that is the only realy area expanding service, you may see some improvements in it.
Acela is a great advancement for Amtrak, but the NE tracks need to be upgraded for higher speeds than it can go. And the rest of the country has a 79 mph speed limit because of the curviness of the tracks.