"Isnt there more stress on the frame/body of the Concorde because of the higher speeds?"
On all details of any plane down to the smallest rivet the stress is calculated, measured and tested. A Concorde has a higher cabin pressure difference, therefore thicker fuselage skin. It has higher take-off and landing speed, therefore stronger landing gear, brakes, wheels, tyres etc. It is heated to high temperatures which are acounted for in different ways. Some component may erode faster, but then they are renewed faster.
All this has nothing with aging to do.
If I remember well, then the Concorde basic airframe structure has been tested to 40,000 hours and 10,000 cycles. At that time the test rig was dismanteled for economic reasons, also because it was anticipated that no Concorde would ever fly that much. No weak points were found. These figures give the Concorde some 25-30 years more at the present utilisation. But I don't think that any Concorde will ever get that old - over fifty years.
The design fatigue life span was somewhat longer - don't remember the exact walue, but the 40,000 hours wasn't even close to that value, maybe it was 60,000 or 80,000. But in any case every single Concorde must, according to the type approval - retire at the latest when it reaches the values of the test rig when its operation was discontinued - which by the way happened several years after the Concorde was put into service.
It all boils down to the fact that the Concorde was designed for 75 or 100 years of operation at the present utilisation. And the interesting thing is that such values are in fact real since corrosion, which is common on all other airliners, is practically non-existing on Concordes because they on each flight are heated to around 200 deg. F in very dry air. Moisture hasn't got a chance to go to work.
One BA Concorde has developed microcracks faster than anticipated in the rear wing spar. Such things are normal on any airliner, and they are discovered at overhouls. Otherwise it could become catastrophic in 5-10 years time. It now gets a new spar, and that's it - ordinary maintenance. Sometimes we have a pair of socks, and one sock developes a hole on the toe or heel faster than the other for no apparent reason. Same on airliners except that they don't wait until a hole has developed, but exchange parts with new when "thinning" can be measured.
Always keep your number of landings equal to your number of take-offs