Sorry I noticed this topic late but, as an ol-timey railfan, thought I'd chime in too. Most all replies are part of the equation, but LHmark added another reason for the early intent (the brakeman). The caboose was the "office" for the conductor who kept tabs on waybills, schedules, etc., looked out for trouble on the train (usually via upstairs cupola or bay windows) and a brakeman, and sometimes a flagman. When two brakemen were used (one in caboose and one forward) they would exit their stations to the tops of railcars and manually adjust the cars' brakes, often meeting near or at the center of the train of cars. The flagman would "flag" missions accomplished to the engineer and fireman. Sleeping bunks and stove for heat and cooking were common.
Better air-actuated brakes eventually eliminated the need for a brakeman (or, brakemen) except while "humping" railcars without power into railyards, and that station was eventually eliminated as was that of flagman. Conductors retained their duty as paperpushers as well as observers for wheel fires and other problems that the engineer and firemen would be too busy to watch for. Eventually, electronics replaced the conductor as well as the station (caboose) at the rear of the train.
As some of you have noted, it's very analogous to the elimination of radio operators/navigators/engineers on commercial aircraft. Trains don't look right without cabooses, and I still miss the perspex bubble behind the cockpit of airliners, too. Regards...Jack
all best; jack