Has anyone here ever attempted colorizing old black and white photos?

I am undertaking such a project just now and I'd sure appreciate any experience anyone has with this. The pictures in question are old, the one I'm working on right now was taken in 1894 or 1895. They are color scans of the originals and are very sepia'd and blotchy. I've worked for hours taking dust and scratches off and other cleanup.

Here's what I've done so far for my best results. While it is still in color, I adjust the RGB to get me the best-looking contrast and detail. Then I take them to grayscale and do the dust cleanup etc. After that I put them back into RGB and begin "hand painting" the color in. I will highlight an area, a clump of bushes or a building or something like that and adjust its colors to the most natural look; blues and greens for some bushes, blues and reds for other types for example.

I'm getting pretty good results but it is really a lot of labor. In Photoshop I'm sometimes working at 1600X, pixel by pixel. The whole project is just to show a group of people a more natural view of a place in another time. It is not for strict historical accuracy, so if I get a house yellow that actually was brown that is okay. The overall result is aesthetically quite pleasing.

I'd sure appreciate any time-saving tips anyone might have though. I'm growing old doing this!