First, you can divide approaches into precision and non-precision. Precision approaches are ILS approaches that provide both glideslope and localizer track for both vertical and horizontal guidance. As far as the different categories, the minimums are different. Example: Category III approach can go lower than Category I, but it is dependant upon special training received by the pilot (many airlines require the Captain to conduct these approaches) and equipment on board. Regular Category I ILS approach has a DH (Decision Height) of 200 feet AGL. At that point, they go around unless they see one of the lights that make up the runway environment, in which case they can descend another 100 feet before going around.
LDA approach is a localizer approach. There is no glideslope provided. Upon reaching the final approach fix (Maltese cross on the profile view on the chart) they can descend down to their minimum descent altitude (MDA), which on LaGuardia's LDA-A is 640 feet. They fly at 640 feet until hitting the missed approach point where if they don't have the runway in sight, they go around. The MAP on this approach appears to be DME 2.0 on the Localizer. The benefit of this kind of approach is that it gets the aircraft low earlier, enabling them more time to get the runway environment in sight. This wouldn't be used on a day when weather is to ILS minimums, but if there is a layer of clouds at say 2000 feet, this would be a good approach.
VOR approach is like a LDA in carrying one out. No glideslope provided, descent profile to a MDA the same way. Same procedure with missed approach point, same benefits. It is just that instead of a localizer, one is using VOR radials. This VOR can be on the field or off the field. One tracks a VOR radial into the airport, with approach fixes being DME points or intersections.
NDB approach... similar to the above except using a NDB instead of a VOR.
Spacing and vectoring required is less when using non-precision approaches. Consequently, the minimums are higher.
As far as approaches with a letter after them... such as LaGuardia's LDA-A.... it doesn't end at a specific runway. Approaches that are more than fifteen degrees off the centerline of a given runway are given letter designations and circling minimums. Basically, they can terminate with any runway.
Just a brief overview, I hope it helped.