After spending many hours out at LGA either picking people up, dropping them off, or just being out there, I think I have found the best places to spot at LGA.

For RWY 31 in use for Landings and departures

Landing Lights park (the one that runs parallel to RWY31 on the river) is good for side shots, but you're still not that close to the arriving a/c. My suggestion would be to go to the Northwest parking lot. IF you're taking public transportation take the M60, Q48, or Q33 to the Delta/Northwest stop. Then get off, walk backwards, (towards Shea Stadium) and bear left, and you'll come to the NW parking lot. If you drive, you can park in the lot, no questions asked for $4.00 for 2 hours. When you get into the lot, go all the way to the end, and you're literally feet away from the runway. This is a great place for head on shots, and low side shots.
When I was there, I didn't get hassled by any security or anyone. Although you may get some weird looks from people, most people will probably think you're weird and leave you alone. Hey, if you get kicked out, you'll at least have some shots, and if you parked there, you'll be out $4. Not too big of a deal. You'll get a good feel for the power of the departing a/c on RWY13 and landing ones too on 31.

Another place for RWY 31 is in Southern College Point. This is a lot tougher to see the a/c, but they literally fly right overhead if you can get along the water. Take College Point Blvd (Q65) to 31st Av, and walk all the way to the end. It'll be kinda tough to get all the way to the water, because there's a lot of security with all the "plants" and all, and you'll be considered trespassing, but it's a great view. Still not better than the NW parking though.

For RWY 4 in use for landings and departures

They have taken away all of the "green siding" along the LGA terminal road on the fence separating the airport with the road, and there is a little concrete path there. If you really want to feel the power of an aircraft, walk along the concrete path all the way, and stand right behind the runway. Depending on who is patrolling the airport, you'll either be asked to leave from standing behind the runway, or you won't be bothered much. You can get great pics there too.

For RWY 22 in use for landings and departures

College Point shore front park is about the only clear spot to stand. This spot is not so good for departures, just landings. You can take the Q25 there and walk to the water.
For the occasional departure, the fence at the end of RWY 22 (beginning of RWY4) is a great place for pics, etc... Again, it is a risky spot as you may get kicked out.

RWY 13 for landings isn't used all that often, but Randall's Is. although far away is a decent spot. If you can manage to remain on the Rikers Is. Bridge. then that's a really good spot.