I believe it has something to do with privacy laws. Supposedly, since those ATC conversations are technically not directed at you, but the tower/radar/aircraft/etc, you are not authorised to hear it. For that same reason, it is LEGAL to listen to ATIS, since that is sent out for the benefit of everyone. It's very strange. Some people argue that if the ATC conversations were supposed to be only for the pilots, they should use some sort of encryption or something so that "commoners" can't hear it. However the law is the law. It can be compared here to listening in to someone's private telephone conversation, or opening someone else's mail.
On a side note, I have heard of people at airports being relatively lenient on this subject. I heard they will either just let you listen, or tell you to turn it off...not arrest you! However don't count on it!
I guess that's what happens in the end, you start thinking about the beginning.