Toilet plunger, kitchen sponges, cleaning sponges (do NOT use one set of sponges for both kitchen and non-kitchen!!), toilet brush, an all-purpose cleaner like Mr. Clean, can opener, bottle opener, corkscrew, food storage containers, a good airtight garbage can, garbage bags, a laundry basket, cutting boards, sealed canisters for dry goods like sugar (keeps them fresher longer and stops bugs etc. from getting into them), vacuum bags if your vacuum uses bags, hand soap (in a pump bottle or a soap dish for bar soap).
One of our cousins gave us the best wedding present ever - a laundry basket full of things like the above. We were just out of school when we got married and moving 2000 miles away from home to our own first place. I still use some of the stuff she gave us on a regular basis, 17 years later!
Remember that ladies always find it impressive if a guy has a clean place. Or on the flip side, a dirty place is a real turnoff. I had a couple of guy friends who I'd make clean the toilet before I'd go over to their place.