First of all; if you're unfamiliar with the exciting GPS device assisted activity known as geocaching, read What is Geocaching? That article gives you a great background and should picque your interest enough to want to know all about where you can do it which you find below.
Geocaching.com is the premier site for finding and logging geocache hunts with literally hundreds of thousands of geocache hunts available. To find geocache hunts on the site, you must start by registering for a membership. A free membership will allow you to search for geocache hunts and share your location, while an upgraded "Premium Membership" lets you organize your favorites, create custom searches, receive notifications of new geocache hunts on your mobile device, search for geocache hunts on route to a destination and more. The Premium Membership is a bargain at only $30 for a year's subscription. However, you may want to try the free membership first to see how well you like it.

click to enlarge
Once you have registered and logged in, all you need to do is select Hide & Seek a Cache from the site's menu. You can then search for a geocache hunt by address, zip code, state, latitude and longitude, keywords and more.