The local government offices are your last stop before opening any type of small business. Start with the city or county clerk, who handles local business registering. Next, go to the planning and zoning office, who will tell you what you can and can't do at specific locations. The building and safety, fire department, and health inspector are the next group. They all must sign off on your location and your intentions for that location. The tax office rounds out the list as these people will get you set up with local taxes and regulation.
Depending on your location, it can take up to six months to get all of this accomplished. But with these steps complete, you can concentrate on making money.
Reference:
Inc. Magazine - http://www.inc.com/guides/2010/10/how-to-start-a-sole-proprietorship.html