Who Does It?
As you might guess, such remote access might take a pretty good hand at hacking. The technical skills required to pull such a thing off are out of the hands of even most programmers.
Cell phone companies, of course, have complete access to your phone and its functions. While the company can't legally just look into your location on a whim, employers do have this power and abuses are always, always possible. A somewhat recent example of some non-tracking abuse is Verizon employees poking into Barack Obama's phone records.
Warrantless cell phone tracking has been a part of the US government as long as there have been cell phones, and cell phone companies are forced by law to hand over any amount of control over these cell phones as either the state or federal government desires. This is for certain once the authorities have a good case for probable cause, and somewhat more borderline if they haven't gone ahead and gotten a proper warrant. This includes location data for tracking purposes. There is a possibility that the FBI does such tracking independently of the phone company, but this is rather difficult to verify.
This doesn't mean they're tracking everyone in the US at all times; only a few Big Brother theorists will make that claim. However, they do have the resources to actively track and analyze many many people at once, and they use this capability frequently for purposes of tracking criminals and suspects.