Why It's a Problem
There are several issues with Gmail's scanning beyond a simple “invasion of privacy" angle, although that can't be ignored.
Obviously, Google isn't alone in gathering advertising information. Grocery stores have been doing this with rewards cards for a while, and big stores like Amazon watch on-site browsing to better target promotions. There is a clear difference though.
First, in those cases, it's a bit more of an altruistic motive. While neither is doing it out of the goodness of their heart, there is a logic to it at least. Your neighborhood store is trading discounts for minor demographics information, which will ultimately help producers better understand their audience. While Amazon has used its marketing information to manipulate prices in the past, their suggestion service is basically just a targeted mailer to remind people of products they may want or deals they otherwise wouldn't have noticed. Gmail is just trying to get extra money (unless you genuinely care whether your ads are relevant).
Second, there is a boundary issue. Grocery stores do not have hidden recorders listening to my conversation to know which flavors of Hamburger Helper I like and why, or that I'm waiting for a sale on soda. Amazon is just watching what I've browsed on their site, using basic information that I knowingly give them. Gmail is looking through emails that will often encompass all aspects of one's life. They can get keywords and information on your workplace, personal life, relationships, hobbies, etc.
One particularly striking note are the liberties that Google is taking with non-users. Anyone who sends an email to a Gmail address will have their message read and analyzed. While this information can't be directly tied to an outside user, it is a fairly gray area to take such a drastic action when the sender never formally consents to it.
And I'd be remiss if I didn't acknowledge the current boogeyman, known as hacking. While I'm not personally an alarmist, the fact that Google is storing information about the content of emails and tying it to computers and accounts is a little alarming given their past experiences with hackers. It seems like they're just one unfortunate exploit away from giving up tons of personal information.