What is Bloat?
Bloat is generally defined as a program taking more resources on a computer (disk space, processor cycles, RAM) than it should. This isn't always a problem – new computers are packed with powerful hardware to handle most tasks. But if you're running on old computer architecture or an ultralight netbook it might slow things down. Maybe you're just a power user or have programs running in the background. And sometimes extra features are so poorly implemented that they cause serious problems even on more powerful computer hardware.
Features You Don’t Need
Modern operating systems come packed with all sorts of features: interoperability between a wide variety of programs, search bars that automatically index everything on your computer for faster searching, and sometimes just technologies that save programmers the hassle of dealing directly with everything a program does on the most basic level. However, the latter also tends to make them run less efficiently (an unfortunate consequence of the way these things work.)
Microsoft's Windows operating system (and its Office suite especially) is notorious for being inefficient and running at a horribly slow pace, but other operating systems run slowly as well. GNOME comes with a full suite of applications that can be quite useful – word processors, calculators, a note-taking program, even games. Many of these programs are also made to do anything you could want – and a lot of stuff you don't need.
Next, we'll look at a few of the alleged causes of bloat in GNOME:
Although this isn't strictly a part of GNOME (it's only “recommended”), Ubuntu and many other distributions will install OpenOffice.org by default, and OpenOffice.org has become the de-facto office suite for the average user. It has a lot of features, and it's quite well-made. However, all those features mean that it's loading a lot of stuff that's not needed to function as a word processor. Again, it runs fine normally, but if you're looking for something with less frills, try something like AbiWord.
On a slightly different note, OpenOffice.org is really a perfect example of what causes bloating in desktop environments like GNOME: It has a lot of features, and you're not going to use all of them. It's just a matter of making the decision of whether you prefer the convenience of having them all there or the speed and efficiency of using something that doesn't always load quite so much.
KDE Applications Running in Gnome
GNOME comes with a full suite of applications, which are all pretty much optimized to work together well. But sometimes you prefer the program that comes with KDE. KDE programs will run in GNOME, but they also require the entire set of KDE libraries to be installed and loaded into memory every time they run. This is a fair bit of bloat; try using something that doesn't rely on KDE instead, or if you like a lot of KDE applications, consider using KDE instead of GNOME.
On the other hand, it you really need those KDE programs and use them a lot, go ahead and keep them.
How to Solve the Problem
Removing bloat basically boils down to two options: Changing preferences to deactivate unused options, and removing unused programs. If you're on a Debian-based system, you can use Synaptic, Aptitude or whatever other package manager you wish, to remove packages. Look for stuff you don't use, but be careful not to delete something important. Generally this shouldn't happen, as other things will depend on it, but make sure you know what you're deleting. Try using the purge option as well, to clean up – and don't forget to install updates; they often contain improvements and bugfixes.
Truth be told, though, GNOME really isn't that bad compared to the other ready-to-use, graphical operating systems out there. If you're really concerned about having an operating system that will run as fast as possible, you'll likely have something custom that will run as fast as possible, and aren't the sort of person who would be running GNOME anyway. If you're really interested, consider running a distribution like SLAX, which is made to be fast and only install what you really need, or a window manager like XFCE or Fluxbox.