Removing bloat basically boils down to two options: Changing preferences to deactivate unused options, and removing unused programs. If you're on a Debian-based system, you can use Synaptic, Aptitude or whatever other package manager you wish, to remove packages. Look for stuff you don't use, but be careful not to delete something important. Generally this shouldn't happen, as other things will depend on it, but make sure you know what you're deleting. Try using the purge option as well, to clean up - and don't forget to install updates; they often contain improvements and bugfixes.
Truth be told, though, GNOME really isn't that bad compared to the other ready-to-use, graphical operating systems out there. If you're really concerned about having an operating system that will run as fast as possible, you'll likely have something custom that will run as fast as possible, and aren't the sort of person who would be running GNOME anyway. If you're really interested, consider running a distribution like SLAX, which is made to be fast and only install what you really need, or a window manager like XFCE or Fluxbox.