How most mp3 sort scripts work is by first reading the ID3 tag of the mp3 file, to retrieve the band name, CD title, and the name of the song. ID3 tags also include other useful data like the year, track number and genre, which may be included into the sort script if desired. These ID3 tags will then be used to rename the mp3 file, usually to something simple and readable like bandname/cdtitle/song.mp3, though this will obviously vary depending on your choice format.
Then, the actual organization. Many mp3 scripts will also create a directory structure based on the ID tags. Typically, the first directory is organized by band, then by CD, and then by song. This will allow your music to be quickly and efficiently navigated to find your music files. If one wanted some other system of organization, perhaps creating directories based on genre, for instance, this is an easy change to make as well.
There are a number of scripts available online, both user generated and merely posted on forums, and somewhat more formal scripts available for download. My personal favorite of the formal downloads is offered through the Linux Softpedia,though many Linux forums also provide scripts that may be more suited to your particular needs (for example here.) If you'd just like to manipulate the ID3 data for files, the command line utility id3tool may suit your needs as well. There are hundreds of little user-created scripts floating around on the Internet, some of which are compiled into application listings like here at Linux.org.
Once you have the script, they are easy to customize. If you want to organize your music by other ID3 categories, for example by genre, or to name the .mp3 file differently, for example including the track number, it will take mere moments on a keyboard and a little scripting know-how to fit these scripts to your precise needs.
It's probably a good idea to do this sorting every so often as your .mp3 collection expands to keep things organized. It only take a few minutes, and the knowledge that you know where all your music files are is well worth it.