In the manner of our previous rule 'If you can't afford to lose it, back it up,' the rule for storing your data backups is 'If you can't afford to lose it, don't get rid of it.’ Determining what you cannot afford to lose is entirely up to you.
If you decide that you can never get rid of any of your backups then it may begin to look like you will be forced to store a lot of backups. However, just because you backup data on a regular (possibly daily or weekly) basis doesn't mean that you are doomed to wade through thousands of disks, or purchase a new external hard drive every couple of months. There are affordable and convenient ways to ensure that you do not lose anything, and that your backup solutions take up minimal space. You can reliably backup data for years with as little as 1 external hard drive and 5 DVD-RW's.
Using Windows built-in backup tool you can schedule daily backups of different types. Store all of your crucial data in one directory. Label 4 of your DVD-RWs with the numbers 1-4, and the last "Daily". The first thing you need to do is a normal backup of all of your crucial data to the external hard drive. Be sure to include the month in the filename. From here on out the only files that will be backed up are those that are changed. Schedule an incremental backup to a DVD-RW (1-4) once weekly. Every remaining day of the week do a differential backup to the "Daily" DVD-RW, being sure to over-write. Alternate the disk being used for the weekly incremental backup (1-4 represents the week of the month). Once per month schedule a normal backup to your external hard drive, and do not overwrite previous backups.
This method takes determination and daily attention, but ensures that your data is safe and doesn't take up hardly any space, therefore allowing you to store your crucial backed up data indefinitely.