PortableApps.com is an open source project which brings together a shell program for your your USB stick with a range of open source applications:

click to enlarge
The advantages of carrying your applications around on your USB stick are:
1.You can use them on any PC
2.You can take your bespoke settings with you, for example your books marks including active bookmarks in Firefox.
3.You can use them on PCs without storing any information on the host PC
The default installation installs with a range of standard applications including OpenOffice, Firefox, an Acrobat reader and an anti virus checker. When minimised, it sits quietly at the botom of your screen. When selected, the shell allows you to add a range of applications including 7-zip, the compression utility covered in http://www.brighthub.com/office/home/reviews/10118.aspx, the Filezilla FTP client covered in http://www.brighthub.com/office/home/reviews/18223.aspx and Audacity, the open source audio editor.
There are some limitations:
1.Performance is limited by the speed of transfer, and with older PCs with USB1.x slots, this may be a particular problem
2.If your favourite is not available in portable form then you have to choose between convenience and the application of choice.
3.Sometimes the portable version lags the latest desktop version eg OpenOffice portable at the time is at version 2.4, behind the latest desktop version 3.0.1.
4.One of two applications are reported to leave some settings on the host machine
Finally, you should always remember to protect data on your USB stick using passwords to protect directories of files.