Like the topic suggests, the final pieces of the puzzle are left. These include input and output devices and peripherals like hard-disks, CD/DVD readers/writers, keyboards, mice and the like. You can easily choose anything here without having to worry about compatibility with Linux. Still, keep in mind that your motherboard should support the relevant interfaces for any devices you buy and install.
Depending on the purpose of the computer, the amount of storage can be decided. For file-serving purposes, use multiple disks with large capacities. For other I/O intensive purposes, use multiple disks which run at 10k RPM or higher.
Ubuntu Linux will support the basic functions of all keyboards and mice you connect to it. For CD/DVD writing purposes, you can refer to Ubuntu's package manager for applications.