The obvious first solution, and the one that Apple has promoted heavily, is their own (free) iBook app. Fans enjoy the real book-like feel of the app, including pages that turn realistically. Load up the app and you'll actually see a set of bookshelves, on which your books are placed; tap on the book cover to open it up. You can also group your books into collections to keep them organized. (We'll discuss this in more detail below).
Books are mostly purchased through the iBookstore (although you can also load in other ebooks you own - see below); while the selection is limited compared to, say, Amazon, it is growing rapidly; Apple claims over 150,000 books available as of this writing. Aside from traditional ebooks (both black and white and full color), the store also includes what Apple calls enhanced books, which are interactive or include additional elements such as audio and video.
Aside from searching for a particular book or author, you can see a list of the most popular books, split into paid and free choices; the free books are mostly in the public domain, but there is also some technical material Apple has put up that relates to the iPad, such as programming books for Objective C and Cocoa (which are used to program apps).