A quick refresher before we begin. Computer memory is written in binary code, that is, in a series of 1s and 0s. Each of these digits is referred to as a binary digit, or bit. Memory can be stored on any physical system that can be easily turned from one easy-to-differentiate state to another, or a bistate, with one state representing 1s and one state representing 0s. So, the medium through which memory is stored can vary a lot.
From here, there's one main division in types of memory: volatile and non-volatile.
Volatile memory is memory that requires power to retain stored information. Volatile memory is vastly more expensive than its counterpart, but much quicker to access, hence its use as temporary memory. It will generally consist of a series of extremely small electronic switches that can be turned on and off, usually with capacitors and transistors.
Non-volatile memory does not require power to sustain it. This tends to make up most of permanent memory, which will use more physical ways to store memory, such as pitted plastic disks or tape with magnetic coating. Non-volatile memory tends to be much cheaper, so it is used whenever speed is not of the essence.