Sharks and rays are known as elasmobranchs (subclass Elasmobranchii), and they share a common ancestor separate from that of chimaeras (subclass Holocephali). Living elasmobranchs descended from a group called Ctenacanthoidea, which went extinct in the early Triassic period, around the time the dinosaurs first evolved.
The descendants of the Ctenacanthoidea are the extinct group Hybodontoidea and the lineage that includes modern sharks, Neoselachii (literally, "new sharks"). Neosalachii radiated into two major lineages during the Mesozoic era: Galeomorpha (one representative is the modern great white shark) and Squalea, which in turn consists of Squalomorpha (one representative is the modern dogfish shark) and Hypnosqualea (one representative is the modern manta ray). Great whites and dogfish are both commonly called "sharks" and superficially resemble each other more than either resembles a ray, so traditionally they were classified together in superorder Selachimorpha (literally, "shark shaped"). Recent analysis using cladistics shows that, in terms of phylogeny, rays and dogfishes are actually more closely related to each other than either is to great whites. The group known as Selachimorpha is, therefore, a paraphyletic group. It can still be found in some classification systems, but in systems that use phylogeny exclusively, it is considered invalid.
The most noticeable difference between the two lineages of sharks is that animals in the lineage that also contains rays, Squalea, have no anal fin, while animals in Galeomorpha do have an anal fin. The anal fin is a synapomorphy (new adaptation) in Galeomorpha, not a secondary loss in Squalea. It acts like a keel, located on the underside of the fish just behind the cloaca (anus). Squalea contains mostly bottom dwellers, so an anal fin would get in the way of their lifestyle. Galeomorpha contains mostly active swimmers, some of which will actually suffocate if they stop swimming because the forward motion is what causes water to pass over their gills. An anal fin is very useful to provide stability to actively swimming fishes.