The question that is the subject here is one of the basic questions of cosmology. The immediate and simple answer is that if you consider the frame of reference as astronomers do, from any single perspective all galaxies are moving away from the observer. (Except for those in our local group which are under mutual gravitational attraction.) This is evidence of the uniform expansion of the universe. A simple balloon example will demonstrate this.
Two ideas made their way out into the scientific public in the 20th century. One was that the universe was expanding, the other was that there was an origin in time to the expansion.
The first idea readily leads to a question, if it is expanding, what is it expanding into? More space? That means that space is expanding into space. But that does not make sense.
Think of a balloon that has dots on the surface and gets blown up. As the balloon gets filled with air it gets bigger and the dots separate, the "expand" in their distance from the other dots. The balloon itself expands, but it expands into space.
This is a common way to look at the expansion of the universe like a ballon, and the galaxies are the dots.
Image Credit, NASA, The Space Place
The second idea that entered the scientific consciousness was the Big Bang. This event occured some 13.7 billion years ago, and it was explostion of space, not an explosion in space. The laws of physics began to take shape and form, but not all at once. For example, It took about 400-500,000 years for atoms to form.The universe that we see now was not the universe that was around at the time of the Big Bang. The physical laws that we see now, were not the physical laws present at the time of the Big Bang.
So what was the Big Bang? And how is an explosion different from the Big Bang? Consider an explosion that takes place some where in the desert. What do you see? First there will be a bright light. Then you will feel the shock wave followed closely by the sound wave, and followed by debris. And even after that, over the course of time, the debris will settle down leaving only a small trail of dust in the air. The geometric sequence of the explosion is like this: there is a central point, followed by a growing diameter (a function of the power of the explosion) getting larger and larger until the outer circumferential edge is reached; it is the furthest point where any debris is scattered. This scenario can explain the explosion, but it cannot explain the Big Bang.
If the Big Bang were an ordinary explosion, in an already existing space, we would be able to look out and see the expanding edge of the explosion with empty space beyond. But that does not happen. Rather the evidence shows that if you look backwards towards the Big Bang's point of origin itself you do not detect some blazing hot point with bright lights showering everything in every direction. Instead you detect a faint background glow from the hot primordial gases of the early universe. This was detected by the COBE , the cosmic microwave background explorer satellite.
You don't see a massive explosion, you see only a faint glow. In other words, there was no massive explosion first followed by a faint glow afterwards of "things" settling down. No, instead you see only a faint glow which is the cosmic microwave background radiation, the remnant of the Big Bank, which is uniform in all directions. This tells us that it is not matter that is expanding outwards from a point, but rather it is space itself that expands evenly.
In the Big Bang model there was no space and time before the Big Bang. There was not even a "before" to speak of. So, the Big Bang was very different from any explosion we are accustomed to and it does not need to have a central point. Put it like this, the "explosion" occurred everywhere at the same time. The result of the Big Bang was the creation of the physical universe that we live in, and are learning the laws of, that govern the motion of all the matter and energy in the cosmos.
Image Credit:The Smoot Group