The key difference is how the two processors function. A GPU is designed to do the same calculation lots of times at once, though with differing values, while a CPU is designed to do lots of different calculations quickly.

A simple way to think about this logic is that a single object on the screen in a game will be on multiple pixels of the screen at once, and each of those pixels will generally need to do the exact same set of calculations with just different input values (think like a*b+c with differing values for a, b, and c). The actual rendering process does the same idea at multiple levels, where you are typically going to position and rotate the points (vertices) of each object in the same way. It also turns out that this same style of calculation is useful for a lot of other stuff: physics calculations*, large math problems*, and artificial intelligence*, to name a few.

However for general program logic you aren't repeating the same calculations over and over with just different data, but instead need to vary the calculations constantly based on what the user is trying to do. This logic often takes the form of "if X do Y else do Z".

Now, modern CPUs will have some hardware designed to function like a GPU, even if you discount any embedded GPU. Using this is very good if you just need to do a small amount of that bulk processing, such that the cost of asking the GPU to do it and receiving the result will be too expensive, however its no where near as fast as the full capabilities of a GPU.

Beyond those design differences which are shared between dedicated and embedded GPUs, a dedicated GPU has the benefit of having its own memory (RAM) and memory bus (the link between the processor and memory). This means both the CPU and GPU can access memory without stepping on each other and slowing each other down. Many uses of a GPU can see massive benefits from this, especially games using what is known as "deferred rendering" which requires a ton of memory.

As a note, there is no reason you couldn't just do everything with one side, and, in fact, older games (eg Doom) did everything on the CPU. In modern computers, both the CPU and GPU are what is known as Turing complete, which means they can theoretically perform every possible calculation. Its just that each is optimized to perform certain types of calculations, at the expense of other kinds.

* As a note, artificial intelligence heavily relies on linear algebra, as does computer rendering. Many other math problems can be described as such, converting the problem into a set of matrix operations, which is specifically the specialization of GPUs.