GPU vs CPU simplified

A Graphics Processing Unit (GPU) is a special purpose processor, optimized for calculations commonly (and repeatedly) required for Computer Graphics, particularly SIMD operations.

A Central Processing Unit (CPU) is a general purpose processor - it can in principle do any computation, but not necessarily in an optimal fashion for any given computation. One can do graphics processing on a CPU - but it likely will not produce the result anywhere nearly as fast as a properly programmed GPU.

The reasons one has at least one of each in the typical computer these days is:

• Computers that interact directly with people (i.e. not servers) typically use computer graphics extensively to do so, and the calculations involved are numerous and must be done in Real Time;
• By having at least one of each, one gains parallelism & greater performance from throughput of the computer system: while your GPU is doing calculations for graphics, your CPU can be doing other non-graphics calculations at the same time.
• Moore's Law has given us such an embarrassment of riches in silicon chip area and capability at such low prices that it's cheap to add special purpose processors of various types nearly everywhere in a computer system these days – often multiple of them, to achieve better throughput & performance.

A GPU generally has a large number of slow and weak processors. By weak I mean lower operating frequency, lower number of registers, simpler ALU's etc. GPU's come strapped with lots of memory and generally have high Memory bandwidth to support  the hundreds of small processors that make up the GPU. GPUs are special purpose and can compute vector maths, matrix maths, pixel transforms and rendering jobs about 10-100x faster than the equivalent CPU performance as all these tasks are embarrassingly parallel, although it should be noted that incredibly optimized CPU code can come close to GPU code. 

However, the amount of work required is not worth the time when GPUs can do the same calculations with significantly less work, and not requiring a programmer with an excellent background in computer architecture.

A CPU generally has higher operating frequency and is designed for general purpose use. They excel in serial tasks, branching operations and file operations. CPUs, traditionally, have been easier to program and more versatile for general purpose usage. Most office computers have rudimentary GPUs as they don't have to perform such highly parallel/ specialized tasks. But a gaming PC or a workstation PC would generally have a powerful GPU(s).