Why not GIF?

GIF is an outdated format, which has some (resolved) patent issues. It has a limited color palette (256 colors at maximum) and you may only define one of thos colors as completely transparent. The required color reduction results in bad quality graphics, once there is some kind of gradient in the image.

PNG uses a lossless compression, with an unlimited amount of colors and each color may be semi transparent with a resolution of 128 steps.

PNG results in higher quality images, usually with a equal or smaller file size and you may use its transparency features.

Why not JPEG for graphics?

For a lot people JPEG is just the best knwon graphics format, so they use it for everything. Why is it a very bad choice for graphics?

JPEG uses a compression based on the Fourier Transformation, which means, that the image is basically transformed into a three dimensional cosine function and this function is actually stored. Cosine functions are curves and therefore it is hard to approximate hard edges with them - you would need to overlay a lot of terms...

The quality factor when storing JPEG is broadly the amount of used cosine terms, so that the hard edges are described better, the higher the quality factor is. But still with a quality factor of 95% you get something like this, when you got hard edges in your image:

JPEG compression

Those edges are very common in all generated graphics, but not in natural pictures - so it is OK to use JPEG for pictures.

Why not PNG for pictures?

PNG uses filters and the deflate algorithm known from ZIP archives. In graphics the adjacent pixel often do not differ that much or are even equal to each other. PNG only stores the difference between adjacent pixels, so that the compression works best, when you have large areas of the same color. After that everything is just compressed with the deflate algorithm.

This usually does not help very much for pictures, that's why you should use JPEG here. The picture is still stored with lossless compression, but the file size might be quite big.