A common area of confusion for developers new to EE is caching. EE provides a number of caching options that operate at different levels and can interact with each other, sometimes in unexpected ways. This tends to mean that performance does not always automatically improve when using caching unless it is applied intelligently. Understanding EE's caching behaviour is thus an important part of learning to use EE "properly".
So, can you explain the different types of caching that ExpressionEngine provides, specifically their pro's and cons, and why you shouldn't automatically turn on caching here there and everywhere, just because it sounds like a "good thing". (I mean, everyone wants a faster website, right?)
I know this is a complex topic, so do link to external resources, but it would be great to provide a general overview directly in the answers here as well.
NB, While I appreciate that on some high traffic sites, paying extensive attention to caching/performance or even implementing Varnish or similar can make a lot of sense, for this answer I'm primarily interested in what makes sense as a baseline/starting point for a "normal" small/mid-sized ExpressionEngine site, and in using the native caching options that EE provides out of the box.
