When a user tries to log in and fails because the username or password is incorrect, he is redirected to the ugly EE error page saying "The existing username and/or password you submitted are not valid". How can I redirect to a custom page (basically the same log in page with a little red error message added to it) instead? The EE user guide doesn't seem to say anything.
Is this within an EE template? Or are you talking about logging into the control panel?
If you're talking templates, you may want to use ObjectiveHTML's add-on Authenticate. Works like a charm and returns errors inline (if that's what you're looking for instead of EE's default page).
Take a look at FreeMember, which lets you display inline error messages for your login/registration templates, as well as update the user's profile details etc.