Troubleshooting ssh "Too many authentication failures"

Some time ago all of a sudden, I was unable to ssh into a host anymore with the above error.  I didn't even have a chance to enter my password.

Eventual the cause of the problem was found.  I have multiple ssh keys but none of them is for the target host.  SSH is configure to use public key authentication before password authentication.  Hence ssh tries all my keys without success, hitting the maximum password attempts, and ends with the error "Too many authentication failures".

The solution:

Force ssh to not use public key authentication, and then copy over
the public key for the target host:

$ ssh-copy-id -i ~/.ssh/id_rsa.pub "-o PubKeyAuthentication=no user@host.com -p 22"

Note the use of double quote, otherwise ssh-copy-id will not recognize 
the ssh options.