It's difficult to sort it out in real time while speaking. Even accomplished speakers err on the side of caution and use "who" whenever "whom" is not the absolutely obvious choice. "Who" is acceptable, especially in America, whereas using "whom" incorrectly is never acceptable. It sounds like all those uneducated people who want to sound educated by overcompensating for bad grammar and pronunciation even when it's not bad, and end up just sounding silly.
- "This is just between you and I" instead of "between you and me."
- "Bei-zhing" instead of "Beijing," as though it were French instead of Chinese.
- "I don't know whom I'm supposed to be." (The Clash, "Should I Stay or Should I Go.")
- "Of-ten" instead of "offen" for the word "often."
As François points out, to get it right you often have to deconstruct your sentence and rebuild your subordinate clause with the word order straightened out to determine whether your who/whom is the subject or the object of the clause. You can do that while you're writing but not while you're speaking.
Oli said:
Never end a sentence with a preposition.
Of course you're being facetious.
"What did you blame that lady I saw you getting mad at for?" --> "For what did you blame that lady at whom I saw you getting mad?"
Winston Churchill is often quoted as saying, "This is something up with which I shall not put." This has to be apocryphal, because Churchill of all people would know that "up" is an adverb in that sentence, as it is more often than not. It should be, "This is something with which I shall not put up."
A question that should be asked is when to use "who" instead of "that." In colloquial speech we often say, "The man that I spoke with at the store..." When the referent is a person, we should say, "The man who I spoke with." This is actually a worse grammatical error than using "who" instead of "whom," and far worse than ending the sentence with a preposition, which is not even an error in our language. (Only in Latin, it comes from 19th century English books, which were clumsy and inadvertently humorous translations of Latin textbooks.)