Personally, I find Temple of Doom the best of all the films. I know the characters are a bit caricatured, and I know Raiders is much better made. But I can't resist the whole "Weird Tales" angle of the film. Sacrificial death cults in underground lairs brainwashing people with voodu and ripping people's hearts out in fire sacrifices. Count me in. Temple of Doom is the closest I have ever experienced to one of those Haunted House amusement park rides. And it has a really genuine intention to scare you. Spielberg is apologetic about it nowadays and tries to pass the buck onto Lucas, but you can tell he was firing on all cylinders to make it a the most terrifying thrill-ride he could. I can forgive the raft scene, and the monkey-brains scene, and the character of Willie on the basis of all this. It's part of the charm. Either you like horror-pulp or you don't, if you do you probably will find all of this is part of the genre, if you don't then you can always watch Raiders and its remake. :P
You know, the stupidest Indy moment for me was always the boat sequence at the start of Last Crusade. How does Indy get back to shore??? Temple of Doom, I can forgive any absurdity because its basically a comic book, but Last Crusade is more in the vein of a realistic family drama in a lot of ways, probably the most down to earth of all four, so this really stands out as unrealistic and convoluted.