I don't think it was a fallout, unlike Kurtz. Kershner found the shoot very tiring, as they ended up going overbudget and so on. It was also a pain for Lucas to have an American director, since he quit the director's guild - he had to pay a fine for doing so. Hence, one of the reasons he hired a Brit for ROTJ.