Prequel-based reasoning so it may not fly here, but could some of the in-universe prejudice stem from the Clone Wars?
If we were retconning I’d like that as a reason. If used in conjunction with the general ignorance of people, it could make a plausible narrative.
There are plenty of helpful droids in the prequels though.
See above re: ignorance of people. Give them just the slightest excuse to justify a prejudice they already have…