But no, it must be “either you’re racist or you’re fine with a racist.”
If you voted for him, then yes, racism wasn’t a big enough factor for you to stop voting for him. Thus, you’re fine with voting for a racist. Sorry.
That’s like saying, “If you voted for Hillary, than you share her views that half the country is deplorable.”
Of course, when you happily subject yourself to either/or fallacies, you might just feel that way, so…