Literally no one has said that colorblind = racist.
Obviously it would be great if race didn’t mean anything but that’s not the world we live in. Even if magically tomorrow everyone stopped caring about race, you can’t overwrite history. For example, black people will still be disproportionally economically disadvantaged due to decades of all types of discrimination.