I think you misunderstand me then. (And it was I who ordered the colours in rainbow order, not a kindergartener. It was part of an exercise in art class that I thought belonged in kindergarten, so that's where that word came in).
I also would be equally annoyed if the rainbow was strongly associated with something else, and somebody thought I was making a statement about that. Or, had I drawn another symbol, for example, a crescent moon with a star beside it (presumably in a picture of the night sky) I would have been equally annoyed to have it associated with support of Islam, despite the fact that I have no problem with Islam itself.
It has nothing to do with the gay issue itself. Nor is it the fact that they chose to represent themselves with a rainbow that annoys me, just that that symbol has become so strongly associated with that movement that I can't order colours in a certain way without someone thinking it's a political statement.