It only does this when I reply. Honestly, I have to play with it for a minute to even get it to let me type in the reply box. When I type in the reply title box or anywhere else, it doesn't do this. After every letter I type in the reply box, it turns it's self back on.
If you want to turn on Cap Locks, go to Settings-General-Keyboard-then it should be on that page somewhere.