It's more that the name "smiler" is just mentioned out of nowhere in the episode, and if you hadn't been keeping up on Doctor Who news/rumors, you wouldn't know what the hell he was talking about. I know my buddy I watched the episode (only his second) with had no idea who the Smilers were by the end of the episode, just that there were those "creepy fortune-teller thingies everywhere," and there was "something else called Smilers that we never saw."

I guess that explanation about Timmy makes sense, I just didn't feel it gelled with the rest of the episode very well.