Proof that they should recognize him comes from TFA. During the battle of Takodana, he is recognized as a stormtrooper by another one even though there’s no real indicator of it. Remember when the guy with the baton thingy shouts “Traitor!” at him?
He had to have been there when Finn escaped with Poe.