When I use the term “cold fish”, I’m using it in this manner:
A person who is very reserved or aloof in manner or who lacks normal cordiality, sympathy, or other feeling.
Not as some sexual slang.
Even if I believe you, it doesn’t change my reaction very much.