I always assumed that the Star Destroyer couldn't detect the location of the Falcon simply because it was in a sweet spot where it could be parked ON the much larger ship, making it appear that the Falcon is just another piece of a huge mass on their radars or sensors, or whatever they were using. Not to mention they were in a nice spot where no naked eyes inside the ship could see them hiding there.

I feel it doesn't need to be any further complicated than that. After all, we've all understood that scene without confusion since 1980, or whenever we first saw the movie.