One simply don’t hang up with rebels on an ice planet performing patrols for fun. Hanging around with rebels is not like being in a social club. You don’t simply associate yourself with people who are being perceived by the Empire as worst-than-criminals just to hang around with “friends”.
Where is that stated in the script? Isn’t this also just an interpretation based on your impression? Or is your impression above everyone else’s?