I guess some people just have a more refined taste when it comes to their “individual impression” beyond whether they can remember a piece of music they heard two years later.
You guess wrong. If something makes an impression on you, then you will remember it. If not, you will most likely forget it. That is just how human brain work.