Most, if not all, of these games are highly rated and must-haves.
Games Still At Retail
Shadow of the Collosus - awesome game. Horse control sucks though.
Killzone - Best PS2 shooter.
God of War - You might have heard of this?
God of War II - Makes me want to eat innards from a decapitated goat.
Resident Evil IV - I have it for the GameCube, not PS2, but you should get it anyway.
Okami - Second Best art design ever
Metal Gear Solid 3 - Snake Eater (lol)
Games From Used Stores/eBay
Katamari Damacy - Cool gameplay, quirky Japanese humour and an infectious soundtrack.
Ico - Pioneered isolation and despair in gaming, worth the story alone.
Devil May Cry - Premiere PS2 action title, often imitated.
Metal Gear Solid 2 - Sons of Liberty - just look it up.
Any of these games are going to give you lots of entertainment - there's not a stinker on that list. If you're into Sports games, which I'm not, and of the Madden titles are top rated.