Of course physical media’s main drawback is that it can take up a lot of space, another one being the environmental harm since we’re talking about disks made out of plastic. I very rarely play my music CDs these days and have digitally transferred all of them, but having them around means I have a permanent backup in case I lose my digital files, and that’s pretty much the benefit of physical media, you own that physical copy. I wish you could have the option to permanently save your media (music, computer games, movies) permanently to your PC without having to depend on an internet connection.
If anything, my three years without internet access have given me a newfound appreciation for physical media, and I lament the fact that PC games are exclusively digital downloads nowadays. That being said, nothing beats reading physical media, be they comic books, magazines, fiction novels, etc.
Apple does allow you to keep your music offline.