I bought the cheapest used computer I could find--a 2001-era Dell--just so I could run Windows 95 games without problems. It just seemed easier to me. It's a Dell computer from 2001, cost me $20. I know it sounds a bit extreme to buy a whole computer to run vintage games--although no one bats an eye at dropping $70 on an NES--but honestly for $20 it's worth the money, and I'm sure you could find one for free as long as you were willing to go pick it up, I was just lazy and actually found a $20 bill on the sidewalk that week so it seemed fair. It's hooked up to my modern LED monitor and the tower itself is pretty small so it doesn't actually intrude on space. It also runs DOS games without problems. And of course the floppy disc drive if you have pre-CD-ROM games like Police Quest.
Is it possible to install a dual OS and actually run your modern computer in W95? I never tried that, but if you were really into vintage PC games but don't want to grab a second computer I would look into that. I've played some old games in W95 compatibility mode, but I've never been fully satisfied with the results, there was usually a glitch or two, so I just went on Craigslist last year and bought the second computer. Usually the seller will give the software they had too, since it would be useless to them. I got a copy of Redneck Rampage which I had forgot even existed!