Harmy said:

But I just figured out something that may make this issue much better, it is a filter called Change to colour, where you pick a colour you want to change and a colour you want to change it to.

Sounds great! Such filters can be very useful but often comes with side effects. Just be careful like you said, with those artefacts and other colors being affected.

Anyway, for everyone interested, here's a couple comparison shots of the file I gave Harmy and the render he uploaded. I think especially the first shot here looks better with Harmy's adjustments.