Also: The shadows/ghosting sometimes appears to be a result of AI upscaling, for it tries to add detail but perhaps confuses it with artefacts still present in the original. Unfortunately I can’t remove them before feeding the file to the AI, that looses more detail (I have tried several times) - the best result is AI first, filters afterwards.
Whenever I did the filter-work before, people or objects in the background looked smeared and their eyes scratchy.

For now, this is the workflow I feel works ok in 90% of cases. Of course, if anyone comes up with some idea to do it better, throw your thoughts at me.