They both go. Not together though. Its implied that Sidious goes in secret to murder the children and the blame is laid on Anakin. Since its not something Anakin is ordered to do or would likely do. Then when Padame mentions the story told to her by Obi, Anakin dismissing it is because its more lies of the Jedi as far as he is concerned
This is a fantastic idea.