I guess the whole thing really comes down to one’s own subjective credulity when it comes to the SW universe. I mean even Yoda’s X-wing gesture opens a can of worms. Okay, size matters not, so technically I should be able to hurl the Death Star into a neighbouring planet then? Clearly a lot of SW ‘logic’ comes down to how it ‘feels’ within the story.
I think you’re right in part about subjective credulity but it is possible to overthink or underthink some things.
Yoda seemed to strain (or at least devote great concentration) to harness the Force sufficient to lift the X-Wing, though he could do it. Though size may not matter in terms of whether one can do something, exerting influence through the Force appears limited by physical/mental capabilities. It may simply be easier to envision moving a small ship than it is to accept moving a planet-sized object. So that you could move the Death Star with great concentration but would be exhausting and you couldn’t just fling it around like it was nothing.
Or maybe Yoda didn’t intend his statement as an absolute (only Siths deal in absolutes 😛) whereby a Jedi can be like Neo in the Matrix if only he can free his mind. If that were the case, Yoda should have been able to toss the X-Wing in the air like it was nothing. Maybe the lesson was limited to the difference between moving rocks and an X-Wing and Yoda felt Luke could accomplish that feat.
There certainly are elements where there is no easily discernible reason for something, but once we go down this path…we sound like obsessed nerds.