Ah yes, you’re all correct actually. I misread an article listing all BBY/ABY for each movie, re-checking it I can see now it is indeed 1 year between TESB and ROTJ (just based on the BBY):
0 BBY - ANH
3 ABY - TESB
4 ABY - ROTJ
So yeah, 3-4 years is getting a bit into the realm of an unbelievable time frame. Not impossible but definitely hard to believe by our standards even in a fictional universe as it still draws understanding from parallels and familiarity to ours.
In saying that, it’s still never bothered me personally as I never knew till looking at that article what the time lapses / jumps actually were and both sequels in the OT make you feel like a sufficient amount of time has passed to account for what has changed with our characters and the story.