Ben sat down on a log in ROTJ. And Yoda was on the Ewok village railing at the end. I imagine some physical contact is possible. Even more so when in a location strong in the Force.
Also, the vision of Vader Luke encountered in the Dagobah cave was solid enough when he fought with it.