well if you think about it, how long would you stick around for if you had nothing to do. a person can only go for so long. it seems that in that conversation ben knows that luke doesnt need him anymore, so there was no point of sticking around any longer. only makes sense to me.