From what I hear, apparently there was a problem between him and the directors, so the 2nd unit or something like that had to film all the scenes with Brimley in. Joe Johnson was heading up the 2nd unit.
To be honest, I heard about this from here.
It may or may not be true.