My dad was kind of like that too. Not rough and cold to such a degree, but quick to anger and he believed in corporal punishment and would get angry just for strange things like making my sister laugh in the backseat when he was driving. Like you, I came to terms with it and I loved my father very much despite his unpredictable temper.
I've found that The Tree of Life dredges up memories in its audience in a way few films do. I think it has to do with Malick's style, his talent for bypassing the intellect and hitting you straight in the gut with some abstract emotion you weren't prepared for. It took me years to appreciate abstract expressionism as not an intellectual style but a viscerally emotional one, and I think Malick is perhaps the best of the best expressionistic filmmakers alive.