C3PX said:
I could be way off here, but I have never had any problem believing that Tolkien really did write Lord of the Rings allegory free (as he claims he did).

I believe the term he used to describe his work was applicable, as opposed to allegorical. He was certainly free to use whatever term he wanted, but they are very similar - in this case, a story being used to explain or demonstrate something that at first may appear unrelated.

I just don't see how it is even possible for a writer to delude himself out of truly understanding what he himself actually wrote about.

I don't think he was deluded about what he wrote at all. I think he may not have wanted to appear as a political writer, so he chose to sometimes deny a connection. He said his work was more of a religious-based story about good vs evil - again, allegory or applicable. He was a devout catholic and very outspoken on the evils of industrialization. To think or state that those core beliefs aren't present - and at times a major theme - seems like quite a stretch.