I think you are making two important points here:

1. It is a good book. He is a great writer and the book being based in reality or not doesn't really take away from that. If anything I would see him as more of a creative storyteller if he simply made most of it up.
2. When we tell stories about ourself they become just that. Stories with us as the unreliable narrator. Sometimes we remember things wrong, sometimes we draw the wrong conclusions when we lack information and sometimes we embellish or lie. But we always experience and look back on things with our own bias, If I have been told that my uncle was crazy then that is going to color my memories of him. And there is not going to be some sort of all knowing and neutral third party to tell us or others what is 'really' true.