Doubting what you see is a very odd experience. And doubting what you remember is a little less odd than doubting what you see. But it's also a pretty odd experience, because some memories come with a very compelling sense of truth about them, and that happens to be the case even for memories that are not true.
You know, the standard state for people is 'mildly pleasant.' Negative emotions are quite rare, and extremely positive emotions are rare. But people are mildly pleased most of the time, they're mildly tired a lot of the time, and they wish they were somewhere else a substantial part of the time - but mostly they're mildly pleased.
The experiencing self lives in the moment; it is the one that answers the question, 'Does it hurt?' or 'What were you thinking about just now?' The remembering self is the one that answers questions about the overall evaluation of episodes or periods of one's life, such as a stay in the hospital or the years since one left college.
Suppose you like someone very much. Then, by a familiar halo effect, you will also be prone to believe many good things about that person - you will be biased in their favor. Most of us like ourselves very much, and that suffices to explain self-assessments that are biased in a particular direction.