My understanding of the ego is that it doesn't go anywhere. It's always with us. Rather, with practice, it exerts less control over our actions. In terms of this discussion, I would describe the ego as a prism or lens that can affect perception. When we become the observer of self we're able, to a degree, to remove the lens to see what is there, uncolored or unaffected by our experience (or the ego's need to interpret and be relevant).