Self atman or anatman?

Question

Im trying to understand the idea of an unchanging Self in Hinduism, though I come from a Buddhist background I am very open. Have not given the idea of the Self much consideration before I wonder how does the Self remain unchanged even though it is affected by karma through subsequent rebirths or reincarnation. Perhaps I am seeing a small s self and not the capital S self, but it would seem that reincarnation effects the small s self and karma does to that the large S is also affected by gathering more or less neg, pos, netural Karma, that is, say an untold set of rebirths the Self would be affected and therefor changed in position to its relation to karma and even brahman. The self at the start of the journey is not necessarily the same at the end of its journey, because it is free of neg karma. I find the very interesting can you give me an answer or suggest reading(I love books). Thank you.