Is it really that we have not been following real vedas and the caste transformation is a trend brought down by the rulers in order to rule since as per vedas and Upanishads, any infant is a Brahman by default - their actions in that lifespan gives him his caste and sects.
In the Chandogya Upanishad (IV. 4. 1-5) we read of Satyakama Jabala who was not sure who his father was and took the name of his mother (Jabala). When asked by the sage he approached to be his teacher (Haridrumata Gautama), Satyakama told the truth of his uncertain lineage. To this the sage replied (verse 5):
To him [Satyakama] he [Haridrumata Gautama] said, ‘A non-Brahmin will not be able to say this. Fetch faggots for sacrifice, O good looking one. I shall initiate you since you did not depart from the truth…
Swami Vivekananda says throughout his works that caste is a social institution, not a religious institution. Although he believed that caste is necessary (there are castes everywhere in the world, all cultures have castes) he was against the institutional caste in India that says that your caste is determined by your birth. He says (Complete Works, V3 p 457, Here under the heading Lectures from Colombo to Almora, subheading The Religion We are born in - http://cwsv.belurmath.org/volume_3/vol_3_frame.htm):
Who are these Rishis? Vâtsyâyana says, "He who has attained through proper means the direct realisation of Dharma, he alone can be a Rishi even if he is a Mlechchha by birth." Thus it is that in ancient times, Vasishtha, born of an illegitimate union, Vyâsa, the son of a fisherwoman, Narada, the son of a maidservant with uncertain parentage, and many others of like nature attained to Rishihood. Truly speaking, it comes to this then, that no distinction should be made with one who has realised the Truth. If the persons just named all became Rishis, then, O ye Kulin Brahmins of the present day, how much greater Rishis you can become! Strive after that Rishihood, stop not till you have attained the goal, and the whole world will of itself bow at your feet! Be a Rishi — that is the secret of power.
This Veda is our only authority, and everyone has the right to it.
— Thus says the Shukla Yajur Veda (XXVI. 2). Can you show any authority from this Veda of ours that everyone has not the right to it? The Purânas, no doubt, say that a certain caste has the right to such and such a recension of the Vedas, or a certain caste has no right to study them, or that this portion of the Vedas is for the Satya Yuga and that portion is for the Kali Yuga. But, mark you, the Veda does not say so; it is only your Puranas that do so. But can the servant dictate to the master? The Smritis, Puranas, Tantras — all these are acceptable only so far as they agree with the Vedas; and wherever they are contradictory, they are to be rejected as unreliable. But nowadays we have put the Puranas on even a higher pedestal than the Vedas! The study of the Vedas has almost disappeared from Bengal. How I wish that day will soon come when in every home the Veda will be worshipped together with Shâlagrâma, the household Deity, when the young, the old, and the women will inaugurate the worship of the Veda!
Some of the other things he said include:
Though our castes and our institutions are apparently linked with our religion, they are not so. These institutions have been necessary to protect us as a nation, and when this necessity for self-preservation will no longer exist, they will die a natural death.
I must frankly tell you that I am neither a caste-breaker nor a mere social reformer. I have nothing to do with your castes ot with your social reformation. Live in any caste you like, but that is no reason why you should hate another man or another caste. It is love and love alone that I preach, and I base my teaching on the great Vedantic truth if the sameness and omnipresence of the Soul of the Universe.
And Ramakrishna Paramahamsa said (The Gospel of Sir Ramakrishna, Chapter 34: Bankim Chandra, available here - http://www.belurmath.org/gospel/index.htm):
The Master used to say that the devotees formed a separate caste by themselves, among them there could be no caste distinctions
And in Chapter 6: The Master with the Brahmo Devotees, he says:
You asked about caste distinctions, There is only one way to remove them, and that is by love of God. Lovers of God have no caste. Through this divine love the untouchable becomes pure, the pariah no longer remains a pariah. Chaitanya embraced all, including pariahs.
To more directly answer your question, no, we are not all born brahmins. Your present birth is determined by your previous karma. This is supported by scripture. The Brahma-Sutras 3.1.8-11. has a discussion on this and both Sankara and Ramanuja in their commentaries on these verses are in agreement as to the meanings. Verse 8 says (Swami Vireswarananda translator):
On the exhaustion of (good) work (the soul) with the residual Karma (descends to this earth) as is known from the Sruti and Smriti, along the path (it) went by (from here) and differently.
The commentary to this verse quotes the Chandogya Upanishad 5.10.7 saying:
Those whose conduct has been good will quickly attain some good birth..."
And further in the commentary it says:
The Smriti also says, "With the remainder of their Karma they are born in a noteworthy place, caste, and family, with becoming appearance, longevity, knowledge, wealth, happiness, and intellect." So the soul is born with residual Karma.
In summary, we are not all born brahmins. Birth is determined by residual Karma. But caste can be determined by the qualities you manifest in your life, not simply your parents caste.