TLDR, Short answer: Vedic Caste is by birth, as it is by birth, it is in body, as soul can take birth in different castes, the caste cannot be in soul. So the caste is in body and it is by birth.
A person of higher caste can loose his/her caste without proper practices and behaviour. But one cannot ascend caste hierarchy by good practices in the same birth, but doing so one can be born in higher castes in next births.
The objection that caste cannot be of the body as it is all panchatattva is meaningless. As caste itself is a feature of the panchatattva body. The Trigunas themselves belong to panchatattvas and matter.
सत्त्वं रजस्तम इति गुणा: प्रकृतिसम्भवा: | निबध्नन्ति महाबाहो
देहे देहिनमव्ययम् ||14.5||
O mighty-armed Arjun, the material energy consists of three guṇas
(modes)—sattva (goodness), rajas (passion), and tamas (ignorance).
These modes bind the eternal soul to the perishable body
-Bhagavad Gita 14.5
Let's understand this with some examples:
It's like asking where is the colour a person, and if the answer is it is in the body in the form of skin. You can't say, "it cannot be so, as all bodies and skins are made of panchatattva and all panchatattva are same", because the way panchatattva is combined in different proportion and structure gives different colour. Colour itself is the feature of the panchatattva.
Why is onion tamasic and wheat is sattvic? It's all panchatattva. But it's difference in quality belongs to the panchatattva itself.
Why is Gold and Iron different? It is all panchatattva. But the difference itself is a feature of panchatattva.
In the same way caste is a feature of the body like hair(Straight or Curly), colour(Dark and Light), height(Short and Tall), gender(Male and Female) etc, caste(Brahmin, Kshatriya, Vaishya and Shudra) is an attribute of body itself.
Long answer :
First of all, many scriptures which speak against birth based caste system, often are not considered necessarily authentic. For Eg: Vajra Suchika Upanishad, speaks against caste system, may not be considered authentic Upanishad by many. It is attributed to Shankaraacharya in some 18th century manuscript. It even has some association with Buddhism.
The date as well as the author of Vajrasuchi Upanishad is unclear. The
Upanishad is attributed to Sankaracharya in the manuscripts discovered
by early 1800s
The relationship between the Vajrasuchi text of Buddhism and
Vajrasuchi Upanishad of Hinduism has long been of interest to
scholars.
Source
So the authenticity of this Upanishad is questionable, as also no accepted great Vedanta Acharya has written a commentry on this.
So to know the authentic opinion of Vedic Hinduism, we need to look at accepted authentic main stream Vedic Hinduism.
Vedic religion is understood by:
1. Vedas and Upanishads
2. Bhagavad Gita
3. Brahma Sutras and Acharyas works.
This is called Prasthana Traya, which is mainstream Vedic religion.
We can also use the following as supporting evidence.
4. Itihaasa (Mahabharata)
5. Smrutis
6. Puranas
This is Vedic Hinduism.
One important thing to note is body is birth related. So wherever birth or yoni is mentioned we can infer that it is related to body. A soul is never born and knowledge of the mind can change through learning and forgetting, richness and respect can change through money and society, but body is always linked to birth. Things like Race and Gender and Heridetary diseases and strengths are related to birth and to this list Caste is to be added as Caste is birth related. We can see that only bodily features are birth related, therefore Caste belongs to body.
- Chandogya Upanishad
तद्य इह रमणीयचरणा अभ्याशो ह यत्ते रमणीयां योनिमापद्येरन्ब्राह्मणयोनिं
वा क्षत्रिययोनिं वा वैश्ययोनिं वाथ य इह कपूयचरणा अभ्याशो ह यत्ते
कपूयां योनिमापद्येरञ्श्वयोनिं वा सूकरयोनिं वा चण्डालयोनिं वा ॥ ५.१०.७
॥
- Among them, those who did good work in this world [in their past life] attain a good birth accordingly. They are born as a brāhmin, a
kṣatriya, or a vaiśya. But those who did bad work in this world [in
their past life] attain a bad birth accordingly, being born as a dog,
a pig, or as a casteless person. - 5.10.7
- Bhagavad Gita
मां हि पार्थ व्यपाश्रित्य येऽपि स्यु: पापयोनय: | स्त्रियो
वैश्यास्तथा शूद्रास्तेऽपि यान्ति परां गतिम् ॥9.32॥
For, O son of
Prtha, even those who are born of sin-women, Vaisyas, as also
Sudras, even they reach the highest Goal by taking shelter under Me. - 9.32
- Brahma Sutras, Samanvaya Adhyaya, Apasudradhikaranam: Topic 9 (Sutras 34-38)
Relevant extracts from Adi Shankaracharya Bhashya.
A Sudra by birth cannot have Upanayana and other Samskaras without
which the Vedas cannot be studied. Hence the Sudras are not entitled
to the study of the Vedas.
- Mahabharata
केन वा कर्मणा विप्रः शूद्रयॊनौ परजायते कषत्रियः शूद्र ताम एति केन
वा कर्मणा विभॊ 13.131.4
Through what acts does a Brahmana take birth in his next life,
in the Sudra order? Through what acts, O puissant deity, does a
Kshatriya also descend to the status of Sudra
बराह्मण्यं देवि दुष्प्रापं निसर्गाद बराह्मणः शुभे कषत्रियॊ
वैश्यशूद्रौ वा निसर्गाद इति मे मतिः 13.131.6
The illustrious one said, 'The status of a Brahmana, O goddess, is
exceedingly difficult to attain. O auspicious lady, one becomes a
Brahmana through original creation or birth. After the same manner
the Kshatriya, the Vaisya, and the Sudra, all become so through
original creation. Even this is my opinion
Kisari Mohan Ganguli writes,
”Nisargat is literally through creation or original nature, or birth.
Of course, what is implied is that one becomes a Brahmana, or
Kshatriya, or Vaisya or Sudra, through original creation as
such, by the Self-born, that is, birth.”
Translation Reference - Book 13 Chapter 143
- Smritis
Manu Smriti - Uses the Jaati for caste, which literally means Birth
जातिमात्रोपजीवी वा कामं स्याद् ब्राह्मणब्रुवः । धर्मप्रवक्ता
नृपतेर्न शूद्रः कथं चन ?? ॥ 8.20 ॥
A Brahmana who is only a Brahman by decent, one has neither studied
nor performed any other act required by the Vedas may, at the king’s
pleasure, interpret the law to him, act as the judge, but never a
Shudra.”(however learned he may be). - 8.20
सर्ववर्णेषु तुल्यासु पत्नीष्वक्षतयोनिषु । आनुलोम्येन सम्भूता
जात्या ज्ञेयास्त एव ते ॥10.5॥
Among all castes, those only who are born of consorts wedded in the natural order, as virgins of equal status, are to be regarded as
the same (as their father)
Apastamba Dharma-sutra
1.1.1.4. (There are) four castes--Brāhmaṇas, Kṣatriyas, Vaiśyas, and Śūdras.
1.1.1.5. Amongst these, each preceding (caste) is superior by birth to the one following.
Yājñavalkya Smruti
Children born to men of a certain caste from women of the same caste,
are also of the same caste;—sons born of unblamable marriages are the
perpetuators of the race - 1.20
Vishnu Smriti
Sons [begotten] on women equal in caste [to their husbands] are
equal in caste [to their fathers]. [Sons begotten] on women of lower
castes, become of the caste of their mothers. Sons begotten on
women of higher castes are despised by the twice-born. - 16.1-3
- Puranas
Shiva Purana
O excellent sage, birth as a man is very difficult to obtain;
especially that as a Brahmin - Uma Samhita 44.13
Agni Purana
Since the binding of the girdle of the muñja grass (at the time of
investiture of the sacred thread), the brahmins and others (are
supposed to) take a second birth. One’s caste is decided from that of
the mother in the natural order of castes. 150.10
Contrary to the natural order the son of a brahmin woman through a
śūdra (is known as) Caṇḍāla, from a kṣatriya (as) sūta and from a
vaiśya (as) devala. 150.11
It is clear that, from all these sources that caste has birth as a necessary condition and not sufficient condition.
That is caste = Birth + Rituals + Behaviour. Even if one of it is missing a person will loose the caste. It is a necessary condition and not a sufficient condition.
Given that ritual and behaviour can be practiced and cultivated by anyone the necessary condition for caste becomes body and birth. Therefore caste is in body.