Short answer to your question is yes caste is hereditary as per Hindu scriptures whether Veidic or itihasas.
I) FROM VEIDIC LITERATURE:
Chhandogya Upanishad 5.10.7 - Those whose conduct has been good here will shortly get birth such as a Brahmana (brahmana yonim), a kshatriya, or a vaishya. But those whose conduct has been evil will be born in evil births shortly such as the birth of a dog (shva yonim), or as a pig, or a chandala.
The word used is "yonim", and it means birth or womb, and is used to refer to birth as a dog (shva), and pig, and since it's also used along with the names of castes, it must be taken to mean that one is born into those castes just like animals, so it doesn't merely mean being born as a person with Brahmanical qualities. It means being born into the womb (yoni) of a Brahmin, and a person born into such a womb naturally acquires Brahminical qualities due to gene transfer.
Maitrayani Samhita 1.4.1 - We know not if we are brahmins or non-Brahmins. ...Therefore, when recounting our gotra-pravaras, say that the devas are our fathers.
This verse from the Maitrayani shakha of the Krishna Yajur Veda is referred to in Jaimini's Mimamsa Sutras, and commented on by Shabara swami. The sutra reads:
On account of the failings of women, (there can be no certainty regarding one's caste); specially as the son belongs to the progenitor.
This sutra is based on the Maitrayani samhita verse. Shabara comments on the sutra as follows:
The meaning of the eulogistic passage is that even a non-Brahmana would become a Brahmana by the recounting of his pravaras [meaning a non-Brahmana can claim he is a Brahmana by recounting Brahmanical gotra pravaras, and hence it is necessary for the Brahmana to also recount his pravaras, as one can never be sure of one's Brahmanahood]. It is difficult to know if one is really a Brahmana; - and this is what is figuratively spoken of as 'we do not know'; and the difficulty in knowing it for certain is due to the 'failings of women'.
This vedic verse shows that caste is determined by gotra pravaras (ancestral lineages), and hence caste is inherited and based on birth.
Before I begin my next point it must be noted that in the Krishna Yajur Veda Taitreya Samhita Khand 2 Prapatakha 10 Anuvak slok 2 states clearly whatever is stated in Manusmriti must be strictly followed.
II) FROM SMRITI AND ITIHASAS:
The Bhagvad Gita is a part of the Mahabharata Bhisma Parva, in the very first chapter of the Bhagvad Gita in verse 41 Arjun talks about Varnasankar and states they are born when women become immoral (the concept of Varnasankar has been ordained in the Manusmriti Chapter 10 clearly as children born out of union between parents belonging to different castes and here castes have been clearly stated to be Bramhan Kshatriya Vaishya and Shudra). Shri Ramanujacharya while commenting on Bhagvad Gita 18 Chapter verse 41 states clearly that Varna is birth based. It is also stated in Mahabharata Adi Parva Chapter 296 that children born to rishis always take up Varna of their father alone. The most distinct example of caste being hereditary is in the Bhagvad Gita Chapter 9 verse 32 which clearly states those who are born in pap yoni such as the Vaishya and Shudra can be purified through worship of the lord which clearly implies caste or Varna is birth based.
Ps: it is my first time that I am writing an answer here hope if someone finds any violation of site rules they shall guide me.
JAI VEERBHADRA JAI SHANKARACHARYA
Edit: I found someone in the comments section trying to misappropriate the verses of the Vajrasuchika Upanishad to suit their own theories so I am making this edit.
There are a total of 9 mantras in the Vajrasuchika Upanishad. Knowledgeable People.
The first mantra talks about the introduction of the Upanishad, The Upanishad is described as "qui ज्ञानहीनानां भूषणं ज्ञानचक्षुषाम्" That it is the contamination of the knowledge-less people and ornament of the knowledgeable people. Every student of Vedanta knows that the knower of the Vedas is also ignorant when viewed from the point of view of Brahmavidya. Similarly, in sanskrit terms, the Jnani of all languages except Sanskrit is also ignorant. That is why this upanishad is said to be the ornament of Knowledgeable Ones, others can even distort it, This is already indicated in the first Mantra.
In the second mantra, It is said, "Brahmins, Kshatriyas, Vaishyas, Shudras are the four varnas In these varnas, the Brahmin is the chief (ब्राह्मक्षत्रियवैष्यशूद्रा.... ब्राह्मण एव ч). Which is described in the Smritis in accordance with to the Vedas (इति वेदवचनानुरूपं स्मृतिभिरप्युक्तम्). The question arises: Who is the Brahmin? Is that an Jeeva? Or is there a body? Or Jaati or Karma or Jnana or Dharmik?
Further mantras in the Upanishads: In 6 mantras from 3 to 8, these six mantras say, Jeeva, Deha, Jaati, Jnana, Karma and Dharmik separately and say that the Jeeva is not a Brahmin, the Deha is not a Brahmin, the Jaati is not a Brahmin, Jnana is not a Brahmin, karma is not a Brahmin, Dharmika is not Brahmin. A few examples of given of Rishis who were Animal born. But not at a single place, they did not say that they are not Brahmins, either by combining the six or by combining any two of them. For example, The Screen is not Smartphone, Battery is not Smart Phone, Microphone is not Smartphone, Software is not Microphone, Semi-Conductor is not Smartphone. Similarly, all of them are not Brahmin if put separately.
Now let's see the definition of Brahmin mentioned in mantra 9 of this Upanishad:
Who is not full of duality of the soul, not even with Jati, Virtue and action, free from all defects. Satya, Jnana, Who always is in आनंद स्वरूप, स्वयं निर्विकल्प । Shruti, Smriti, Purana conclude that he is the Brahmin who is free from defects like lust, anger, hatred, etc., who is grateful, endowed with sham-dum, devoid of emotions like Matsarya, trishna, asha, moha, etc., is the one who keeps the mind completely separate from the faults like dambha, ego, etc. No other Brahmanatva can be proved in any other way. The Upanishads says that the brahmin is the one who is in this Bhava that He is Satchitanand Brahm"
The usage of Brahmin here, The word Brahmin is used here for the person in Brahmabhava.
It is said that,
"वज्रसूच्यां तु ब्रह्मज्ञानमूलकत्वमेव ब्राह्मण्यमुक्तम्। नात्र वर्णव्यवस्थायाः कर्मप्रयोजकं ब्राह्मण्यस्य वा प्रतिपादनम् । अत एव ब्राह्मणस्यैव विचारस्तत्र दृश्यते न क्षत्रियादीनाम्। वर्णविचारे तु तेषामपि विचारेणावश्यम् भवितव्यम्।"
The Vajrasuchi talks about Brahmanatva from the point of Brahmagyan. There the Brahmin Varna which is bound to Karma as propounded in the Varna Vyavastha is not talked about. That is why the long description is only about the word Brahmin, not about kshatriyas, etc. If there was a Varna Specific Vaishya, there must have been an idea about Kshatriyas etc.
Firstly, if the person in Brahmabhava is called a Brahmin (in context of varna), will the description of the origin of the Brahmin character from the mouth of Virata in the Purushsukta not become false? Is it possible to produce such a Varna by creating a Varna out of the one who has attained Kaivalya?
Is it possible through maya to take a birth for the one who has attained kaivalya from the doors of PanchaBhutas? Then the knowledge of the Vedas will prove to be false. If that were to be possible, then entire JnanaKand of Vedas would be falsified. This is not possible, so it is ignorance to consider this adjective used for the one who's in brahmabhava to be a Brahmin (varna Brahmana).
If we don't take the Brahmabhava meaning here, The contradiction with Chhandogya 5.10.7 and Brihadaranyaka 1.4 would occur. Only a fool would do that and conclude with contradiction.
It must be noted that at starting it is said that brahmins are one varna in four varna and that he is the best. Then further the questioning arises, "Is the living being a Brahmin?" Or is the body a Brahmin? Or is the caste brahmin? Or is knowledge a Brahmin? Or is karma a Brahmin? Or is Dharmik a Brahmin?
Then Jati, karma and Dharmik were said not to be Brahmins. In the definition of Brahmin, what is not associated with Jati, virtue and action was called Brahmin. In this condition What would happen to the Vachana of Bhagwan चातुर्वर्ण्यं मया सृष्टं गुणकर्मविभागशः।" (गीता 4.13) Guna and Karma is used here. It clearly talks about the creation of four varnas, brahmins, etc., with guna and karma.
If you consider this Brahmin word to be a Brahmin (varna) and if you refute the Brahmin (varna) by birth from the fifth mantra, then according to the seventh mantra, the Karmana (by karma) Brahmin (varna) will also be refuted. Then neither Will be one Brahmana by Karma nor Janma. Then How will one interpret it?
In such a situation, if this Brahmin is described as a Varna of Varna Vyavastha, it will result in a non pervading defect (³), because the varnaashram, which applies in both the ways of Pravritti and Nivritti, Their parts will only be true in Nivritti Marga, not in Pravritti.
If the Brahmagyani can only be of Brahmin varna, why was Vidurji Shudra, Dharmavaidha Antyaj, Janakji, Sri Krishna and Shriram Kshatriya, Tuladhar Vaishya, Why did the Scriptures not call him brahmin varna? Anyone can be Brahmgyani, whether it is a woman or a man, a Brahmin or a Chandal.
There are accounts of the pre-birth and rebirth of many Brahmins in the Itihas-Purana, and many Brahmins went to Swarga etc., after death. If Brahm gyan-related Brahmanatva is considered to be a Brahmin of varnaashram, how is rebirth of all of the Brahmagyani who attained advaita bhava in the above way possible? Then consider it opposed to Vedic Gyan Kand and Prasthanatrayi.
That is why Clearly, the word Brahmin here, is not of Varna Vyavastha but is Brahmgyan-representative. In fact, this Vajrasuchika Upanishad is a sannyasa upanishad. The tendency is not related to the Pravritti path, it should have a Nivritti meaning, not a Pravritti related.