The answer depends on what you think is meant by caste. The answer is no if you think of caste as jati or community. The answer is yes if you think of caste as varna which should depend on conduct.
I am posting the relevant Gita verses on varna:
O great hero! The duties of Brahmanas, Ksatriyas, Vaisyas and also
Sudras have been divided according to the quality born of their own
nature. [Gita 18.41]
Serenity, control of the sense, austerity,
purity, straight-forwardness, knowledge, insight, and faith in the
Supreme Being - these are a Brahman's duties born of his own nature.
Prowess, splendor of personality, unfailing courage,
resourcefulness, dauntless in battle, generosity, leadership - these
are a Ksatriya's duties born of his specific nature. [Gita 18.43]
Agriculture, cattle-rearing and trade form the duty of the Vaisya
springing from his own nature, while the natural duty of a Sudra
consists in subordinate service under others. [Gita 18.44]
devoted to one's own natural duty, man attains to spiritual
competency. Now hear how devotion to one's own natural duty generates
spiritual competency. [Gita 18.45]
From whom all beings have
emanated and by whom all this universe is pervaded - by worshipping
Him through the dedicated performance of one's duty, man attains to
spiritual competency. [Gita 18.46]
One's own duty, even if without
excellence (i.e. inferior in the scale of worldy values)is more
meritorious spiritually than the apparently well-performed duty of
another. For no sin is incurred by one doing works ordained according
to one's nature. [Gita 18.47]
These verses do not support the last name based hereditary Hindu caste system which led to division in Hindu society. It is a curious fact that the ancient Acharyas ignored ancient interpretations of varna/caste given in the Mahabharata. There have been many points of view regarding the Varna-jati system apart from the one practiced by Hindus. Some of these proposals are:
Yudhisthira said, " In human society, O mighty and highly
intelligent serpent, it is difficult to ascertain one's caste, because
of promiscuous intercourse among the four orders. This is my opinion.
Men belonging to all orders (promiscuously) begat offspring upon women
of all the orders. And of men, speech, sexual intercourse, birth and
death are common. And to this the Rishis have borne testimony by using
as the beginning of a sacrifice such expressions as -- of what caste
server may be, we celebrate the sacrifice. Therefore, those that are
wise have asserted that CHARACTER IS THE CHIEF ESSENTIAL REQUISITE.
.... WHATSOEVER NOW CONFORMS TO THE RULES OF PURE AND VIRTUOUS CONDUCT,
HIM HAVE I, ERE NOW, DESIGNATED AS A BRAHMANA.' (Mahabharata Aranya
Parva Section CLXXIX)
The sage Bhrigu said, ‘There is really no distinction between the
different orders. The whole world at first consisted of Brahmanas.
Created (equal) by Brahman, men have, in consequence of their acts,
become distributed into different orders. They that became fond of
indulging in desire and enjoying pleasures, possessed of the
attributes of severity and wrath, endued with courage, and unmindful
of the duties of piety and worship, - these Brahmanas possessing the
attributes of passion, - became Kshatriyas. Those Brahmanas again who,
without attending to the duties laid down for them, became possessed
of both the attributes of Goodness and Passion, and took to the
professions of cattle-rearing and agriculture, became Vaisyas. Those
Brahmanas again that became fond of untruth and injuring other
creatures, possessed of cupidity, - engaged in all kinds of acts for a
living, and fallen away from the purity of behaviour, and thus wedded
to the attributes of Darkness, became Sudras. Separated by these
occupations, Brahmanas, falling away from their order, became members
of the other three orders. All the four orders, therefore, have always
the right to the performance of all pious duties and of sacrifices.
Even thus were the four orders at first created equal by Brahman who
ordained for all of them (the observances disclosed in) the words of
Brahma (in the Vedas). Through cupidity alone, many fell away, and
became possessed by ignorance. The Brahmanas are always devoted to the
scripture on Brahma; and mindful of vows and restraints, are capable
of grasping the conception of Brahma. Their penances therefore never
go for nothing. They amongst them are not Brahmanas that are incapable
of understanding that every created thing is Supreme Brahma. (Mahabharata Santi Parva Section CLXXXVIII )
The famous Rishi Yajnavalkya continued,’ ..All orders of men are
Brahmanas. All are sprung from Brahma. All men utter Brahma. Aided by
an understanding that is derived from and directed to Brahma. I
inculcated this science teaching of Prakriti and Purusha. Indeed this
whole universe is Brahma. From the mouth of Brahma sprung the
Brahmanas; from his arms, sprung the Kshatriyas; from his navel, the
Vaisysa; and from his feet, the Sudras. All the orders, (having sprung
in this way) should not be regarded as pilfering from one another.
Impelled by ignorance, all men meet with death and attain, O King, to
birth that is the cause of acts. Divested of knowledge, all orders of
men, dragged in terrible ignorance, fall into varied orders of being
due to the principles that flow from Prakriti. For this reason, all
should, by every means, seek to acquire knowledge. I have told thee
that every person is entitled to strive for its acquisition. One that
is possessed of knowledge is a Brahmana. Others, (viz., Kshatriyas,
and Vaishyas and Sudras) are possessed of knowledge. Hence this
science of emancipation is always open to them all. (Mahabharata Santi Parva Section CCCXIX)
From these 3 views and that of the Gita what comes across is that the Hindu view is that all humans are not equal in conduct. There are good people and there are bad people. That is a fact of life. However, they should all be treated equally and given equal opportunity to improve themselves precisely because they are all supreme Brahma. Where Hindus have gone wrong with their practice is to freeze an entire jati or community into a single Varna. That is the nonsensical thing about Hindu practice. The Varna system is for an individual and all individuals have to be respected and given equal opportunity to improve themselves even though there are real differences between individuals. The idea that all humans are equal is clearly nonsense (then why should we celebrate Einstein or Rabindranath specially?) but all humans should be treated with an equal eye as suggested by the Gita.
The varna idea applies even to non-Hindus. Some people accuse the Rig Veda of casteism because of its conception of different orders of humans as emerging from different parts of Brahman. A Mahabharata passage from Anusasana Parva Section CXLIII even rejects such an idea:
Maheshwara said, ‘..Neither birth, nor the purificatory rites, nor
learning, nor offspring, can be regarded as grounds for conferring
upon one the regenerate status. Verily, the conduct is the only
ground. All Brahmanas in this world are Brahmanas in consequence of
conduct. A Sudra, if he is established on good conduct, is regarded as
possessed of the status of a Brahmana. The status of a Brahma, O
auspicious lady [Uma], is equal wherever it exists. Even this is my
opinion. He, indeed, is a Brahmana in which the status of Brahma
exists – that condition which is bereft of attributes and which has no
stain attached to it. of human beings in four orders dependent on birth is only for purposes of classification.The boon giving Brahma, when he created all creatures, himself said that the distribution of human beings in four orders dependent on birth is only for purposes of classification.'
Varna system was flexible and changing of Varna was allowed.
Change of Varna
Narada said, 'If in members born in a certain Varna the qualities
pertaining to another Varna are seen, they (the former) are to be
classified as belonging to the latter Varna.'
Srimad Bhagavata Purana VII.11.35
Actual Examples of Change of Varna
I am supplementing my answer with actual examples of persons changing Varna.
Bhagavan Rishabha, realising that the region of his advent was a place
dominated by Vedic rituals, adopted the life of a religious student
under a teacher with gifts, came back home with his blessings. He
adopted the householder’s station of life in order to teach the world
about the duties of that order, observed all the ceremonials and
duties laid down in the scriptures, married a girl named Jayanti given
to him by Indra, and begot by her a hundred sons equal to himself in
all respects. Of all these sons, Bharata was the eldest and noblest.
This Ajanabha Varsha came to be known after him as Bharatavarsha. Next
to him, the eldest nine other sons ….. were elder to the remaining
ninety. Among these ninety, another nine … became great devotees and
teachers of the Bhakti cult. ….. The remaining eighty one of the
brothers, who were humble in nature, learned in the Vedas, adepts in
sacrificial rites, and extremely pure through their observances,
became Brahmanas according to their father’s instruction.
Srimad Bhagavata Purana, V.4.8-13
Several sons of King Rishava changed their Varna under the instruction of their father.