I presume you are asking the difference between Varna (वर्ण)system and Jaati (जाति). The concept known as a caste system is an anglicised explanation of the Indian concepts of Varna (वर्ण) and Jaati (जाति).
About Varna:
In ancient India there was a concept of Varnashrama (वर्णाश्रम) wherein, besides 4 Ashramas, there were 4 Varnas or a primary broad classification based on occupations and attributes and each Varna had the duty of serving all the Varnas above it.
Lord Krishna says in the Srimad Bhagavadgita:
चातुर्वर्ण्यं मया सृष्टं गुणकर्मविभागशः। (first half of verse 4.13)
The four varnas have been created by me, being divided based on attributes (guna) and occupation (karma)
The four varnas and their duties can be described as follows:-
- Brahmins: the priests who studied and preached the Vedas and helped others attain spiritual objectives.
Peacefulness, self-control, austerity, purity, tolerance, honesty, knowledge, wisdom and religiousness – these are the natural qualities by which the brāhmaṇas work. BG 18.42
That Brahmin who always perform his ablutions, who always wears his sacred thread, who always studies of the Vedas, who donates food to the lowly, who tells the truth and serves his guru, never falls off from Brahmaloka. Mbh Udyoga Parva 40.25
- Kshatriyas: the ruling, martial people who took up arms for protecting the upper varna i.e. the Brahmins and ultimately the Vedas.
Heroism, power, determination, resourcefulness, courage in battle, generosity and leadership are the natural qualities of work for the kṣatriyas. BG 18.43
Having studied the Vedas, performed sacrifices, and protected subjects, a Kshatriya who dies in battle protecting cattle and Brahmins attains heaven, being purified by his weapons. Mbh Udyoga Parva 40.26
- Vaishyas: indulged in commerce and agriculture, to serve the upper two varnas with his wealth i.e. provide finance for upholding the Vedas.
Farming, cow protection and business are the natural work for the vaiśyas. BG 18.44
Having studied the Vedas, distributing timely, his wealth among Brahmins, Kshatriyas, and dependents, and inhaled the sanctified smoke of the three kinds of fires, the Vaisya enjoys heavenly bliss after death.Mbh Udyoga Parva 40.27
- Shudras: engaged in various vocational activities, such as barbers, weavers, etc. which result in service to upper three varnas.
and for the śūdras there is service to others. BG 18.44
Having properly served and gratified Brahmins, Kshatriyas, and Vaisayas in due orde, and having burnt his sins, and then peacefully casting off his body, the Sudra enjoysbthe bliss of heaven. Mbh Udyoga Parva 40.28
(Ref.: Chapter 40 of the Udyōga Parva of the Mahabharata and Chapter 18 of Srimad Bhagavadgita)
As per Yaska’s Nirukta 2.1.4 - the meaning of Varna is given as:
वर्णो वृणोतेः।
Varna is that which is chosen.
The basic meaning of the word Varna stems from the Sanskrit root “वृ” which means ‘to choose’ and the word ‘Varna’ means that which is chosen. Thus that which was chosen (based on a person’s attributes/karma) was called Varna. A person could therefore be the son of a Brahmin but choose his Varna as a Vaishya, engaged in commerce, depending on his attributes. As the Manusmriti 10.65 states:
The Śūdra attains the position of the Brāhmaṇa and the Brāhmaṇa sinks to the position of the Śūdra; the same should be understood to be the case with the offspring of the Kṣatriya or of the vaiśya.
It is pertinent to note as marriage within the same Varna was advised as per dharmashastras, the children born of wedlock had the same guna/ karma as per their parents and would therefore belong to the same Varna as their parents (birth based). We have Brahmins being children of Brahmins only generally. So though primarily north based, yet as per the above there was flexibility and no hard and fast rule. If by chance, someone showed different guna/ karma than his birth Varna there were scriptural prescriptions as above for Varna migration or choosing a new Varna.
Refer: Any references where varna/caste is determined independent of birth/parentage?
The concept of Jaati.
Jaati is linked to birth. It stems from the Sanskrit route जन् that is ‘to be born’. The word was used initially in the sense of a race or creatures (not only humans) having similar characteristics and genetic makeup i.e. belonging to the Yoni.
Hence you have - Manushya-Jaati (मनुष्यजाति) or the human race, Vaanara-Jaati (वानरजाति) or the monkey race, Asura-Jaati (असुरजाति) or the demon race, etc. By this logic, initially all humans thus belonged to the same Jaati i.e. they were born in the same Yoni (मनुष्ययोनि) and hence were the Manushya-Jaati. Also a family lineage could be considered as Jaati.
With time, Varna became based on Jaati, i.e. birth based. Though even earlier and later (after the concept changed) children displayed the same Guna karma as their parents, the flexibility to change based on Guna karma was done away with.
A - a Vaishya
B (son of A) - his birth or Jaati is Vaishya birth or Vaishya Jaati. Since he is born of a Vaishya his occupation also becomes that of a Vaishya as per the distorted concept. He has to “choose” (वरणम्) the occupation his father “chose”. Thus jaati and Varna began to be used synonymously.
Since the English applied the word caste to the Hindu system and distorted the original concept, the three words began to be used interchangeably. Though it would be more appropriate to associate caste with Jaati (the new concept of professional birth and not the old concept of merely a family lineage or same yoni).
(A better understanding of the distortion mechanism can be obtained here)
