What are the order & duties of the children born from intermixing/hybridisation of 4 classes?

Question

There are 4 classes in descending order:

1. BrAhmaNa
2. Kshatriya
3. Vaishya
4. Shudra

If a man of class X mates with a woman of class X, then definitely the child will inherit the class X.

How the intermixing of 2 classes (X & Y) works with respect to the child's class & duties?

Note: caste-system is not appropriate for this Qn, as we are discussing about "classes" and not the "castes". See this.

Update: Some people may get offended due to wrong tagging in our site, so I will make a clarification here. There is a big difference between "caste-system" (tag available) and "class-system" (tag rejected). See this answer for a detailed reference.
In short, if we consider "Bill Gates" and "Bill Clinton" as head of their family trees, then "Gates" & "Clinton" are castes, however "businessman" and "politician" are classes. Some may find former as racist, however this Qn intends to discuss about the latter.

Answer 1

First of all, the sons who are born out of proscribed marriages will not belong to any of the 4 castes (varna-s). As there is no 5th caste, they all will belong to the unknown caste or the out-cast. But they have different names so that they can be differentiated.

10.4. Brahmana, the Kshatriya, and the Vaisya castes (varna) are the twice-born ones, but the fourth, the Sudra, has one birth only; there is no fifth (caste).

Secondly, this topic is already dealt with in great detail in the Manu Smriti. See the following collection of verses:

10.8. From a Brahmana a with the daughter of a Vaisya is born (a son) called an Ambashtha, with the daughter of a sudra a Nishada, who is also called Parasava.

10.9. From a Kshatriya and the daughter of a Sudra springs a being, called Ugra, resembling both a Kshatriya and a Sudra, ferocious in his manners, and delighting in cruelty

10.10. Children of a Brahmana by (women of) the three (lower) castes, of a Kshatriya by (wives of) the two (lower) castes, and of a Vaisya by (a wife of) the one caste (below him) are all six called base-born (apasada).

10.11. From a Kshatriya by the daughter of a Brahmana is born (a son called) according to his caste (gati) a Suta; from a Vaisya by females of the royal and the Brahmana (castes) spring a Magadha and a Vaideha.

10.12. From a Sudra are born an Ayogava, a Kshattri, and a Kandala, the lowest of men, by Vaisya, Kshatriya, and Brahmana) females, (sons who owe their origin to) a confusion of the castes.

10.13. As an Ambashtha and an Ugra, (begotten) in the direct order on (women) one degree lower (than their husbands) are declared (to be), even so are a Kshattri and a Vaidehaka, though they were born in the inverse order of the castes (from mothers one degree higher than the fathers).

10.14. Those sons of the twice-born, begotten on wives of the next lower castes, who have been enumerated in due order, they call by the name Anantaras (belonging to the next lower caste), on account of the blemish (inherent) in their mothers.

10.15. A Brahmana begets on the daughter of an Ugra an Avrita, on the daughter of an Ambashtha an Abhira, but on a female of the Ayogava (caste) a Dhigvana.

10.16. From a Sudra spring in the inverse order (by females of the higher castes) three base-born (sons, apasada), an Ayogava, a Kshattri, and a Kandala, the lowest of men;

10.17. From a Vaisya are born in the inverse order of the castes a Magadha and a Vaideha, but from a Kshatriya a Suta only; these are three other base-born ones (apasada).

10.18. The son of a Nishada by a Sudra female becomes a Pukkasa by caste (jati), but the son of a Sudra by a Nishada female is declared to be a Kukkutaka.

10.19. Moreover, the son of by Kshattri by an Ugra female is called a Svapaka; but one begotten by a Vaidehaka on an Ambashtha female is named a Vena.

Some occupations are mentioned in the following verses:

10.32. A Dasyu begets on an Ayogava (woman) a Sairandhra, who is skilled in adorning and attending (his master), who, (though) not a slave, lives like a slave, (or) subsists by snaring (animals).

10.33. A Vaideha produces (with the same) a sweet-voiced Maitreyaka, who, ringing a bell at the appearance of dawn, continually. praises (great) men.

10.34. A Nishada begets (on the same) a Margava (or) Dasa, who subsists by working as a boatman, (and) whom the inhabitants of Aryavarta call a Kaivarta.

10.35. Those three base-born ones are severally begot on Ayogava women, who wear the clothes of the dead, are wicked, and eat reprehensible food.

In general, for all practical purposes, they can be treated as a Sudra. So, they can betake the jobs which are prescribed for the Sudras in the scriptures.

10.41. Six sons, begotten (by Aryans) on women of equal and the next lower castes (Anantara), have the duties of twice-born men; but all those born in consequence of a violation (of the law) are, as regards their duties, equal to Sudras.

All verses are from the Manu Smriti.

Answer 2

Note: Actually it's better, if I make a chart of all the intermixed children names & duties. However, I am posting this answer to make a reference elsewhere. Later depending on time & interest, I will be creating a chart and posting here.

The names in "bold" are the different castes.

---

From AnusAsana Parva:

Yudhishthira said, 'Through inducements offered by wealth, or through mere lust, or through ignorance of the true order of birth (of both males and females), or through folly, intermixture happens of the several order What, O grandsire, are the duties of persons that are born in the mixed classes and what are the acts laid down for them?

---

Bhishma said, 'In the beginning, the Lord of all creatures created the four orders and laid down their respective acts or duties, for the sake of sacrifice.
The Brahmana may take four wives, one from each of the four orders. In two of them (viz., the wife taken from his own order and that taken from the one next below), he takes birth himself (the children begotten upon them being regarded as invested with the same status as his own). Those sons, however, that are begotten by him on the two spouses that belong to the next two orders (viz., Vaisya and Sudra), are inferior, their status being determined not by that of their father but by that of their mothers. The son that is begotten by a Brahmana upon a Sudra wife is called Parasara, implying one born of a corpse, for the Sudra woman's body is as inauspicious as a corpse. He should serve the persons of his (father's) race. Indeed, it is not proper for him to give up the duty of service that has been laid down for him. Adopting all means in his power, he should uphold the burden of his family. Even if he happens to be elder in age, he should still dutifully serve the other children of his father who may be younger to him in years, and bestow upon them whatever he may succeed in earning.

A Kshatriya may take three wives. In two of them (viz., the one taken from his own order and the other that is taken from the order immediately below), he takes birth himself (so that those children are invested with the status of his own order). His third wife being of the Sudra order is regarded as very inferior. The son that he begets upon her comes to be called as an Ugra.

The Vaisya may take two spouses. In both of them (viz., the one taken from his own order, and the other from the lowest of the four pure orders), he takes birth himself (so that those children become invested with the status of his own order).

The Sudra can take only one wife, viz., she that is taken from his own order. The son begotten by him upon her becomes a Sudra.

---

A son that takes birth under circumstances other than those mentioned above, comes to be looked upon as a very inferior one If a person of a lower order begets a son upon a woman of a superior order, such a son is regarded as outside the pale of the four pure orders. Indeed, such a son becomes on object of censure with the four principal orders.

---

If a Kshatriya begets a son upon a Brahmana woman, such a son, without being included in any of the four pure orders, comes to be regarded as a Suta The duties of a Suta are all connected with the reciting of eulogies and encomiums of kings and other great men.

The son begotten by a Vaisya upon a woman of the Brahmana order comes to be regarded as a Vaidehaka. The duties assigned to him are the charge of bars and bolts for protecting the privacy of women of respectable households. Such sons have no cleansing rites laid down for them.

If a Sudra unites with a woman belonging to the foremost of the four orders, the son that is begotten is called a Chandala. Endued with a fierce disposition, he must live in the outskirts of cities and towns and the duty assigned to him is that of the public executioner. Such sons are always regarded as wretches of their race. These, O foremost of intelligent persons, are the offspring of intermixed orders.

The son begotten by a Vaisya upon a Kshatriya woman becomes a Vandi or Magadha. The duties assigned to him are eloquent recitations of praise.

The son begotten through transgression, by a Sudra upon a Kshatriya women, becomes a Nishada and the duties assigned to him have reference to the catching of fish.

If a Sudra happens to have intercourse with a Vaisya woman, the son begotten upon her comes to be called Ayogava. The duty assigned to such a person are those of a Takshan (carpenter). They that are Brahmanas should never accept gifts from such a person. They are not entitled to possess any kind of wealth.

---

Persons belonging to the mixed castes beget upon spouses taken from their own castes children invested with the status that is their own. When they beget children in women taken from castes that are inferior to theirs, such children become inferior to their fathers, for they become invested with the status that belongs to their mothers Thus as regards the four pure orders, persons beget children invested with their own status upon spouses taken from their own orders as also upon them that are taken from the orders immediately below their own. When, however, offspring are begotten upon other spouses, they come to be regarded as invested with a status that is, principally, outside the pale of the four pure orders. When such children beget sons in women taken from their own classes, those sons take the status of their sires. It is only when they take spouse from castes other than their own, that the children they beget become invested with inferior status. As an example of this it may be said that a Sudra begets upon a woman belonging to the most superior order a son that is outside the pale of the four orders (for such a son comes to be regarded as a Chandala who is much inferior). The son that is outside the pale of the four orders by uniting with women belonging to the four principal orders, begets offspring that are further degraded in point of status. From those outside the pale of the four orders and those again that are further outside that pale, children multiply in consequence of the union of persons with women of classes superior to their own. In this way, from persons of inferior status classes spring up, altogether fifteen in number, that are equally low or still lower in status. It is only from sexual union of women with persons who should not have such union with them that mixed classes spring up. Among the classes that are thus outside the pale of the four principal or pure orders, children are begotten upon women belonging to the class called Sairindhri by men of the class called Magadha. The occupation of such offspring is the adornment of the bodies of kinds and others. They are well-acquainted with the preparation of unguents, the making of wreaths, and the manufacture of articles used for the decoration of the person. Though free by the status that attaches to them by birth, they should yet lead a life of service. From the union of Magadhas of a certain class with women of the caste called Sairindhri, there springs up another caste called Ayogava. Their occupation consists in the making of nets (for catching fish and fowl and animals of the chase). Vaidehas, by uniting themselves with women of the Sairindhri caste, beget children called Maireyakas whose occupation consists in the manufacture of wines and spirits. From the Nishadas spring a caste called Madgura and another known by the name of Dasas whose occupation consists in plying boats. From the Chandala springs a race called Swapaka whose occupation consists in keeping guard over the dead. The women of the Magadhi caste, by union with these four castes of wicked dispositions produce four others who live by practising deceit. These are Mansa, Swadukara, Kshaudra, and Saugandha. From the Vaideha springs up a cruel and sinful caste that lives by practising deception. From the Nishadas again springs up the Madranabha caste whose members are seen to ride on cars drawn by asses. From the Chandalas springs up the caste called Pukkasa whose members are seen to eat the flesh of asses, horses and elephants. These cover themselves with the garments obtained by stripping human corpses. They are again seen to eat from broken earthenware These three castes of very low status are born of women of the Ayogava caste (by fathers taken from different castes). The caste called Kshudra springs from the Vaidehaka. The caste called Andhra which takes up its residence in the outskirts of towns and cities, also springs up (from the Vaidehakas). Then again the Charmakara, uniting himself with a woman of Nishada caste, begets the class called Karavara. From the Chandala again springs up the caste known by the name of Pandusaupaka whose occupation consists in making baskets and other things with cleft bamboos. From the union of the Nishada with a woman of the Vaidehi caste springs one who is called by the name of Ahindaka. The Chandala begets upon a Saupaka woman, a son that does not differ from the Chandala in status or occupation. A Nishada woman, by union with a Chandala, brings forth a son who lives in the outskirts of villages and towns. Indeed, the members of such a caste live in crematoria and are regarded by the very lowest orders as incapable of being numbered among them. Thus to these mixed castes spring up from improper and sinful union of fathers and mothers belonging to different castes. Whether they live in concealment or openly, they should be known by their occupations. The duties have been laid down in the scriptures for only the four principal orders. As regards the others the scriptures are entirely silent. Among all the orders, the members of those castes that have no duties assigned to them by the scriptures, need have no fears as to what they do (to earn their livelihood).

Overall it's recommended Not to let the marriage happen among unauthorised combination for the good of society.

Answer 3

In the 'History of Dharmaśastras', P.V Kane gives a definite list of varnasaṅkaras & points out to the fact that scriptures disagree on the origins of most of them. Here is the list & their profession

Name	Occupation
Aṭṭalikākāra	Builder
Andhra
Ambaṣṭha	Physician
Avarāṭa
Ayaskāra	Blacksmith
Abhīra	Herdsman
Āghasika	Dealer of cooked food
Āpīta
Āśapa
Āyogava	Woodcutter, weaver, manufacture of bronze-made products, farmer, acting, masonry, whitewashing
Ārdhika	Agricultural labourer
Āvṛta
Āśvika	Trader of horses
Āhituṇḍika	Snake-charmer
Āhiṇḍika	Guarding jails
Ugra	Staff-bearer, inflictor of punishments, killing animals that live underground
Upakruṣṭa	Carpenter
Oḍra
Aurabhra	Manufacturer of woollen products
Kaṁsakāra	Manufacturer of brass-made products
Kaṁsavaṇik	Dealer of brass-made products
Kaṭakāra	Manufacturer of straw-mattresses
Kapālin	Manufacturer of jute-made products
Karaṇa	Record-keeper
Kākavacha	Fodder-dealer
Kāmboja
Kārāvara	Bearer of torches & umbrellas
Kirāta
Kukunda
Kukkuṭaka	Organiser of cock-fights, sword-manufacturer
Kuvacha
Kuvinda	Weaver
Kulāla	Potter
Kuśilava	Royal bard
Kaivarta	Fisherman & boatman
Koṭika	Thatcher
Kolaka	Disposer of dead bodies
Kouali
Kṣudraka
Khanaka	Digger
Khasa
Gandhika	Dealer of spices & perfumes
Guhaka
Gopāla	Milkman
Ghaṭṭajivi	Boatman
Gholika	Rat-catcher
Chakrin	Dealer of oil & salt
Chaṇḍāla	Maintaining śmaśānas
Charmakāra	Leatherwork
Chākrika	Oil manufacturing & ringing bells
Chitrakāra	Painting
Chīna
Chunchu	Hunter
Chuchuka	Dealer of tāmbula
Jālika	Manufacturer of nets
Jhalla
Domba	Disposal of cremated human remains
Tivara	Hunting, selling wild animals & fishing
Tāmropajīvin	Manufacturer of copper-made products
Tailika	Oilpresser
Darada
Dāsa	Labourer
Durbhara	Tanner
Dolavāhin	Palaquin-bearer
Dausyanta	Watchman
Draviṛa
Dhigvaṇa	Dealer of animal-skins, cobbler
Naṭa	Actor
Nartaka	Dancer
Nāpita	Barber
Niṣada
Pahlava
Pārada
Pāraśava	Drummer
Piṅgala
Pukkasa	Hunting, killing animals that live underground, manufacturing & selling drugs & alcohol
Pulinda	Hunter
Puspadha
Pauṇḍraka	Manufacturing jaggery
Plava	Dealer of animal bones
Bhāṭa	Bard
Bhaḍa
Bhilla
Bhūpa
Bhoja
Maṇikāra	Dealer of precious stones
Madgu	Hunter
Madranābha	Donkey-keeper
Malegrahin	Sweepers
Malla
Matsyabandhaka	Dealer of fish
Māṁsacchedin	Butcher
Māgadha	Carrying messages, royal flatterer, singer
Mātara
Madhuka	Manufacturer of alcohol
Manyu	Law enforcement
Mālakāra	Dealer of flowers
Māhiśya	Cultivation of crops
Mūrdhavasikta	Astrologer, exorcist
Meda	Sweeper
Modaka	Manufacturer of sweetmeat
Rañjaka	Dyer
Rajaka	Washerman
Rathakāra	Manufacturer of chariots
Rājaputra
Romika	Manufacturer of salt
Varuḍa	Manufacturer of bamboo-made products
Vāgatīta	Bodyguards, henchmen
Vādaka
Varujīvin	Betel-leaf cultivator
Vaiṇa	Making announcements, cutting bamboo
Veṇuka	Player of musical instruments
Vaitālika	Bell-ringer
Velava
Vaidehaka	Attending upon women, rearing animals for milk & selling milk-made products
Vyadha	Hunter, fowler
Śavara
Śūlika	Executioner of death by impalement
Śaṅkhakāra	Manufacturer of conchshell-made articles
Śaṅkhavanik	Dealer of conchshell-made articles
Śaraka
Śuddhamārjaka	Musicians
Śekhara	Sweeper
Sauṇḍika	Dealer of alcohol
Svarṇakāra	Goldsmith
Suvarṇavanik	Dealer of gold
Sauchika	Weaver
Sūta	Driving chariots & elephants
Sūtradhara	Masonry & stone-work
Sairindhra	Masseur & beautician
Saupaka	Hangman, dealer of meat

Note : Agriculture & labour are the universal professions amongst all varnasaṅkara communities.

This list is incomplete because Kane's sources were heavily centered around Marathi Hindu society to which I added those groups which are endemic to Bengali Hindu society. It may happen that some areas possess caste-groups endemic to that particular area only. For example within the Axomiya Hindu society, there is a caste-group called Kalitā who claim to be the descendants of vrātya kṣatriyas who fled to Kāmarūpa to avoid Paraśurāma's wrath (as stated in the Kālikapurāṇa) & married local women. Here in Bengal, the last surviving Buddhists who converted into Gauḍiya Vaiṣṇava faith describe themselves as belonging to the Vaiṣṇava caste.

Brahmavaivartapurāṇa divides the varnasaṅkaras into 2 categories - sat-śūdra & asat-śudra. It is the asat-śudra who are considered as untouchables. However, around Uttar Pradesh, sat-śūdra communities decline to accept their śūdra status & instead claim the status of kṣatriyas & vaiśyas (without undergoing the requisite saṁskāras like vrātyaṣṭoma or upanayana) & club the śūdra varṇa exclusively with the asat-śūdra jāti (leading to the misconception of all śūdras & varṇasaṅkaras being untouchables by default). For example, the record-keeper is considered as a sat-śūdra in Bengali Hindu society but in Uttar Pradesh, they claim the status of kṣatriyas due to their origin from an intercaste union involving a kṣatriya father.

Again in certain areas like Bengal & Nepal, sat-śudras are classified into 2 categories - those from whom brāhmaṇas can accept water & those from whom brāhmaṇas can't accept water but other sat-śūdras can. These rules varies across regions. For example, the weaver is a sat-śūdra in Bengali, Axomiya & Odiya Hindu societies from whom brāhmaṇas accept water but in Bihar (i.e in Maithili & Bhojpuri Hindu societies) & undivided Uttar Pradesh (i.e Hindusthani, Avadhi, Brajbhashi, Kumaoni & Garhwali Hindu societies), they are considered as untouchables. This might be due to absence a flourishing textile industry in these regions as the areas where the weaver is touchable, boasts of a rich legacy of producing superior handloom-based products (like Muga silk sari of Assam, Tānt sari of Bengal & Sambalpuri sari of Odisha). Similarly, kaivartas are considered as sat-śūdras from whom brahmaṇas can't accept water in Bengali, Oriya & Maithili Hindu societies but in Uttar Pradesh, they are considered as untouchables. Coincidentally, fish consumption is normalised among the Bengalis, Oriyas & Maithilis but is a taboo amongst the Bhojpuri, Avadhi, Brajbhashi & Hindusthani people

Recently, various varṇasaṅkara communities have attempted to raise their status in the Hindu society through measures like claiming descent from vrātya groups or directly from various deities, changing their surnames through affidavits, changing community names (some examples are allocation of the names Namaḥśūdra to caṇḍālas & Yādava to ābhīras and a section of gopālas) & mimicking customs of upper-castes like abandoning pork & alcohol and wearing a sacred thread. For example, here in Bengal, a section of kaivartas & washermen who have practiced agriculture for more than 3 generations (called Cāṣī-kaibarta & Cāṣā-dhopā respectively in Bengali) have successfully re-branded themselves as māhiśyas (called Satcāṣi in Bengali) in an attempt to raise their social status (in Bengali Hindu society, washermen & kaivartas are sat-śūdras from whom brāhmaṇas can't accept water but māhiśyas are sat-śūdras from whom brāhmaṇas can accept water). I expect similar cases to have also occurred elsewhere.

Another case worthy of mention is that of grahavipras in Bengali Hindu society. This is a community of Śākadvīpi brāhmaṇas who exclusively practice astrology for which they are looked down upon by other brāhmaṇas (who are mostly descendants of Kānyakubja brāhmaṇas) in Bengal. The hatred for grahavipras sometimes went to such extents that they would be even denied of their brāhmaṇa status & placed among varṇasaṅkaras by interpolating scriptures. This goes to show that the relative status of varṇasaṅkaras in the society depended to some extent upon the local brāhmaṇas. Another example is the status of Hinduised Boḍo-Kacārī tribes like the Koch (Kuvacha in Sanskrit), Hājoṅ, Kacāri, Ḍimāsā, Cutīyā, Āhom (who migrated from Yunnan province of China into the Brahmaputra Valley in the 13th century), Deurī, Rābhā, Tiwā & Morān of Assam who were accorded the status of sat-śūdras from whom brahmaṇas can't accept water by Axomiya brāhmaṇas due to their patronization of Hindu culture in the aftermath of the collapse of the Kāmarūpa kingdom (Koch king Naranārāyaṇa reconsecrated the shrine of *Kāmākhyā & patronized paṇḍita Pītāmbara Siddhāntavāgīśa, the codifier of Axomiya Hindu laws and Āhom king Suklenmung persecuted the followers of Śaṅkaradeva at the insistence of local brāhmaṇas. Suklengmung's descendant Sutanphaa, who assumed the Hindu name of Śivasiṁha donated large amounts of lands to brāhmaṇas, reconstructed important shrines around that of Kāmākhyā like those of Dīrgheśvarī, Ugratārā & Aśvakrāntā and constructed the large Śivadeul of Śivasāgara district) & periodic raids of Muslims from Bengal, Nāgā tribals from surrounding hilly regions & the Burmese from the southeast (repelled by the Cutīyā, Ḍimāsā & Āhom rulers) but brāhmaṇas in areas like Uttar Pradesh would consider them on par with their non-Hindu counterparts like the Boḍo & Gāro due to their Sino-Tibeto-Burmese appearance.

The current consensus is that in case of an intercaste marriage, the offspring bears the caste of it's father, as in the case of Agastya & Lopāmudrā, Cyavana & Sukanyā, Ṛcika & Satyavatī, Jamadgni & Reṇukā, Paraśara & Satyavatī and Viśravā & Nikaṣā.