The wearing of the sacred thread is part of the Dharma (duty) of the first three castes because as the Manu Smriti describes, they are the ones who can read the Vedas:
87) But in order to protect this universe He, the most resplendent one, assigned separate (duties and) occupations to those who sprang from his mouth, arms, thighs, and feet.
88) To Brahmanas he assigned teaching and studying (the Veda), sacrificing for their own benefit and for others, giving and accepting (of alms).
89) The Kshatriya he commanded to protect the people, to bestow gifts, to offer sacrifices, to study (the Veda), and to abstain from attaching himself to sensual pleasures;
90) The Vaisya to tend cattle, to bestow gifts, to offer sacrifices, to study (the Veda), to trade, to lend money, and to cultivate land.
91) One occupation only the lord prescribed to the Sudra, to serve meekly even these (other) three castes.
The Manu Smriti also describes how the age when the sacred thread ceremony (Upanayanam) can be performed is different for each of the first three castes; for Brahmins it's age 5-16, for Kshatriyas it's 6-22, and for Vaishyas it's 8-24:
36) In the eighth year after conception, one should perform the initiation (upanayana) of a Brahmana, in the eleventh after conception (that) of a Kshatriya, but in the twelfth that of a Vaisya.
37) (The initiation) of a Brahmana who desires proficiency in sacred learning should take place in the fifth (year after conception), (that) of a Kshatriya who wishes to become powerful in the sixth, (and that) of a Vaisya who longs for (success in his) business in the eighth.
38) The (time for the) Savitri (initiation) of a Brahmana does not pass until the completion of the sixteenth year (after conception), of a Kshatriya until the completion of the twenty-second, and of a Vaisya until the completion of the twenty-fourth.
The material the sacred thread is made up of is also different:
44) The sacrificial string of a Brahmana shall be made of cotton, (shall be) twisted to the right, (and consist) of three threads, that of a Kshatriya of hempen threads, (and) that of a Vaisya of woollen threads.
By the way, the fact that the first three castes wear the sacred thread is why these castes are called Dvija or twice-born; the wearing of the sacred thread is thought to be a second birth for a person.