This restriction, a by-product of the caste system is due to one interpretation of Hindu scripture. There are other interpretations that do not impose any restriction.
For example, in Mahabharata Santi Parva Section CCCXIX, Rishi Yajnavalkya is quoted as as saying,
'.. All orders of men are Brahmanas. All are sprung from Brahma. All
men utter Brahma. Aided by an understanding that is derived from and
directed to Brahma.
I inculcated this science teaching of Prakriti and
Purusha. Indeed this whole universe is Brahma. From the mouth of
Brahma sprung the Brahmanas; from his arms, sprung the Kshatriyas;
from his navel, the Vaisysa; and from his feet, the Sudras. All the
orders, (having sprung in this way) should not be regarded as
pilfering from one another.
Impelled by ignorance, all men meet with
death and attain, O King, to birth that is the cause of acts. Divested
of knowledge, all orders of men, dragged in terrible ignorance, fall
into varied orders of being due to the principles that flow from
Prakriti. For this reason, all should, by every means, seek to acquire
knowledge.
I have told thee that every person is entitled to strive
for its acquisition. One that is possessed of knowledge is a Brahmana.
Others, (viz., Kshatriyas, and Vaishyas and Sudras) are possessed of
knowledge. Hence this science of emancipation is always open to them
all.'