To understand why virginity is desired, but not required of men before marriage, we need to look at the purpose of marriage according to Hindu scripture.
On duties never to be neglected, Taittirīya Upaniṣad (1.11) says:
svādhyāyānmā pramadaḥ | ācāryāya priyaṃ dhanamāhṛtya prajātantuṃ mā vyavacchetsīḥ | satyānna pramaditavyam | dharmānna pramaditavyam | kuśalānna pramaditavyam | bhūtyai na pramaditavyam | svādhyāyapravacanābhyāṃ na pramaditavyam | devapitṛkāryābhyāṃ na pramaditavyam || 3 ||
3. From study swerve thou not. Having offered dear wealth to the teacher, cut thou not the progeny's line. From the true it will not do to swerve, nor from Dharma, nor from welfare. Neither will it do to swerve from well-being, nor from study and teaching, nor from duties to Devas and Pitṛs.
Śaṅkarāchārya comments:
Then, with the permission of the teacher, secure a suitable wife and prevent break in the line of descent. It will not do to bring about a break in the line of descent. That is to say, if a son is not born, attempts should de made to get a son by means of sacrificial rites such as the Putrakamya-iṣṭi, a rite performed with a view to get sons. This appears to be the meaning of the śruti because of the mention of three duties, "offspring, begetting, and propagation." Otherwise, the śruti would have mentioned only one,—that of begetting.
Swami Sarvananda commenting on the same:
Then should...progeny - i.e., after returning from the guru's house, or in other words, after the performance of the ceremony known as Samavartanam, one should take to a suitable bride for the procreation of children and thereby pay the debt of the fathers. This is an injunction of the Sruti not to remain outside any Ashrama. As soon as the student life is finished, one should enter into the householder's life or become a Sannyasin, but never remain in a state which is neither the one nor the other, that is called Antarashrama.
There is a further hint in this passage to the attitude one should have towards, the married life. One should look upon marriage not as an opportunity given for sexual enjoyment, but as a sacred duty towards the forefathers and the society inasmuch as by procreation of children the perpetuity of the family line is kept up and the departed forefathers get their offering from the family without a break, and also the social strength is maintained.
Manusmṛti and other dharma śāstras even suggest that parents who don't produce male offspring here on Earth have to spend time in a special hell in their afterlife:
Because the Son delivers his father from the hell called Put, therefore has he been called Putra, Deliverer from Put, by the Self-existent One Himself.—(9.138)
Comparative notes by various authors:
Hārīta (Vivādaratnākara, p. 583).—'There is a hell named Put; one whose line is broken goes to hell; hence as saving his father from that hell, the male child is called Putra.'
Bṛhaspati (Do., p. 584).—'Since the male child saves the father from the hell called Put, by his mere looking at his face,—therefore a man should make an effort to procure a son.'
What if sacrificial rites such as Putrakāmeṣṭi do not yield a son?
The couple used to go for niyoga if the man is either impotent or suffers from other diseases.
On failure of issue, the woman, on being authorised, may obtain, in the proper manner, the desired offspring, either from her younger brother-in-law or from a Sapiṇḍa.—(9.59)
If a son is born of the wife of a man, either dead or impotent or diseased, by one who has been duly authorised,—that son is declared to be kṣetraja, soil-born.—(9.167)
Or the man takes another wife (polygamy).
Āpastamba (Aparārka, p. 100).—‘So long as one’s wife is endowed with virtue and offspring, one shall not take to another wife; if she be wanting in either of the two, he shall take to another.’
As you can see, marital fidelity rules are a bit relaxed as one of the main objectives of marriage is to produce a (male) child. So, insisting that a man be virgin before marriage runs counter to this idea.
Why then is virginity required of women? And why isn't polyandry allowed the same way polygamy is?
There is no rational justification for this but there are plenty of scriptural reasons driven by prejudice against women to keep them in check. E.g., in the Mahābhārata we find Yudhiṣṭhira asking Bhīṣma, 'How can a marriage really work when the woman is wicked (anṛtāḥ striya) by nature?' and the learned Bhīṣma, instead of correcting this notion, goes on to narrate a story of an old female ascetic, Diśa, who tries to seduce a young Aṣṭāvakra, furthering the notion that, by nature, women are promiscuous and overcome by desire they would do anything:
Yudhishthira said,
I ask, O chief of Bharata's race, what is the origin of the saying, about discharging all duties jointly at the time of a person's taking the hand of his spouse in marriage? Is that saying in respect of discharging all duties together, due only to what is laid down by the great Rishis in days of yore, or does it refer to the duty of begetting offspring from religious motives, or has it reference to only the carnal pleasure that is expected from such union? The doubt that fills my mind in this respect is very great. What is spoken of as joint duties by the sages is in my consideration incorrect. That which is called in this world the union for practising all duties together ceases with death and is not to be seen to subsist hereafter. This union for practising all duties together leads to heaven. But heaven, O grandsire, is attained to by persons that are dead. Of a married couple it is seen that only one dies at a time. Where does the other then remain? Do tell me this. Men attain to diverse kinds of fruits by practising diverse kinds of duties. The occupations again, to which men betake themselves are of diverse kinds. Diverse, again, are the hells to which they go in consequence of such diversity of duties and acts. Women, in particular, the Rishis have said, are false in behaviour. When human beings are such, and when women in particular have been declared in the ordinances to be false, how, O sire, can there be a union between the sexes for purposes of practising all duties together? In the very Vedas one may read that women are false.
...
Bhishma continued,
Thus addressed, all those damsels circumambulated the Rishi and then left the chamber. Only that aged lady remained there. The day quickly passed and night came. The Rishi seated on a splendid bed, addressed the old lady, saying,--O blessed lady, the night is deepening. Do thou address thyself to sleep. Their conversation being thus put a stop to by the Rishi, the old lady laid herself down on an excellent bed of great splendour. Soon after, she rose from her bed and pretending to tremble with cold, she left it for going to the bed of the Rishi. The illustrious Ashtavakra welcomed her with courtesy. The lady however, stretching her arms, tenderly embraced the Rishi, O foremost of men. Beholding the Rishi quite unmoved and as inanimate as a piece of wood, she became very sorry and began to converse with him. There is no pleasure, save that which waits upon Kama (desire), which women can derive from a person of the other sex. I am now under the influence of desire. I seek thee for that reason.
Ashtavakra said,
O blessed lady, I never approach one that is another's spouse. One's congress with another man's wife is condemned by persons conversant with the scriptures on morality. I am an utter stranger to enjoyments of every kind. O blessed lady, know that I have become desirous of wedlock for obtaining offspring. I swear by truth itself. Through the aid of offspring righteously obtained, I shall proceed to those regions of felicity which cannot be attained without such aid. O good lady, know what is consistent with morality, and knowing it, desist from thy efforts.
The lady said,
The very deities of wind and fire and water, or the other celestials, O regenerate one, are not so agreeable to women as the deity of desire. Verily, women are exceedingly fond of sexual congress. Among a thousand women, or, perhaps, among hundreds of thousands, sometimes only one may be found that is devoted to her husband. When under the influence of desire, they care not for family or father or mother or brother or husband or sons or husband's brother (but pursue the way that desire points out). Verily, in pursuit of what they consider happiness, they destroy the family (to which they belong by birth or marriage) even as many queenly rivers eat away the banks that contain them. The Creator himself had said this, quickly marking the faults of women.
Bhishma continued,
The Rishi, bent upon finding out the faults of women...
Elsewhere in Mahābhārata we see a woman ready to give up her life because her husband is allowed to take another wife:
Therefore, revered sir, protect thy own self by abandoning me. O, give me leave to sacrifice myself, and cherish thou my children. Those that are conversant with the morals have, in their treatises, said, that women should never be slaughtered and that Rakshasas are not ignorant of the rules of morality. Therefore, while it is certain that the Rakshasa will kill a man, it is doubtful whether he will kill a woman. It behoveth thee, therefore, being conversant with the rules of morality, to place me before the Rakshasa. I have enjoyed much happiness, have obtained much that is agreeable to me, and have also acquired great religious merit. I have also obtained from thee children that are so dear to me. Therefore, it grieveth not me to die. I have borne thee children and have also grown old; I am ever desirous of doing good to thee; remembering all these I have come to this resolution. O revered sir, abandoning me thou mayest obtain another wife. By her thou mayest again acquire religious merit. There is no sin in this. For a man polygamy is an act of merit, but for a woman it is very sinful to betake herself to a second husband after the first. Considering all this, and remembering too that sacrifice of thy own self is censurable, O, liberate today without loss of time thy own self, thy race, and these thy children (by abandoning me).
What could be the real reason behind some of these sexist and misogynist stories?
Are writers of these stories (obviously, men) projecting their own insecurities onto the opposite sex? Varāhamihira in Bṛhatsaṃhitā seems to think so:
Chapter 74 - Praise of Women (strī-praśaṃsā)
Śloka 15. — How impudent are the immoral fellows that slander blameless women! They are like the thieves actually found stealing and yet crying — "Thief, stop!"