Why are Apsaras frequently sent to distract a sage from his ascetic practices? This theme seems to appear over and over again in many Puranic stories. Is there any significance attached to this theme or is it just part of the stories?
Well Indra is the title of the leader of the gods and it is not easy to get there. One has to do an intense tapasya or a hundred Aswamedhs to attain the position as mentioned in Section XXXIII of Shanti Parva of Mahabharat.
So anyone who gets the post after so much hard work is naturally wary of others who might be trying to replace him. Sending a beautiful Apsara to seduce a tapasvi and break his penance is one of the ways for Indra to eliminate competition & it is sort of a political strategy to distract the competition.
Vishwamitra says the same thing in Valmiki Ramayan Baal-Kand:
'All this is the mischief of gods to defraud me of the great merit of my ascesis. Ten years have rolled by as if they are just a day and a night. Furthermore, I who am under the influence of lust and lure had to encounter this hindrance in my ascesis. [1-63-11, 12a]
In Skanda Purana section titled Dharmaranya Kshetra a similar story is told:
Once, Sage Vyasa went to meet Yudhisthira, who requested him to describe the significance of Dharmaranya kshetra.
Sage Vyasa replied-- 'O king! Once, Dharmaraj did an austere penance to please Lord Shiva. As usual, Indra became scared because he thought that Dharmaraj was doing penance with the intention of acquiring Indraloka. He and other deities went to seek the help of Lord Brahma, who did not have any clue. So, all of them went to Kailash mountain to seek Lord Shiva's help. Since Shiva was aware of Dharma's motive, he explained to the deities that there was nothing to worry about. But, Indra was not satisfied and the thought of loosing his kingdom continued to torment him.
Indra then instructed a beautiful Apsara named Vardhini to go to the place where Dharma was doing penance and disturb him by corrupting his thoughts. Vardhini went to the place where Dharma was engrossed in his penance and was successful in disturbing him. When Dharma opened his eyes he found a beautiful Apsara in front of him. Vardhini asked Dharma--'O Lord! What is the objective with which you are doing such an austere penance? Being an embodiment of virtuosity yourself, you prevail in the whole world. So, in a sense you already the lord of the world - what else do you need?'
Dharma told Vardhini that he was doing penance with the objective of having a divine glimpse of Lord Shiva. Vardhini informed him that Indra was scared of losing his kingdom and hence had sent her to disturb his penance. Dharma was pleased by her truthfulness and wanted to reward her for that. Vardhini expressed her desire of having her abode in Indraloka for eternity and also of having a place of pilgrimage named on her. Dharma blessed her after which she returned to Indraloka.
While this seems to be a tried and tested way to make the person performing a penance lose his way, in some episodes Indras have even resorted to stealing the horses meant for the sacrifice as is evident from this incident at Prithu's Ashwamedh and again when King Sagar performed the same ritual.
There are several reasons why severe penances are performed, and Indra does not always knows them. He does however, almost always suspects the intentions of being “hurled from his high seat in heaven”. Sometimes his interference works, and the penance is disturbed. Sometimes, his obstacles fail, Indra succumbs, and the sage gets what he wants.
In order to demonstrate the suspicious nature of Indra, refer to the legend of “cakra-tirtha” found in chapter 18 of the Gautami-mahatmya (final part of the Brahma-purana):
The Sun said: O Indra, Yama is now on the banks of Godāvarī performing a very severe penance. I do not know what the reason is.
Brahmā said: On hearing these words of the Sun Indra became suspicious.
Indra said :Alas! Woeful! O woeful! My lordship of Devas has come to an end. Yama of evil activities performing penance at Godavari! It is my opinion that he is surely desirous of seizing my position, O Devas.
It is however, not realistic to say that Indra’s position in heaven is the sole reason for disturbing penances by sending beautiful Apsaras. The Ramayana for example, has a long narration of sage Vishvamitra who performed severe penances for thousands of years, which formed a danger to Indra, the devas and all of the three worlds. The epilogue sums it all up:
(1.65.8) “That eminent saint remained without respiration for another thousand years, and then fumes have started to emit from the head of sage who is controlling his breath, by which fumes the triad of worlds looked as if it is searing, and this startled all the worlds.”
(1.65.12) “Now, even an imperceptible imperfection does not really appear in him, but if his heartfelt desire is not meted out, he will devastate the triad of worlds with his ascetic power.”
The essence of this legend explains what other Puranic stories also say: the sages, although possessive of great ascetic power, are known for their anger and wrath (hence their curses). They are, for example, capable of destroying the three worlds with a glance. In order to demonstrate this, see the Mahabharata (book 1 chapter 71) which also recounts legend of Vishvamitra and Menaka:
Hearing all this, Menaka replied,
The illustrious Viswamitra is endued with great energy and is a mighty ascetic. He is very short-tempered too, as is known to thee. The energy, penances, and wrath of the high-souled one have made even thee anxious. Why should I not also be anxious? He it was who made even the illustrious Vasishtha bear the pangs of witnessing the premature death of his children.
… (here follows a list of the various acts resulting from Vishvamitra’s wrath)…
Tell me, O Indra, the means that should be adopted so that I may not be burnt by his wrath. He can burn the three worlds by his splendour, can, by a stamp (of his foot), cause the earth to quake. He can sever the great Meru from the earth and hurl it to any distance. He can go round the ten points of the earth in a moment.
… etc …
One example of Indra failing to distract a Brahmana is found in the Padma Purana:
In this way, Menaka failed in her mission. Indeed, again and again, Indra put obstacles on Vishnusharman's path, but each one failed. Understanding that Indra was trying hard to impede him, Vishnusharman became very angry. He said, "Today, I shall cause Indra to fall from heaven! Then, I shall appoint someone else as king of the demigods."At this moment, Indra arrived there. After glorifying the brahmana, Indra begged for forgiveness and offered a benediction. Vishnudharman said, "You should never offend a brahmana, because their wrath can destroy an entire dynasty. Had you not come to me now, I would have taken away your kingdom and awarded it to someone else. If you desire to give me a benediction, this is what I want: Please give me some nectar, as well as steady devotion to my father."Indra delivered some nectar to Vishnudharman, who then returned home to his father."
The above story also demonstrates the wrath of the Brahmana Vishnusharman that might be released upon not fulfilling his objectives.
Finally, here is a list of names of Apsarasas that are known to interrupt the penances of sages, so that you might investigate yourself: Menakā, Sahajanyā, Ghritācī, Pramlocā, Visvāci, and Pūrvacitti.