Do the Vedas talk directly about avataras?
Are there any direct references in main Vedic texts?
Note: kindly limit the references to samhita, brahmanas, aranyaka and 10 upanishads.
Yes Vedas do talk about avataras. Shathapatha Brahmana tells us that Prajapati assumed the Kurma Avatara. In Shukla Yajur Veda Samhita we find mantras which says Supreme Brahman took birth as Brahma. It also points towards Vaman Avatara.
सुभू: स्वयम्भू: प्रथमोन्तर्महत्यर्णवे ।
दधे ह गर्भमृत्वियं यतो जात: प्रजापतिः।।23.63।।
Then originator of whole universe "Swayambhu Paramatma" (Supreme Soul) in the middle of a great lake ,laid down embryo timely ,from which Brahma (Prajapati) was born.Yajurveda Samhita 23.63
Here we can see that ,the supreme soul himself incarnated as Brahma. Through womb.
5 And as to its being called 'kūrma' (tortoise);Prajāpati, having assumed that form, created living beings. Now what he created, he made; and inasmuch as he made (kar), he is (called) 'kūrma;' and 'kūrma' being (the same as) 'kaśyapa' (a tortoise), therefore all creatures are said to be descended from Kaśyapa.
This mantra indirectly indicates towards Lord Vishnu's "Narasimha Avatara". i.e. Man-Lion form .As it says Lord Vishnu roam around the earth in the form of fierce Lion etc.
प्र तद्विष्णु: स्तवते वीर्येण मृगो न भीमः कुचरो गिरिष्ठाः।
यस्योरुषु त्रिषु विक्रमणेष्वधिक्षियान्ति भुवनानि विश्वा ।।5.20।।
Pra TadVishnuhu Stavate Viryena Mrugo Na Bhima Kucharo Girishtha
YasyoRushu Trishu VikramNeShawaDhiKshiYanti Buvanani Vishwa
Roaming around the earth in fiearc avatars like Lion etc. Residing on mountains , all-pervading Vishnu is highly adorable through his Manhood (Bravery). All the lokas resides under the three giant strides of him. Vishnu deva is praised here as thus.Yajurveda Samhita 5.20
Above mantra is also giving us indirect reference about Lord Vishnu's "Vaman Avatara" as it also talks about three strides of him.
As we know the concept of avatars is more clear in puranas. In Vedas we can see primary source of them. Vedas being abstract than puranas do not describe them in much details. The word avatar does not directly appear in the Vedic literature. But from the definition we can see that avatara also means"to make one's appearance" ,this concept is visible in vedas.
The concept of taking Avatara is not alien to Vedas. For example in Adhyatma Prakarana Sukta of AtharvaVeda Samhita avatara of Mahadeva is described:
व्रात्य आसीदीयमान एव स प्रजापतिं समैरयत् ।।
स प्रजापति: सुवर्णमात्मन्नपश्यत्तत्प्राजनयत् ।।
तदेकमभवत्तल्ललाममभवत्तन्महदभवत्तज्जेष्ठमभवत्तद्ब्रह्माभवत्तत्तपोऽभवत्तत्सत्यमभवत्तेन प्राजायत ।।
सोऽवर्धत स महानभवत्स महादेवोऽभवत् ।।
स देवानामीशां पर्यैत्स ईशानोऽभवत् ।।
स एकव्रात्योऽभवत्स धनुरादत्त तदेवेन्द्रधनुः ।।
नीलमस्योदरं लोहितं पृष्ठम् ।।
नीलेनैवाप्रियं भ्रातृव्यं प्रोर्णोति लोहितेन द्विषन्तं विध्यतीति ब्रह्मवादिनो वदन्ति ।। [Atharva Veda 15:1]
There was Vratya. He roused Prajapati to action. Prajapati beheld gold in himself and engendered it. That became unique, that became distinguished, that became great, that became excellent, that became Brahman, that became Tapas, that became Truth: through that he (Vratya) was born. He grew, he became great, he became Mahadeva. He gained the lordship of the Gods. He became Ishana. He became Eka Vratya. He held a bow, even that Bow of Indra. His belly is dark-blue (Nila), his back is red (Lohitam). With dark-blue he envelops a detested rival, with red he pierces the man who hates him: so the knowers of Brahman say.
So, the theory seen above is:
Vratya (Supreme Being) ➡Takes birth through Prajapati➡becomes Mahadeva/Ishana/Eka Vratya
Vamana Avatar is mentioned in Rig Veda:
तरीणि पदा वि चक्रमे विष्णुर्गोपा अदाभ्यः |
अतो धर्माणि धारयन || 1.22.18
Viṣṇu, the Guardian, he whom none deceiveth, made three steps; thenceforth Establishing his high decrees.
पर विष्णवे शूषमेतु मन्म गिरिक्षित उरुगायाय वर्ष्णे | य इदं दीर्घं
परयतं सधस्थमेको विममे तरिभिरित पदेभिः || 1.154.3
Let the hymn lift itself as strength to Viṣṇu, the Bull far-striding, dwelling on the mountains, Him who alone with triple step hath measured this common dwelling-place, long, far extended.
यदा ते विष्णुरोजसा तरीणि पदा विचक्रमे | आदित ते || 8.12.27
When Viṣṇu, through thine energy, strode wide those three great steps of his, Then thy two beautiful Bay Steeds carried thee on.
Satapatha Brahamana mentions Matsya Avatar:
In the morning they brought to Manu water for washing, just as now also they (are wont to) bring (water) for washing the hands. When he was washing himself, a fish came into his hands.
It spake to him the word, 'Rear me, I will save thee!' 'Where from wilt thou save me?' 'A flood will carry away all these creatures: from that I will save thee!' 'How am I to rear thee?'
It said, 'As long as we are small, there is great destruction for us: fish devours fish. Thou wilt first keep me in a jar. When I outgrow that, thou wilt dig a pit and keep me in it. When I outgrow that, thou wilt take me down to the sea, for then I shall be beyond destruction.'
It soon became a jhaṣa (a large fish); for that grows largest (of all fish). Thereupon it said, 'In such and such a year that flood will come. Thou shalt then attend to me (i.e. to my advice) by preparing a ship; and when the flood has risen thou shalt enter into the ship, and I will save thee from it.'
After he had reared it in this way, he took it down to the sea. And in the same year which the fish had indicated to him, he attended to (the advice of the fish) by preparing a ship; and when the flood had risen, he entered into the ship. The fish then swam up to him, and to its horn he tied the rope of the ship, and by that means he passed swiftly up to yonder northern mountain.
It then said, 'I have saved thee. Fasten the ship to a tree; but let not the water cut thee off, whilst thou art on the mountain. As the water subsides, thou mayest gradually descend!' Accordingly he gradually descended, and hence that (slope) of the northern mountain is called 'Alarm's descent.' The flood then swept away all these creatures, and Manu alone remained here.
Though it doesn't say Vishnu incarnated as Matsya, this description is almost the same as Matsya Avatar described in Ithihasa and Puranas.