I don't have much knowledge about Lord Hayagriva, just known that he was the one who provided vedic knowledge to Lord Brahma,he has horse head, was an avatar of Lord Vishnu, white in color, and symbol of wisdom and knowledge. And this is also not from very trustworthy sources.
Wanted to know, his history as per written in trustworthy textures. Please try to avoid Wikipedia, and throw me light with actual textures(translated).
Yes, Lord Hayagriva is the avatar of Vishnu with a horse head. Haya means horse and griva means neck. So the deity with a horse head from his neck is known as Hayagriva. The incident of how He got that horse head is present in different scriptures with slight variations.
Once in the ancient time the gods decided to do a Yajna. But they couldn't find where was Vishnu. They searched for Him at different places, and then after meditating they got to know where He was present. When they got there they saw that Shri Vishnu was in meditation supporting His head on His bow. To wake Him up, Brahma created some ants (or that ant like creatures that eat away wood and things) and told them to eat away one part of the bow so that when it breaks the Lord's meditation will break by losing the support. However, when it was done, the Lord's head got cut up getting tangled with the string of the bow. Then to correct it a horse head was fixed there by the god Viswakarma. After that incident Vishnu did penance and got back His original head. [Skanda Purana - 3.2.14,15 / Devi Bhagavata - 1.5]
The scriptures also mention the reason why the Lord's head was lost. But the reasons given are different. While Skanda purana states it was due to a curse from Brahma, the Devi Bhagavata says it was due to a curse from Laxmi and also because a demon by the same name who steals the Vedas from Brahma eventually (SB - 8.24), has asked the boon to be killed by someone who had a horse head.
So overall, even the accounts differ slightly in the scriptures, Hayagriva is the horse headed incarnation as you have heard.
When Madhu and Kaitabha stole the Vedas from Brahma, He become overtaken by grief and requested Lord Vishnu to rescue the Vedas. So the Lord as Hayagriva brings the Vedas back and gives it to Brahma:
The two Asuras, making an appointment with the Vedas in respect of the time when they would come back to take them up again, threw them down in the nether region, and ran towards the spot whence those sounds appeared to come. Meanwhile, O king, the Supreme Lord with the equine head, otherwise called Hari, who was himself in the nether region, took up all the Vedas. Returning to where Brahma was staying, he gave the Vedas unto him. Having restored the Vedas unto Brahma, the Supreme Lord once more returned to his own nature.
Having compassed the destruction of the two Asuras and restored the Vedas to Brahma, the Supreme Being dispelled the grief of Brahma. Aided then by Hari and assisted by the Vedas, Brahma created all the worlds with their mobile and immobile creatures. After this, Hari, granting unto the Grandsire intelligence of the foremost order relating to the Creation, disappeared there and then for going to the place he had come from. It was thus that Narayana, having assumed the form equipt with the horse-head, slew the two Danavas Madhu and Kaitabha (and disappeared from the sight of Brahma).
Reference
Mahabharat - 12.348
12.348
Sri Hayagriva is in fact an avatar of Lord Vishnu, usually depicted with the head and neck of a horse (He is not to be confused with the Asura Hayagriva (also depicted with a horse head) who stole the Vedas from Brahma and was killed by Sri Matsya Avatar).
Probably the best description we have of Him comes from the Hayagriva Stotram, a prayer in his praise.
We meditate upon that Supreme One, who has the neck and face of a horse and who is the embodiment of Jn~AnA (divine Knowledge) and AnandhA (Bliss). He has a ThirumEni (body) like a radiant, blemishles Spatikam (Crystal) and is the abode of all VidhyAs (branches of Learning).
You will notice that this stanza introduces the concept that Sri Hayagriva is considered sacred to academic and educational pursuits. This idea is expanded upon in a later stanza which describes the neighing sound that He produces to be the very Vedas themselves.
The Hala-Hala sounds in the form of neighings emanating from the divine throat of Sri HayagrIvan, the Horse-faced Lord, are the aggregate of Saama VedA mantrAs, the word for word meanings of the Rg vedA (Rks) and the abode of Yajur VedA manthrams .These Hala-Hala nadhams are the destroyers of all obstacles to true Jn~Anam about ChEthana-AchEthana-Isvara tatthvams; these Hala-Hala sounds are like the movement of the waves from the ocean of true knowledge (Jn~Anam). May these sunAdhams chase away the inner darkness (ajn~nam) arising from the uproar (loud and confusing noise) of debaters whose minds are driven by their own pride, arrogance and viparItha Jn~Anam.
But if Sri Hayagriva is the source of the Vedas, whence did Brahma learn them? The answer lies just a few stanzas below.
The Horse-faced Lord worshipped as the Lord of Speech (VaagIsan )is indeed the avathAra roopam of Para VaasudEvan of Sri VaikuNtam. He instructs Brahma Devan on the VedAs and is like the unique , morning light ( prAtha: SandhyA )to drive away our inner darkness . He is the matchless , sacred collyrium (anjanam) for revelation of the dhivya Jn~Anam .May this subhAsraya ThirumEni of this incarnation of Para VaasudEvan shine with in and before me always!
So it was Sri Hayagriva Himself who originally instructed Brahma in the Vedas. In fact, it was He too who instructed Saraswati and Veda Vyasa, as is described later.
The pretty-to-behold (ramyA) form of KailAsanAthan (GirIsan) known as DakshiNAmUrthy, the "Goddess of Learning " and the wife of Brahma Devan with the name of Sarasvathi having the white lotus as Her abode and even Sage VyAsA and others famous for their powers of divine Vaak --all of them derive their Saarasvatham (power of divine speech ) from a mere fragment of the power of the Adhi VidhyA Moorthy, Lord HayagrIvan.
Now after Brahma had received the Vedas, he did in fact lose them and Sri Hayagriva had to retrieve them for him. From Srimad Bhagavatam 8:24:57:
At the end of each kalpa, when the demon Ajnana steals the Vedic knowledge, Lord Hayagriva appears and preserves it. Then He delivers it to Lord Brahma.
At the end of the millennium, ignorance personified assumed the form of a demon, stole all the Vedas and took them down to the planet of Rasatala. The Supreme Lord, however, in His form of Hayagriva retrieved the Vedas and returned them to Lord Brahma when he begged for them. I offer my respectful obeisances unto the Supreme Lord, whose determination never fails.
There are actually multiple accounts of demons stealing the Vedas and Sri Hayagriva recovering them, but since @jabahar has already addressed the incident detailed in the Mahabharatam, I have described the one from Srimad Bhagavatam.
In essence, Sri Hayagriva is the source of all divine knowledge, and worship of him is considered extremely auspicious for any academic pursuits.
-This supplements the previous answers:
-This information i got from Srivaishnavism blog spot
The avatara of Hayagriva Bhagavan took place to restore the Vedas to Brahma. Lord Vishnu/Narayana had taught BrahmA the Creation through His breath of VedAs. Then, Brahma could understand how various kalpams had begun due to Lord’s grace. Brahma became extremely proud and head strong of his position as the creator and about his powers. Lord Narayana as usual wanted to teach him a lesson and decided to remove Brahma's pride. A couple of water droplets from the lotus seat of the Lord incarnated as two Asuras, Madhu and Kaitabha; kaitabha was of thamO guNA and the other (madhu) was of rajO guNA. They stole the Vedas from Brahma. Lord Vishnu smiled.
Unable to carry on his work of creation without the Vedas, Brahma rushed to the Lord and pleaded Lord Narayana for mercy and saving Vedas. BrahmA prostrated to the Lord and said:
"Vedas alone are my eyes; they are my wealth; They are my Lord. The whole world is surrounded by darkness due to the absence of Vedas. How am I to proceed on my creation without the Vedas ? Please arise from the Yoga nithrA and help me Lord! Please give me back my eyes which have been blinded by my own pride."
"veda: mE paramam sakshI: veda: mE paramam paramveda: mE paramam dharma, veda: mE brahma sOtthamam"
Lord Narayana incarnated as Hayagriva, appeared as the white Horse faced, faultless sphatika hued form, sattva form, with Divya tEjas, and lustrous form and destroyed the Asuras and restored the Vedas to Brahma.
With a lovely long nose, like the heaven surrounded by white bright stars, the asva siras (head of horse) illumined the whole world. The upper world and the lower world became His ears. The rays of Sun's brightest lustres are His hairs at the nape of His neck (pidari). BhUmi became His forhead; Ganga and Saraswathi became two lovely eyebrows; Chandra Sooryar (The Moon and the Sun) became His two eyes; SandhyA dEvathai became His nostrils; Pithru devathAs became His teeth; GolOkam and Brahma lOkam became His two lips; kalaraathri became His neck; The Divya tEjas Sathva form Sri Hayagreevan thus, in a grandest beautiful manner appeared. He rushed to PaathaaLa lOkam and raised His "uthGitha" in samavEdha swara and terrified Madhukaitabha asurAs; They hid the VedhAs (which were in the form of babies) and ran away from the scene. Lord Hayagreeva handed over the Vedas to BrahmA and went back. Madhukaitabha searched for the sound which terrified them earlier but found the Vedas missing. They rushed to BrahmA who in turn was terribly scared and sought the help of Lord Narayanan, again. Lord Hayagreeva fought with asurAs and killed them. BrahmA continued his work of creation.
This avataara is described in VishNu puraaNam as "mathsya kUrma varaaha ashva simha rUpaathibhi:".
-In BrahmANda purANam
During maheshwara-naradha samvaadham, Naradha describes this avthaar. Even when naradha praises Sri ranga kshetra in this PurANam, he says "ashva sirasE namah:". That is why the Lord is called Madhusoodhanan. (for having killed madhu and kaitabha).
In MahAbharatham: Shanthi parvam, hayasira upaakhyaanam details Hayagreeva avathaara. Srimad Bhagawatham, too, describes Hayagreeva avtaara.
There is another version of this avtaar and killing of asurAs. When paraLaya was about to end, Lord Vishnu out of His dirt from the ear drum made two small solid balls and dropped on the Lotus leaf and BrahmA activated the PrANa vaayu, which gave life to these two, as madhu and kaitabha. These asurAs appeared and grew up.
BrahmA started off his creation and Vedhas were snatched away by these two asurAs. When the most compassionate Lord Vishnu, then wished to please the asurAs, without killing them, asked them as to what they want as a boon. These two asurAs said "we can give You what You want". That is it! The Lord decided to kill them. They said "Can You kill us only where there is no cover for the sky?". The Lord immediately took Hayagreeva avtaara and removed His cloth on the Thighs, put them on His thighs and killed them. Lord’s legs house the Earth and the space/sky. Since the Lord removed the cloth from His thighs and killed them, He still complied with what they challenged Him. What they thought was impossible to happen, He could make it happen with no effort. Lord Hayagreeva, thus saved the Vedas and the world. The moral is: thamas, rajas guNAs are to be destroyed to reach Him.
-There is yet another interesting "different" version in BrahmANda PuraaNam and Sri Devi Bhagawatham.
There was a asurA, by name Hayagreeva (with horse face) who did his severe penance/ tapas on Parvathi Devi. Parvathi, pleased with his tapas, appeared in his front and the asurA asked for "no death". She said "Impossible-cannot be granted". Then the asurA amended the request and said "Except for a horse faced one, no one else can kill me". (thinking, that it can never happen!). She consented. He became tremendously proud of his achievement and started harassing every deva and rishi. He troubles all three worlds. He snatched away the Vedas from BrahmA and disappeared. The Lord appeared as Hayagreeva and killed the asurA to save the world and bring back the Vedas.
-There is yet another puraaNic narration on Hayagreevar.
At Kanchi, Sage Agasthya was on severe penance on Lord Narayanan and the Lord appeared as Hayagreeva and was immensely pleased with his tapas. He blessed the muni with Devi mahaathmyam. This is described in BrahmANda puraaNam Sri Hayagreeva agasthya samvaadham.
During Tripurasura samhaara to entice the asurAs, the Lord appeared as "other religious" saint (buddhism?) and appeared as Hayagreeva to mislead the asurAs from the Veda maargham (from the path of Vedic traditions and sayings). Thus, He made them lose their ability to get saved and the Lord won. (As also claimed and reported in Buddhism in a web site as follows: Like Mahakala, Hayagriva is one of the Eight Great Protectors of Buddhism, a guardian and a destroyer of obstacles to enlightenment(!). He is a popular personal, or tutelary, deity among the Gelug order of Tibetan Buddhism.
The terrific neigh that emanates from this horse's head is said to pierce through the illusory nature of reality. Although he was also popular in Tibet and China, Hayagriva's association with the horse may have had a particular appeal to the Mongols. (This, I read in the Net in some buddhism web site!) Scriptures on
Since, Lord Hayagreeva killed the asurAs, madhu and kaitapar, He is referred to as Madhusoodhana.
In BrahmANda puraaNam, Sri RangarajamahAthmyam, Om namO VishnavE Deva MadhussodhanathE Namah: refers to Madhusoodhana Sri Hayagreeva.
Srimad Valmiki RamayaNam BalakaaNdam 76th sargam, 17th slOkam Parasuraama says to Rama "akshayam madhuhanthaaram jaanaami tvaam surOtthamam". I realise that You are the DevaadhiDevan, immortal, MahAVishNu, the One who killed madhu,kaitapha.
Also, when Rama (before proceeding for His PattabhishEkha), gets up in the wee hours, in Brahma muhUrtham, He performs His anushtaanam and pays obeisance to Lord Hayagreevan "dhushtaava praNadhaischaiva sirasaa madhsoodhanam" (6th sargam-7th slokam).
-In Kishkinthaa kaanDam, Vaali tells Sri Rama
"Even if that rAvaNan hides SitA in PaathaaLa lOkam, no problem. You could have just told me so. I would have brought SitA back from anywhere wheresoever like Sri Hayagreevar brought the Vedas killing Madhukaitapar. (17th sargam-49th slOkam)
There is one Upanishad called shveta ashvatara Upanishad and when it was in the shape of white female horse, this horse was stolen by Madhu and Kaitabha demon-brothers and later retrieved by Hayagreeva, another aspect of Vishnu.
When Hanumaan went in search of sanjeevini, and other aushadhaas, he saw the place where chathurmukha brahmA worshipped Lord Hayagreeva. Commentators, are reported to have praised this slOkam as "hayaananam, hayagreeva araadhana sthaanam", and "hayaananam - BhagawathO Hayagreevasya sthaanam".
Sri MahAbhAratham, Santhi parvam mentions Haygreeva avataar.
Sri HayagreevOpanishad, naturally elaborates the greatness of Haygreevar.
In Rk vEdam, "vaagham bruNi sookhtham", "apradhiratham", Yajur vEdham, "yajnya prakaraNam", Saama vEdham, udgItam, AdharvaNa vEdham, Shanthikam, Bhaushtikam are all referring to the greatness of Lord Hayagreevar.
The story of killing Madhu Kaitabha is the real fact of Bhagavan Hayagriva. I was in Himalayas Ksheera Saagara with Almighty for a long span of time & also got preaching with Many Great Rishis like Veda Vyasa blessed gnana deeksha during my tenure. They also said killing Madhu Kaitabha is the real fact of Bhagavan Hayagriva. Am not so good in English & Secondly many things cannot be briefed in English.
The other history in the name of Hayagriva is not story or part of Lord Vishnu. The story is of demon got form of Horse Neck Face & was destroyed. Many people have manipulated scriptures & texts. So due to this there is lot of confusion has caused for the people based on outer world.
