Sri Hayagriva is in fact an avatar of Lord Vishnu, usually depicted with the head and neck of a horse (He is not to be confused with the Asura Hayagriva (also depicted with a horse head) who stole the Vedas from Brahma and was killed by Sri Matsya Avatar).
Probably the best description we have of Him comes from the Hayagriva Stotram, a prayer in his praise.
We meditate upon that Supreme One, who has the neck and face of a
horse and who is the embodiment of Jn~AnA (divine Knowledge) and
AnandhA (Bliss). He has a ThirumEni (body) like a radiant, blemishles
Spatikam (Crystal) and is the abode of all VidhyAs (branches of
You will notice that this stanza introduces the concept that Sri Hayagriva is considered sacred to academic and educational pursuits. This idea is expanded upon in a later stanza which describes the neighing sound that He produces to be the very Vedas themselves.
The Hala-Hala sounds in the form of neighings emanating from the
divine throat of Sri HayagrIvan, the Horse-faced Lord, are the
aggregate of Saama VedA mantrAs, the word for word meanings of the Rg
vedA (Rks) and the abode of Yajur VedA manthrams .These Hala-Hala
nadhams are the destroyers of all obstacles to true Jn~Anam about
ChEthana-AchEthana-Isvara tatthvams; these Hala-Hala sounds are like
the movement of the waves from the ocean of true knowledge (Jn~Anam).
May these sunAdhams chase away the inner darkness (ajn~nam) arising
from the uproar (loud and confusing noise) of debaters whose minds are
driven by their own pride, arrogance and viparItha Jn~Anam.
But if Sri Hayagriva is the source of the Vedas, whence did Brahma learn them? The answer lies just a few stanzas below.
The Horse-faced Lord worshipped as the Lord of Speech (VaagIsan )is
indeed the avathAra roopam of Para VaasudEvan of Sri VaikuNtam. He
instructs Brahma Devan on the VedAs and is like the unique , morning
light ( prAtha: SandhyA )to drive away our inner darkness . He is the
matchless , sacred collyrium (anjanam) for revelation of the dhivya
Jn~Anam .May this subhAsraya ThirumEni of this incarnation of Para
VaasudEvan shine with in and before me always!
So it was Sri Hayagriva Himself who originally instructed Brahma in the Vedas. In fact, it was He too who instructed Saraswati and Veda Vyasa, as is described later.
The pretty-to-behold (ramyA) form of KailAsanAthan (GirIsan) known as
DakshiNAmUrthy, the "Goddess of Learning " and the wife of Brahma
Devan with the name of Sarasvathi having the white lotus as Her abode
and even Sage VyAsA and others famous for their powers of divine Vaak
--all of them derive their Saarasvatham (power of divine speech ) from a mere fragment of the power of the Adhi VidhyA Moorthy, Lord
Now after Brahma had received the Vedas, he did in fact lose them and Sri Hayagriva had to retrieve them for him. From Srimad Bhagavatam 8:24:57:
At the end of each kalpa, when the demon Ajnana steals the Vedic
knowledge, Lord Hayagriva appears and preserves it. Then He delivers
it to Lord Brahma.
At the end of the millennium, ignorance personified assumed the form
of a demon, stole all the Vedas and took them down to the planet of
Rasatala. The Supreme Lord, however, in His form of Hayagriva
retrieved the Vedas and returned them to Lord Brahma when he begged
for them. I offer my respectful obeisances unto the Supreme Lord,
whose determination never fails.
There are actually multiple accounts of demons stealing the Vedas and Sri Hayagriva recovering them, but since @jabahar has already addressed the incident detailed in the Mahabharatam, I have described the one from Srimad Bhagavatam.
In essence, Sri Hayagriva is the source of all divine knowledge, and worship of him is considered extremely auspicious for any academic pursuits.
Source for Translation and Transliteration