This avatar of Vishnu is described in Srimad Bhagavatam Canto 11 Chapter 13 known as the Hamsa avatar. The story of his manifestation is cited below.
Uddhava asks Krishna how he(as Vishnu) dispelled the doubts of the Sanatakumara.
The Sanatkumara once went to Brahma to inquire about the science of Yoga. Brahma was confounded by their questions as he was in the process of creation and meditated on Lord Vishnu for answers.
Vishnu appears in the avatar of a Hamsa (swan) and imparts the essential knowledge to them.
SB 11.13.15 — Śrī Uddhava said: My dear Keśava, at what time and in
what form did You instruct the science of yoga to Sanaka and his
brothers? I now desire to know about these things.
SB 11.13.16 — The Supreme Personality of Godhead said: Once, the
mental sons of Lord Brahmā, namely, the sages headed by Sanaka,
inquired from their father about the difficult subject matter of the
supreme goal of yoga.
SB 11.13.17 — The sages headed by Sanaka said: O Lord, people’s minds
are naturally attracted to material sense objects, and similarly the
sense objects in the form of desire enter within the mind. Therefore,
how can a person who desires liberation, who desires to cross over
activities of sense gratification, destroy this mutual relationship
between the sense objects and the mind? Please explain this to us.
SB 11.13.18 — The Supreme Personality of Godhead said: My dear
Uddhava, Brahmā himself, who is born directly from the body of the
Lord and who is the creator of all living entities within the material
world, being the best of the demigods, seriously contemplated the
question of his sons headed by Sanaka. The intelligence of Brahmā,
however, was affected by his own activities of creation, and thus he
could not discover the essential answer to this question.
SB 11.13.19 — Lord Brahmā desired to attain the answer to the question
that was puzzling him, and thus he fixed his mind on Me, the Supreme
Lord. At that time, in My form of Haṁsa, I became visible to Lord