Did the ancient rishis create these various gods, goddesses, devas etc. out of thin air, which later ended up in the puranas and other scriptures? In other words, are they simply a figment of our imagination?
An excellent question. Non-Hindus certainly find it difficult to understand the numerous Hindu Devatas.
Hindu Gods are not personifications of human ideals. They are also not states of higher consciousness. Nor are they personal. Nor are they figment of our imagination.
Sri Ramakrishna compares the various forms with icebergs floating in the infinite ocean of consciousness.
No one can say with finality that God is only 'this' and nothing else. He is formless and again He has forms. For the bhakta He assumes forms. But He is formless for the jnani, that is, for him who looks on the world as a mere dream. The bhakta feels that he is one entity and the world as another. Therefore God reveals Himself to him as a Person. But the jnani – the Vedantist, for instance - always reasons, applying the process of 'Not this, not this'. Through this discrimination he realizes, by his inner perception, that the ego and the universe are both illusory, like a dream. Then the jnani realizes Brahman in his own consciousness. He can not describe what Brahman is.
Do you know what I mean? Think of Brahman, Existence-Knowledge-Bliss Absolute, as a shoreless ocean. Through the cooling influence as it were, of the bhakta's love, the water has frozen at places into blocks of ice. In other words, God now and then assumes various forms for His lovers and reveals Himself to them as a Person. But with the rising of the sun of knowledge, the blocks of ice melt. Then one doesn't feel any more that God is a Person, nor does one see God's forms. What He is can not be described. Who will describe Him? He who would do so disappears. He cannot find his 'I' anymore.
The Gospel of Sri Ramakrishna, The Master with the Brahmo Devotees (I), October 28, 1882
Swami Harshananda has a good discussion about the nature of Hindu Gods in his book ‘Hindu Gods and Goddesses’. I am posting excerpts from his essay to explain the nature of Hindu Gods..
The variety of [Hindu] deities is as fascinating as it is bewildering. However, as long as we do not forget that the divine form we worship is an embodiment of the attributes that reveal the Supreme Principle in one way or the other, we are on safe ground. ….
One more thing: This Reality, which is solidified consciousness as it were, can and does assume the forms of these various deities described here, in response to the devotees who supplicate it. That innumerable mystics and seers have realized these forms is proof enough. Hence these deities are not just symbols but real.
Siva is not just the god of destruction, dwelling on the Himalayas or the cremation ground. He is the embodiment of renunciation and destruction of all evil. He is the personification of contemplation and divine consciousness. He is ‘the one Brahman, without a second, the All’ (Skanda Purana 184.108.40.206)
Is Visnu merely the lord of protection and preservation? He is the embodiment of the divine Principle that permeates the entire universe in which the world-play of creation, preservation and dissolution is enacted. He ‘abideth in all.’ He is ‘all’. He assumes all forms (vide Vishnu purana 1.12.71)
Similarly with the Mother Divine. Whether She is worshipped as the goddess of prosperity or propitiated out of fear for Her deadly dance of destruction, She is always the Power Supreme, the same as Brahman.
Notwithstanding all our logic and explanations, there still are academicians steeped in Western traditions of comparative religion who discover fetishism or polytheism or henotheism or other ‘isms’ in our concept of gods and goddesses. These explanations or theories may hold good in the case of the Semetic religions but not with the religions of Aryan or Vedic origin.
..there is in Hinduism no polytheism as understood by the western thinkers. No doubt there is a tendency occasionally to exalt one god over all the others. This is due to Istanistha or singular devotion to one’s own Chosen Deity and hence, cannot be dubbed as ‘henotheism’ as Max Mueller does.
The statement of Sri ramakrishna that there are several ice-bergs in the boundless ocean clinches this issue very well. All the ice-bergs as also the ocean itself, are all WATER only! Once this Vedantic background is grasped, the idea of polytheism vanishes completely. The worship of Siva, Sakti or Visnu becomes the adoration of the one Supreme Being who is Personal-Impersonal. The Bhagavad Gita (4.11; 7.21) states the same truth in an unmistakable language.
This simile of the Supreme God or the Godhead as the infinite ocean can now be looked upon from a slightly different angle, useful to the Sadhakas or spiritual aspirants. The various gods can then be considered as Its waves. We ordinary mortals that we are, with extraordinary attachment to our bodies and minds, are like bubbles. It is only when the bubble gets attached to the wave that it becomes conscious of its unity with the ocean.
Hindu Gods and Goddesses, From Gods to Godhead, by Swami Harshananda
Yes, the gods of the Hindu pantheon „are simply a figment of our imagination“. Of course this characterization holds for any gods, for any ghosts, demons, witches and also e.g., for Santa Claus.
Why? Otherwise it is very difficult to account for the contradicting answers to the question who is the highest deity, who is subordinate? How to consider Krishna an avatar of Vishnu or vice versa? How do the gods of the different religions relate to each other?
Why do people in India mostly adhere to one of the Hindu gods while most people in the Islamic world adhere to Allah, why do the Jewish people adhere to Jahwe and why do most people in Europe adhere to the Christian god alias Jahwe? In general, the faithful of a particular god follow the creed and the tradition of the community in which they have been socialized.
I consider the most simple hypothesis that all these god figures are fictions, not comparable to historical persons with a well-determined individuality. This hypothesie avoids all the unsolvable, rational and also logical problems connected with the opposite hypothesis about the existence of a pantheon.
I am well aware that the issue raised in your question cannot be decided by just the few lines of my answer. But a well-grounded answer can be obtained by studying some fundamental works from the sociology of religion. A good start could be „Sigmund Freud: The Future of an Illusion (1927)“.