The following is another perspective for this verse:
Does Heart mean the physical heart?
Actually the Heart mentioned by the text does not mean literally the physical blood pumping organ in the body.
Rather it means the spiritual heart.
This is what Sri Ramana Maharishi says in this matter [Reference]:
[The Heart] is the Centre of spiritual experience according to the testimony of Sages. The spiritual Heart-centre is quite different from the blood- propelling, muscular organ known by the same name. The spiritual Heart-centre is not an organ of the body. All that you can say of the Heart is that it is the very Core of your being, that [with] which you are really identical (as the word in Sanskrit literally means), whether you are awake, asleep or dreaming, whether you are engaged in work or immersed in Samadhi. (Ramana Maharshi, MG, 73.)
The Heart … is different from the blood vessel, so called, and is not the Anahata Chakra in the middle of the chest, one of the six centres spoken of in books on Yoga. (Ramana Maharshi, KOL, 150.)
This heart is different from the physical heart; beating is the function of the latter. The former is the seat of spiritual experience. That is all that can be said of it.
So what does the Heart as the center or cause of all knowledge?
So in other words Heart here does not refer to the physical heart, but the Heart of our being. The spiritual heart. It is actually the consciousness which is the core of our being, thus it is called as Heart since it is the cause of all causes.
Now why is it that knowledge finds its seat in the Heart?
This is because it is from consciousness that all knowledge arises. If you do not know that you exist, then you cannot know anything else. Thus this consciousness is the core and the cause of all knowledge. This consciousness is the Heart.
Sri Swami Krishnananda says in the Philosophy of Life, Theory of Perception as that:
The real seer and the senser of things is this consciousness which is at the background of the perceiving subject as its existence and essence. The ultimate knower of the world is an absolute being whose presence is established by the nature of knowledge itself.
What is the difference between Manas, Buddhi, Chitta and Prajna?
Buddhi - Buddhi means the intellect. It is the Mind engaging in analysis and discrimination.
Manas - Manas implies the emotional nature of the Mind. The Mind with its thoughts and emotions is called Manas.
Chitta - Chitta essentially means the subconscious mind. It is literally translated 'mind-stuff'. It is the Mind which penetrates into a very subtle realms.
Prajna - Prajna is pure wisdom. According to Mandukya Upanishad, the causal body (karana sarira) where the Mind-stuffs dissolves and only a mass of consciousness remains, is called Prajna. In this state there is an abundance of bliss called anandamaya. This is called Prajna. [Mandukya Upanishad, 5]
Essentially speaking, manas, buddhi and chitta all refer to the Mind. But they are used to highlight various modes or functions of the Mind. In essence there is only the Mind. This Mind when it engages in intellectual stuff is called Buddhi, when emotional, it is called Manas, and when generally speaking, it is called chitta. [Here is more information about these various faculties - Various modes of Mind]
All the best!