Many aspects of Hindu rituals are closely tied with astrology, star gazing and charts. How and when did this start to happen?
Astrology was a defining aspect since the beginning of Hinduism. The core of Hinduism is the Vedas and in one way, the core of the Vedas are the rituals and yajnas (sacrifices). The Vedas define particular moments, planetary positions etc. when the rituals or yajna should be started. Hence, since the prehistoric times rishis (sages) and munis started star gazing.
The thing is, to understand the Veda and its mantras properly and to understand the time table and planetary positions properly, knowledge of certain other things like grammar, meter (to chant the mantras), astronomy (to know the time of ritual), etc. are also required. Hence, these (six in number) are known as the limb of the Vedas (Vedanga):
तत्रापरा रुग्वेदो यजुर्वेदः सामवेदोऽथर्ववेदः
शिक्षा कल्पो व्याकरणं निरुक्तं छन्दो ज्योतिषमिति |
अथ परा यया तदक्षरमधिगम्यते [Mundak. Up. - 1.1.5]
Of these (Knowledge), Rig Veda, Yajur Veda, Sam Veda, Atharv Veda, Sikhsa (phonetics), kalpa (ritual), vyakaran (grammar), nirukta (word definitions), chanda (meter), jyotisha (astrology) are lower. And the higher than these is the knowledge through which the imperishable is attained.
So astronomy is as old as the Vedas, but later on sages and scholars like Parashara, Garga, Varah Mihir, etc. formulated more precise rules and helped gain it today's form which are used to calculate birth charts, forecast future, etc.
I don't know whether it count's as a proper reference, but in the Shri Krishna serial of Ramananda Sagar I had heard that Garga was the sage who first created the planetary chart system (in the episode where he names Krishna). The Vishnu Puran also states Garga as the ancient rishi who learned jyotishi from Sesha (Ananta Naga):
yamārādhya purāṇarvirgagoṃ jyortīṣi tattvataḥ
jñātavānsakalaṃ caiva nimittapaṭhitaṃ phalam [VP - 2.5.26]
Worshipping him (Sesha), the ancient rishi Garga obtained the complete knowledge of jyotishi (astrology) and the auspicious and inauspicious effects of planetary positions.