The oldest scriptural references that I could find after going through many books are from the Padma and Skanda Purans. Sharing the relevant excerpts below.
According to the Uttar Khand of the Padma Puran:
Śrī Śiva said: O Kārtikeya, on the thirteenth day of the dark half of Kārtika, a man should offer a light to Yama outside (his house). Thereby untimely death is avoided. “May the Sun’s son, with Death having a noose in his hand and with his wife, be pleased due to this offering of the light”.
Those who are scared of sins, should necessarily bathe at moon-rise on the fourteenth day of the dark half of Kārtika. He, being careful, should bathe early in the morning on the fourteenth day pierced (i.e. mixed) with the previous day of the dark half of Kārtika. Lakṣmī (resides) in oil and Gaṅgā in water. He who would bathe in the morning on Caturdaśī in Dīpāvalī does not see Yama’s world. To destroy (i.e. to avoid) hell, he should whirl (round himself) apāmārga, tumbī, prapunnāṭa, vāhvala (twigs) while bathing. “O Apāmārga, being with a clod of furrowed land and with thorn and leaf, remove my sin, when repeatedly whirled.” He should whirl over his head apāmārga and prapunnāṭa. Then with (i.e. by reciting) Yama’s names, he should offer libations of water. “Salutation to Yama, Dharmarāja, Mṛtyu, Antaka, Vaivasvata, Kāla and Sarvabhūtakṣaya (Destroyer of all beings), Audumbara, Dadhna, Nīla, Parameṣṭhin, Vṛkodara, Citra, Citragupta.” Having worshipped the deities, he (i.e. the king) should offer a light to Naraka.
Then at the break of night he should offer pleasing lights at the temples of Brahmā, Viṣṇu, Śiva etc., and especially at apartments on the tops of the houses, sanctuaries, assembly halls, rivers. ramparts, gardens, wells, streets, pleasure-groves near houses, stables and abodes of elephants also.
O Kārtikeya, having, like this, bathed in the morning on the new-moon day, and having devoutly worshipped and saluted deities and dead ancestors, and having offered the pārvaṇa śrāddha with curd, ghee, milk etc., he should feed brāhmaṇas with various kinds of food, and apologise to them. Then, O dear one, in the afternoon the king should gratify the citizens. Having chitchatted with them, honoured them he should converse with them. O Kārtikeya, for one year love is generated among those who talk.
So the 13th and 14th days are primarily dedicated to the worship of Yamaraj as per the Padma Puran while the night of Amavas is for Lakshmi Pujan:
Before Viṣṇu has got up, he should awaken Lakṣmī through the women. When a man has awakened Lakṣmī at the time of her getting up through a good woman, for one year Lakṣmī does not leave him. The haters of gods (i.e. the demons) scared of Viṣṇu obtained fearlessness from the brāhmaṇas, after knowing (that Viṣṇu) was asleep in the Milky Ocean, and Lakṣmī had resorted to the lotus. “You are the lustre, Śrī Ravi (i.e. the Sun), the Moon, the Lightning, the Golden Star; the lustre occurring in the lustre of the lamp is the lustre of all the lustres. That Lakṣmī that lives on the earth, in the cowpen on the auspicious day of Dīpāvalī in Kārtika, may grant me a boon.” He who gets victory first (passes) the year happily.
When the night passes like this, and people have their eyes half-closed, Alakṣmī is driven out of the courtyard of the house by the joyful city-women by means of sounding the musical instruments and drums. In the case of a defeat (in the game of dice) opposite will be the case.
Next, the day of Pratipada is mentioned to be associated with Govardhan Puja and Bali Maharaj both:
On the first day, when the sun has risen, Govardhana should be worshipped in the morning and one should play the game of dice at night. Then cows should be decorated; and they should not be used to carry (loads etc.) or should not be milked. “O Govardhana, O support of the earth, O protector of Gokula, O you who were lifted by the hand of Viṣṇu, give (us) crores of cows. That Lakṣmī who remains in the form of the cow of the regents of the quarter, and that carries ghee for the sacrifice, may remove my sin. May cows stand before me. May cows be behind me. May cows be in my heart. I live among cows.” Thus worship of Govardhana (should be done).
The tradition of gifting new clothes and other desirable material to one's family and friends also finds mention here along with the worship of cattle:
Having pleased with sincere devotion deities and good men, he should please others by giving food and (should please) the learned men by offering (i.e. speaking gentle) words. (He should please) the inmates of the harem by (giving them) garments, tāmbūlas, lights, flowers, camphor, saffron, food and eatables of superior and inferior quality. The king should please the chief of the village with gifts and vassals with wealth, and the hosts of foot soldiers good neck-ornaments and bracelets. The king should also please his ministers and his own people separately. Then having properly gratified wrestlers and actors, so also the bulls and large oxen fighting with one another and other soldiers and foot soldiers who are well-adorned being seated on a raised seat, he should see the actors, dancers and bards, and should cause to fight and roar the cows and buffaloes etc. which (he possesses). By means of sounding words (of calls) and responses he makes the cows attract their calves.
Next comes a worship of the guardian-of-roads referred to as Margapali:
Then, O Kārtikeya, he should put up a divine arch over the path in the eastern direction in the afternoon, tied to a pillar of the fort or a tree; (it should be made) of kuśa (grass) and should have many hanging strings of flowers. Observing many horses and elephants, he should take them and bulls with cows, so also he-buffaloes and she-buffaloes furious with bells to the base of the arch. Through the best brāhmaṇas who have offered a sacrifice, he should put up the arch. Then, he, of a good vow, should make obeisance by (reciting) this hymn:
“O mārgapāli, salutation to you who give happiness to all the world.” O Kārtikeya, at the base of the mārgapāli cows and great bulls go. Kings, princes and especially brāhmaṇas on crossing the mārgapāli, become free from disease and happy.
At night Bali Maharaj, the future Indra should be worshipped in a manner that reminds us of the Onam celebrations in Kerala:
Having done all this, at night he should actually worship Bali, the lord of demons, in a circle made on the ground. Having drawn, with five colours, (the picture of) Bali, the lord of demons, complete with all ornaments, accompanied by Vindhyāvali, surrounded by the demons Kūṣmāṇḍa, Maya, Jambhāru, Madhu, and with his entire face delighted, and with a crown and bright earrings, and again (drawing the picture of) the lord of demons with two arms in a small or big chamber in his own house, he should then worship it. The lord of kings, who, being pleased and with his ministers and priests, and with his mother, brothers, and kinsmen, worships (the demon-lord) with lotuses, red lotuses, flowers, white lotuses and blue lotuses, with sandal, flowers, offerings of eatables with milk, jaggery and sweetened milk, with liquor, flesh, wine, things eaten by licking or by sucking, (other) eatables and offerings, and by (reciting) this hymn, gets happiness during that year: “O king Bali, O lord, O son of Virocana, salutation to you. O future Indra, O enemy of gods, accept this worship.”
Having thus offered the worship and keeping awake at night, he should make (others) keep awake at night with actors, dancers, singers, and make people place (the image of) king Bali with white rice inside the house on a couch, and should worship him with fruits and flowers. O Kārtikeya, everything there should be done with reference to Bali. The sages, the seers of truth say that all those (objects) that are inexhaustible (come to him). Whatever small or big gift is made here, all that would be inexhaustible, auspicious, and would please Viṣṇu. May all the untractable practices of those men who do not worship you, Bali, at night, come to you.
The reason for this worship is also mentioned in the text as the directive of Lord Vishnu:
O child, Viṣṇu himself, being pleased, has granted this great festival to Bali who had obliged the demons. O Kārtikeya, since then (this festival of) Kaumudī has commenced (for) ever. It melts all troubles and destroys all difficulties. It removes the grief of people, it fulfils desires, and brings about wealth, nourishment and happiness.........If he enjoys (the festival) he enjoys the year; if he is happy, the year will be happy. Therefore, good men should joyfully celebrate Kaumudī. This day in Kārtika is said to be sacred to Viṣṇu and demons.
The fifth day of festivities is the celebration of Bhai-Duj which is also clearly mentioned in the text as being associated with Yama again:
A man, having bathed on the Bhānujā on the second day of Kārtika, should worship Yama in the morning. (Thereby) he does not see Yama. O Śaunka, formerly, on the second day of the bright half of Kārtika, Yama was fed and honoured by Yamunā in her house. On the second day a great gift (is given). Denizens of hell are gratified. They, separated (i.e. freed) from sins, are free from all bondage. They all, being praised, remain pleased as they like. This great festival, giving pleasure to Yama’s region, takes place for them. Therefore, this Yamadvitīyā is well known in the three worlds, and so the wise should not eat at home (on this day). They should affectionately take a nourishing meal at their sister’s hand; gifts should duly be given to sisters.
Then along with the gift of golden ornaments and garments and honouring (their sister) they should eat from the hand of their sister of whole blood. The meal from the sister’s hand should be taken on all (these days). It is nourishing. Yama is gratified on the second day of bright half of Kārtika. Yama is mounted upon the seat (i.e. back) of a he-buffalo. The lord holds a staff and a mallet. He is surrounded by his jubilant servants. Obeisance to him of the nature of Yāmyā. For those who have pleased their sisters whose husbands are alive, with gifts of garments etc. there is no quarrel (with anyone), nor any fear from their enemy during the year. O sinless one, O my son, I have told you the entire account along with its secret. It is blessed, gives success, increases the (span of) life, and is the means of righteous acts and enjoyments. Since on this day god Yamarāja was well-fed by Yamunā with a sister’s affection (for her brother), therefore he who, on this day, eats from the hand of his sister, gets wealth and excellent riches.
The Skanda Puran account is very similar but additionally it mentions the worship of calves on the twelfth day:
Kṛṣṇa described to Dharma (i.e. Yudhiṣṭhira) (the holy day) named Vatsadvādaśī. The twelfth lunar day must be in conjunction with the time of sunset in worshipping a calf.
“On the first day, worship of a calf is to be performed under the shade of a banyan tree. One should apply sandal paste etc. to a domesticated milch cow along with its calf of the same colour. Then he should worship them with flower-garlands. On that day, O Yudhiṣṭhira, the devotee should abstain from foodstuffs deep-fried in oil, cooked in pots, cow’s milk, cow’s ghee as well as mixture of curds and milk.
At the time of sunset, either one ghaṭikā (24 minutes) before or one ghaṭikā after the disc of the sun is half-set, the rite of Nīrājana should be performed and it should be ascertained whether the omens are auspicious or inauspicious. At the outset many lamps are lit. They are then placed in golden vessels etc. Then the rite of Nīrājana should be performed and the omens observed. Then he should take all the lamps and place them facing the North. The important lamps are said to be nine. He should have others too.
Interestingly the Nirajan or Aarati is to be performed by men for all those around them including:
After consecrating it with the above incantation, the Nīrājana rite should be performed in due order to the following: Devas at the outset, then Brāhmaṇas, elephants, horses, elders, excellent people, lower people and the women, the chief of whom is mother.
The Skanda Puran also mentions the use of firebrands in Diwali, the birth of Goddess Maharatri, and lighting of lamps on the Amavasya:
O great sages, a goddess called Mahārātri was born (appeared) on the fourteenth day (of the dark half of Āśvina). Hence those who are devoted to worship of Śakti should celebrate her festival.
When the sun is in Libra, on the nights of Caturdaśī and new-moon day, men should celebrate the festival of “showing the path unto the Pitṛs” with firebrands in their hands. The dead men and ghosts who are in hell see the path, due to this Vrata always. No doubt need be entertained in this respect by leading sages.
So to summarise, the festival of Diwali seems to have been associated with Yamaraj, Pitris, cattle, Devi and the future Indra Bali. It is hard to say which of these two Puranas are the oldest but both have very similar descriptions of the festivities. The texts highlight:
- Lighting of lamps for Yama
- Early morning Abhyanga Snana
- Decorations with lamps in the homes and public places across the city,
- Drawing of Rangoli etc. especially to welcome Bali,
- Exchanging of gifts with people of all backgrounds,
- Worshipping of cattle & nature (in the form of Govardhan mountain),
- Use of firebrands to show the way back to Pitris
- Visiting sisters and showering them with gifts who in return prepare delectable food for their brothers.
Clearly its a beautiful festival with many varied traditions that aims to bring together all friends and family!