Is there any mention in the scriptures to regularly visit temples?
As per this chapter of Skanda Purana:
Sage Lomesh told the assembled sages--- One who brooms the courtyard of Shiva temple" attains to Shiva loka. Those who make offerings of incense to Lord Shiva not only liberate himself from the bondage of this world but also both his 'Kulas'. One who gets constructed a Shiva temple or renovates it, attains incomparable virtue. One who chants the holy mantra 'Om Namah Shivay' daily and begins his day by visiting Shiva temple in the morning becomes absolved of all his sins.
So visiting temple is recommended but it's not mandatory. However, most of the Puranas recommend visiting pilgrimage/holy places which includes temples as well.