He used the Chakra a lot and also his bow. I can't think of an occasion when he used his mace either as himself or an avatar.
Yes, Lord Vishnu used his Kaumodaki mace to kill a demon name Andhaka. The story is mentioned in Devi Bhagavata, Book 5, Chapter 6.
Wanting to kill Andhaka, Janârdana became infuriated, and, drawing his bow made of horn, call S'ârnga, shot at him arrows after arrows. The Dânava cut off all those arrows to pieces with his own mass of arrows. Then, becoming very angry, he shot fifty sharp arrows at Hari. Vâsudeva quickly made all those arrows useless and hurled Sudars'ana Chakra with thousand spokes on the Dânava with great violence. Andhaka thwarted this with his own discus and shouted aloud with such a great force that all the Devas became confused and confounded. Visnu's Chakra being baffled, the Devas became distressed with grief and the Dânavas got elated. Seeing the Devas thus grieved, Visnu held aloft his Kaumodakî Gadâ (club) and came hurriedly before the Dânava. Hari struck then with his Gadâ on the Dânava's head whereon he fell senseless on the ground.
The battle between Hirnayaksha and Varaha at the sea floor was fought with maces. I presume Varaha used His mace Kaumodaki. There is no mention of Varaha using the Sudarshana Chakra to kill Hiranyaksha in Srimad Bhagavad Mahapurana and the description iss of Him armed with the conch, discus and mace.
Did Vishnu ever use his mace Kaumodaki?
Following are some of the references of Purāṇas:
Lord Vishnu did use his mace as VarAha Avatara to kill the demon Hiranyaksha. It is mentioned in the 3rd Canto Chapter 18 Verse 17 of Śrīmad-Bhāgavatam (Bhāgavata Purāṇa) :
ततश्च गदयारातिं दक्षिणस्यां भ्रुवि प्रभु: ।
आजघ्ने स तु तां सौम्य गदया कोविदोऽहनत् ॥ १७ ॥
tataś ca gadayārātiṁ
dakṣiṇasyāṁ bhruvi prabhuḥ
ājaghne sa tu tāṁ saumya
gadayā kovido ’hanat
Then with His mace the Lord struck the enemy on the right of his brow, but since the demon was expert in fighting, O gentle Vidura, he protected himself by a maneuver of his own mace.
And from the same episode in 3rd Canto Chapter 19 and Verse 3 of Śrīmad-Bhāgavatam (Bhāgavata Purāṇa)
सा हता तेन गदया विहता भगवत्करात् ।
विघूर्णितापतद्रेजे तदद्भुतमिवाभवत् ॥ ३ ॥
sā hatā tena gadayā
tad adbhutam ivābhavat
Struck by the demon’s mace, however, the Lord’s mace slipped from His hand and looked splendid as it fell down whirling. This was miraculous, for the mace was blazing wonderfully.
Also in Viṣṇu-Purāṇa Ansh 5, Chapter 22, Verse-6 in the battle with Jarasandha, it is mentioned as:
अनन्तरं हरेश्शार्ङ्गं तूणौ चाक्षयसायकौ ।
आकाशादागतौ विप्र तथा कौमोदकी गदा ।।६।।
हे विप्र! हरिके स्मरण करते ही उनका शार्ङ्ग धनुष, अक्षय बाणयुक्त दो तरकश और कौमोदकी नामकी गदा आकाशसे आकर उपस्थित हो गये ।।६।।
O Brahmana! Hari's Sharanga Bow with 2 inexhaustible quiver and his Kaumodaki mace came from the sky and stood before him.
इत्युक्त्वा भगवाञ्छाल्वं गदया भीमवेगया ।
तताड जत्रौ संरब्ध: स चकम्पे वमन्नसृक् ॥ २० ॥
ity uktvā bhagavāñ chālvaṁ
tatāḍa jatrau saṁrabdhaḥ
sa cakampe vamann asṛk
Having said this, the furious Lord swung His club with frightening power and speed and hit Śālva on the collarbone, making him tremble and vomit blood.
गदयाभिहतोऽप्याजौ न चचाल यदूद्वह: ।
कृष्णोऽपि तमहन् गुर्व्या कौमोदक्या स्तनान्तरे ॥ ८ ॥
gadayābhihato ’py ājau
na cacāla yadūdvahaḥ
kṛṣṇo ’pi tam ahan gurvyā
Although hit by Dantavakra’s club, Lord Kṛṣṇa, the deliverer of the Yadus, did not budge from His place on the battlefield. Rather, with His massive Kaumodakī club the Lord struck Dantavakra in the middle of his chest.