It is doubtful if Sri Krishna and Sri Hanuman had ever met, as per Mahabharata.
Varuna gave Arjuna the great bow Gaandiva and a Chariot, whose banner is a large ape, before commencement of Khandava vana dahana.
Varuna replied unto Pavaka, saying, 'Well, I am giving them.'
He then gave that wonderful jewel of a bow that was endued with great
Sri Hanuman did not receive any shrap or curse, due to which he forgot about his strength.
The authentic text on Ramayana is Valmiki's Ramayana. It does not contain any such incident.
We can notice that in the entire Kishkinda Kanda and in Yuddha Kanda, Sri Hanuman acts at the orders of Sri Rama, Sugreeva, Jambavanta, and will not show his strength ...
Rather than the size, please focus on the arrangement of deities. There is nothing wrong in keeping a large idol of Hanumanji if you have a big house!
Couple of sites for your reference: