As far as I know (and I have not read all of the Vedas), Vedas don't explicitly say that human life span is 100 years, but there are definitely some Mantras in them that suggest that the Parama Ayu of a human being might be just that.
For example, see the following Mantra:
Pashyema sharada shatam (1)
Jivema sharada shatam (2)
sharada shatam (3)
Rohema sharada shatam (4)
Bhavema sharada shatam (6)
Bhuyema sharada shatam
Bhuyasi sharada shatam (7).
For a hundred autumns, may we see (1), For a hundred autumns may we
live (2), For a hundred autumns may we know (3), For a hundred
autumns may we rise (4), For a hundred autumns may we thrive (5),
For a hundred autumns may we be (6), For a hundred autumns may we
become (7), and even more than a hundred autumns. (8)
Atharva Veda 19.67
So, when the ParamAyu suggested by this Mantra is clearly 100 years, but living for more than that duration is not ruled out explicitly.
From what I have read so far, the Mantras almost always do indicate that complete lifespan is that of 100 years. For example, we often find such prayers in the Vedas like - "May we live for 100 years/winters/autumns" etc.
But the following Mantra from the Atharva Veda is also talking about 1000 years of life.
sahasra Ayuh sukritah chareyam || ------ For all thousand years of our life, may we be doing good work. (Atharva Veda 17.1.27)
Now, after the Vedas, the highest authority belongs to the Smritis (this rule is well known, but if you want I can cite sources) And, the figures that Manu Smriti gives are not "unbelievable" by any means.
ArogAh sarvasiddhArthash Charurvarshashat ayushah |
dishu hoshAm Ayur hrashati pAdashah ||
(Men are) free from disease, accomplish all their aims, and live four
hundred years in the Krita age, but in the Treta and (in each of) the
succeeding (ages) their life is lessened by one quarter.
Manu Smriti 1.83.
So, people used to live for 400 years in Krita, for 300 years in TretA, for 200 years in DwApara and for only 100 years in Kali Yuga.
And, we can rule these figures out only if they are in direct contradiction with the Vedas. Otherwise, we have to accept them as authoritative.
(BTW, the Parama Ayu, that the Agamas suggest is also 100 years.)