Both silver and gold are considered as very pure.. so what it means is that if they get sullied then purification is achieved back very easily.
And, that's why householders are also prescribed to take food in utensils made of gold or silver.
A house-holder should always take his food in golden or silver
vessels, or in these made of bell-metal, substituting the fragrant
leaves of the Palasha, Lodhra, or Padma for them in cases where they
would be unavailable. A yati or a Brahmachdrin should use the kind of
utensil which he thinks proper for his cult. (63-64) ---- Vyasa Smriti verses
As regards, whether they (gold and silver) can get impure, then yes:
The purification of gold and silver [is encompassed] by air and the
rays of the sun and moon. (44)
A woolen cloth is not sullied when marked with semen or touched by a
dead body. It becomes pure when the part [affected] is washed by water
and earth. (45) --- Angirasa Smriti verses
But, when it is said that, by mere air something gets pure, then that has to be considered as ever pure for all practical purposes.
However, other scriptures prescribe some purifying methods for silver as well:
When articles become sullied, they are purified in the following
ways––if they are metal, by scrubbing them with cowdung, earth, and
water, or with just one of them; if they are copper, silver, or
gold, by using an acidic cleanser; if they are earthenware, by
firing them; if they are wooden, by scraping them; ---- Baudhayana
Dharma Sutras 1.8.27
Gold is cleaned with just water, as also silver. Copper is cleaned with an acidic cleanser.----- Vashista Dharma Sutras 3.54
So, i don't think we can say that they (silver and gold) do not get polluted ever, but it is relatively a lot easy to remove impurities of them.