Atheism is not the pillar of Hinduism, but only a part of Hinduism. Moreover, the term nastika (atheist) means different in Hinduism than its usual meaning. Generally atheist means a person who doesn't believe in God, afterlife, etc. But in Hinduism atheist means a person who doesn't accept the authority of the Vedas (veda neendako nastika), that is, a person who doesn't accept words of the Vedas as the infallible truth. After that it is only optional whether he believes in the existence of God or not. So in Hinduism one may not believe God but still be called as a theist as long as he believes the Vedas.
Now you may wonder how it is possible for someone to believe the Vedas but not believe in God? Well, this is because the Vedas don not maintain a consistent theory about who or what the God is and even statements that are skeptic of existence of a Supreme Being are present in the Vedas. So how can be the truth ascertained? That's where logic and philosophy comes in. Sages have tried to reconcile the different message of the Vedas accepting different kinds of proofs and given their own conclusion regarding the existence of God. So out of the six primary schools of thought in Hinduism, three say God, the supreme being, doesn't exist and three say God exists.
Nyāya School of Thought - God exists
Sage Gautam was the founder of this school of thought and it says in favour of God's existence. It laid out foundations of logic and arguments basing upon which the truthfulness of a certain thing can be ascertained.
Mimasa School of Thought (purva mīmāṃsā) - God doesn't exist
Sage Jaimini was the founder of this school of thought. They accepted Vedas as the personal experiences of the ultimate reality by various sages in the form of sound patterns. Hence, Vedas not being the creation of any person are called as apauruṣeya. Just like there was no need for an author to compose the Vedas, they argue, there was no need for a God to create the universe.
Mimansakas hold the belief that the gods named in the Vedas have no independent or separate existence other than the mantras that mention their names. It is only the power of these mantras which is considered as the power of the God.
Vaishesika School of Thought - God doesn't exist
Sage Kaṇāda was the founder of this school of thought. Just like the modern scientist, they speculated the existence of an unseen force that binds the atoms together but never gave it any human like attributes. So from the perspective of their area of research, Vaiśeṣika School didn’t accept the existence of any personal God or God with attributes. But it can’t be said for sure that they rejected the possibility of His existence.
Sāṃkhya School of Thought - God doesn't exist
Sage Kapila’s sāṃkhya school of thought did not accept the doctrine of a creator God. It contributed many useful concepts to Indian philosophy but argued against the presence of a creator being. It holds primordial nature (pradhāna) as the root from which everything originates.
Yoga School of Thought - God exists
Sage Patanjali was the proponent of this school of thought. They believe in the existence of God along with all the other twenty five elements of existence postulated by the system of sāṃkhya.
Vedanta School of Thought (uttara mīmāṃsā) - God exists
Sage Badarāyana was the founder of this school of thought. This school of thought is widely known and it argues in favor of the existence of God.
So as you can see, believing God is only but optional depending upon the school of thought one follows and it is not limited to any varna or cast like Brahmins etc. Although it is another thing that each school of thought argues and tries to refute the tenets of the other and the Vedanta school of thought is the most popular one. It is however another thing that the Vedanta schools of thought also have varying philosophies.
Please Note: I only tried to answer in brief due to shortage of time as this is a very long concept. I'll add and update this answer with verses and quotations as soon as I get time (possibly tomorrow). I hope this post provides some insight in the mean time regarding atheism in Hinduism
Atheist is generally translated as nastika in Sanskrit or Hindi language. People have different idea regarding what nastika means the most common one being one who doesn't believe in God. Well, that's ok, but in Hinduism due to existence of different schools of thought (some of which don't even accept the existence of a creator God), the definition of nastika is one who rejects the authority of the Vedas. Some people (like Hindu who has commented below) out of love and admiration deny this definition to be of Hindu thought. But that's only their love speaking, the scriptures do speak of this:
nāstiko vedanindakaḥ [Manu Smrt. - 2.11]
sarvataḥ pāpadarśī ca nāstiko vedanindakaḥ [Mahabharat - 12.162.8]
vedavādāpaviddhāṃs tu tān viddhi bhṛśanāstikān [MB - 12.12.4]
- Know them for downright atheists that reject the declaration of the Vedas
There are many other instances where the nastikas have been mentioned and often their characteristics are given as ignorant people and who are sinful. But it is true that, this kind of people were not always there everywhere and their number has increased in the course of time . For example, during the time of Lord Rama there were no atheists in Ayodhya as the Ramayana describes:
kāmī vā na kadaryo vā nṛśaṃsaḥ puruṣaḥ kvacit
draṣṭum śakyam ayodhyāyām na avidvān na ca nāstikaḥ [VR- 1.6.8]
The lustful persons, the miserly, the unscholarly and the atheists were not seen in that city of Ayodhya anywhere.
Reagrding the philosophical systems rejecting God, it should be known as the Supreme Being. That is, the God with attributes. In Hinduism God is defined both with and without attributes. And philosophical schools like Samkhya deny the existence of any such creator God with attributes. They either don't deny or keep silent or describe it differently regarding the existence of the supreme entity. For example, the Vaisesika school calls the unknown force as adrishta rather than giving it God like attributes. Please see this wikipedia article for some more information as I am struggling with time to update this post.