According to Rāma, in the words of Vālmīki, if a person is capable of helping a person in danger and yet does not do his or her part will lose all the accumulated puṇya to the helpless person and will also amass enormous amount of pāpa.
स चेद् भयाद् वा मोहाद् वा कामाद् वा अपि न रक्षति || ६-१८-२९
स्वया शक्त्या यथा तत्त्वम् तत् पापम् लोक गर्हितम् |
If he does not protect rightly through his strength, by fear or by ignorance or by desire, it is a sin to be reproached by the world.
नष्टः पश्यतस् तस्य रक्षिणः शरण आगतः || ६-१८-३०
आदाय सुकृतम् तस्य सर्वम् गच्चेद् अरक्षितः |
If having not been protected, a refugee dies before the eyes of a man who is able to protect him, the former takes along all his moral merit and goes.
एवम् दोषो महान् अत्र प्रपन्नानाम् अरक्षणे ||६-१८-३१
अस्वर्ग्यम् च अयशस्यम् च बल वीर्य विनाशनम्
In not protecting thus, the persons who take refuge, there is a great blemish involved in it. It does not bestow heaven. It destroys reputation. It devastates strength and valor.
EDIT:
On this topic, this is what Vidura says during Draupadī's disrobing episode:
Ye kings, Vikarna hath answered the question, according to his own knowledge and judgment. Ye should also answer it as ye think proper. Knowing the rules of morality, and having attended an assembly, he that doth not answer a query that is put, incurreth half the demerit that attacheth to a lie. He, on the other hand, who, knowing the rules of morality and having joined an assembly answereth falsely, assuredly incurreth the sin of a lie.
Vidura then goes on to cite a conversation between Prahlāda and Kaśyapa according to which, one 1/2 of the pāpa of the actual act goes to the account of the head of the assembly, 1/4th into the perpetrator's and the remaining 1/4th into that of the rest who are a silent witness to the whole thing.
Kasyapa thus asked answered.--'He that knoweth, but answereth not a question from temptation, anger or fear, casteth upon himself a thousand nooses of Varuna. And the person who, cited as a witness with respect to any matter of ocular or auricular knowledge, speaketh carelessly, casteth a thousand nooses of Varuna upon his own person. On the completion of one full year, one such noose is loosened. Therefore, he that knoweth, should speak the truth without concealment. If virtue, pierced by sin, repaireth to an assembly (for aid), it is the duty of every body in the assembly to take off the dart, otherwise they themselves would be pierced with it. In an assembly where a truly censurable act is not rebuked, half the demerit of that act attacheth to the head of that assembly, a fourth to the person acting censurably and a fourth unto those others that are there. In that assembly, on the other hand, when he that deserveth censure is rebuked, the head of the assembly becometh freed from all sins, and the other members also incur none. It is only the perpetrator himself of the act that becometh responsible for it.
EDIT 2:
Manusmṛti 8.13 also supports the idea that an influential person who chooses to remain silent amidst injustice accumulates sin. Medhātithi, however, adds that a weak person whose opinion is of not much value and fears retaliation by his superiors can simply walk away from the scene or remain silent (without incurring any sin).
One should either not enter the Court at all, or he should speak out what is equitable; one who either speaks nothing, or speaks falsely, becomes tainted with sin. — (8.13)
Medhātithi's commentary:
'Speaks nothing'; – i.e., he who remains silent when another person is committing an injustice, ... – 'becomes tainted with sin' – i.e., comes to partake of the sin. Hence the man should not entertain the hope that – 'it is another judge who is judging wrongly, and he may incur sin, I am only sitting silent and indifferent, why should I be affected by the sin?'
...
This has been taken to imply that when even an unauthorized person happens to be present, if he finds that the judges are acting wrongly, he should not remain silent. To this end who have the assertion – 'Authorized or unauthorized, the man who knows what is just should always speak out' (Nārada 2.2). If he fears molestation at the hands of the king's officers as to why he should speak, when he is not authorized to do so, – then he should go away from that place. In support of this we have the following assertion – 'When a wrong is being inflicted upon a weak person, if one does not save him from it, he incurs sin, only if he has the power to save him' (Gautama, 21.19). – (13)