Scientists claim that the underlying notion of a theory is falsifiability i.e. the ability to disprove a statement by performing an experiment laid out by the scientist.
The criterion of falsifiability does not come from science itself or from any scientist, but from the so-called philosophy of science. It was first propounded by Karl Popper, who was not a scientist, but a philosopher. Let us examine falsifiability in detail, by taking into account some well-known scientific theories.
Closely related to the idea of falsifiability is the ability of a scientific theory to make predictions. For example, the General Theory of Relaitivity of Einstein makes a prediction that time is affected due to gravity and it involves gravitational red shift. When Einstein propounded this theory, there was no experiment that was conducted before, that actually showed this. Therefore, this is an example of a prediction from General Relativity. (For simplicity, let us assume this was the only prediction from Einstein’s General Relativity). This prediction, of course, was confirmed by later experiments.
However, what would have happened if a particular experiment had contradicted the prediction? Because the theory was new and completely non-intuitive, perhaps it would have immediately fallen into disrepute? May be. More likely, the experiment would have been repeated by another independent set of researchers, and if this repeated experiment fails to confirm the theory, the theory would seriously start losing steam. At this point, the theory would be just a few more repeated experiments away from being completely discarded. This, is an example of falsifiability.
Yes, the experiments need to be done properly. That is the reason why one single experiment alone cannot be used to completely discard a theory.
The above is a very simplistic example though.
A better example would be classical physics vs moden physics. Both are paradigms. Both paradigms have good experimental evidence and modern physics is closer to the truth. But that does not mean that classical physics is wrong. It is still a good approximation at low velocites and macroscopic levels. Modern Physics, as the more correct understanding of nature, did not replace classical physics overnight, or due to a single experiment supposedly “falsifying” classical physics. It was a slow and gradual process, which started from the realization that there is something about black body radiation that was not properly explained by classical physics. This realization came about probably by the end of 19th century. However, it is not as if classical physics was immediately disproved due to this realization. (Classical physics is still considered a valid approximation even today). The discoveries in quantum mechanics were gradual and took upto the late 1920s to become a coherent theory.
So the understanding that theory X is falsified overnight and/or replaced by theory Y is not realistic. To reiterate this point, there have been recent claims, later proved false, (https://en.wikipedia.org/wiki/Faster-than-light_neutrino_anomaly) about discovery of faster than light travel. It is not as if Physicists dumped special theory of relativity overnight due to this claim.
So, is falsifiability really a yard stick for scientific theories? May be yes, to some extent in the past. But in the present day, where experimental science is probably lagging behind theory, this may not deter researchers from pursuing their favorite candidate for a theory-of-everything. The best example of this today is the String theory, which currently has no significant predictions, but is an active research topic for about 40 years. What researchers in String theory are more likely to be concerned, at least currently, is mathematical aspects of the theory. Even when the theory of General Relativity was proposed, it was “made” to have certain mathematical characteristics before it came out.
However, there are other cases, where theories have been modified, either based on new observations or in order to explain some problems. An example is the big bang cosmological model, which did have predictions that were confirmed (microwave background raditation), but which also have undergone subsequent modifications (addition of inflation) later.
So long story short, even though scientific theories tend to be falsifiable in general, science rarely works by over-night dumping of a theory due to contradicting experimental evidence. Theories build up their reputation over time, as reliable predictors of nature, due to experimental evidence, which can be repeated independently. Of course, in the history of science, there have been theories that have been discarded due to contradicting evidence from experiments. One thing to note is that scientific understanding of nature is constantly evolving. There are many things currently which are not yet explainable by any scientific theory. Science acknowledges its current boundaries. However, there is no assumption that these boundaries are permanent and cannot be stretched with the passage of time. So there is no assumption that a phenomenon XYZ is completely and forever beyond science, even if science does not explain it currently.
This, then is the science side of things. What about religion?
Rishis claim that the underlying notion of a shastra is falsifiability i.e. the ability to disprove a statement by performing an experiment laid out by the rishi.
Actually, religion does make and has made many falsifiable claims. And many claims of religion have been falsified (similar situation as for many scientific theories, which have been falsified). Specific to the Hindu religion, I can think of geographical claims about the earth that are present in some puranas that have been falisified. Claims about Rahu or Ketu swallowing the sun or moon during the eclipses have also been falsified. (The reader can similarly note that many Abrahamic religious claims about creation happening about 4000 or 6000 years ago, or flat earth or earth being center of the universe, have similarly been falsified).
Ignoring other religions, and sticking to Hindu religion, a few thoughts will be shared here. It is often difficult to understand what exactly a particular religious claim is. For exmaple, when pointed out that Rahu or Ketu swallowing the sun or moon has been shown to be false, the response from defenders of religion might be of this sort- “These claims should be interpreted differently. Rahu and Ketu represent ignorance and sun or moon represent knowledge or mind and eclipse represents the overcoming of mind by ignorance etc.”
Where as, if science makes a claim that is later falsified, scientists in general do not have a problem acknowledging that the claim was false, the same amount of honesty is not seen in religion. Seldom do religions acknowledge that their claims are false, even if proved outrightly false. It is always a problem of interpretation and never a problem of the religious claim.
Which brings us to the most important difference between science and religion – scientists do not make claims of infallibility, while religious personalities are generally considered infallible by their followers.
Again, there is no necessity for applying science to doubt religious claims, nor is doubting religious claims a monopoly of atheists, agnostics or scientists. Some claims can be doubted by mere application of the most basic skepticism. The Ramayana has Rama, a human being (he is a human incarnation), living for thousands of years. Meanwhile, Ashwatthama, Parashurama etc – all human beings - are immortal. Even an average person will doubt such claims. Whether there was any time when humans lived for thousands of years? Is it possible for a human to be immortal? (Or is all this just a matter of interpretation? How would you interpret these claims away?) How do you verify/falsify such claims? You do meditation and realize that Ashwatthama is still alive? How does that work out? You will be able to see him? How would you verify that he is really Drona’s son and not an imposter? Perhaps this claim of Ashwatthama’s immortality is indeed falsifiable, but I think it is upto the religious people making this claim to show how this is falsifiable. In fact, believing such claims is more taxing on the average mind than questioning them.
Another religious claim is about references in Mahabharata of Arjuna shooting many arrows within a short span of time, without missing the target. Mind you, these are not supernatural claims. Implicit in Mahabharata is the assumption that Arjuna, by practice has achieved mastery over archery and never misses his target. An average human would marvel at this ability of Arjuna. There is no statistical randomness in Arjuna’s aim. Even a fair coin, when tossed a 1000 times, is very unlikely to give exactly 500 heads and 500 tails. But the arrows of Arjuna know no such statistical laws. They always hit the target with 100% certainty, no matter even if the target is small and at a large distance away (though within visible range). It is not even the case that the arrows land very close to the target. They are right on target. And it is not as if this mastery has been achieved due to any mantra-s. It is pure practice, which is why Ekalavya was also able to achieve it. The presence of such claims definitely raises eye-brows. Now how does anyone falsify the claim that it is possible for a human being to achieve this degree of perfection in archery? Mind you, as mentioned before, we are not even in the realm of the supernatural here.
Theory - Yagna produces rain
How to disprove it ? - Perform yagna. If rain doesn't pour, theory is disproved.
If I am sinful, or I refuse to perform yagna, or don't follow the thousands of rules/regulations required - all those are my faults. If I am faulty, and still claim that rishis are fake, no rishi will change his claim because of some faulty experimenter.
The question answers itself, or at least hints at the answer. Is the claim – Yagna produces rain, falsifiable? If I am sinful, my Yagna does not produce rain. Who is to verify whether or not I am sinful? Who is the person qualified to give certificate of sinfulness or otherwise? If such a person exists, he or she must be all-knowing or at least know everything about everyone, including the ability to read minds. And who will certify that this all-knowing person is all-knowing? Another all-knowing person who would further require a certification from some one else? It is not clear how exactly the claim – Yagna produces rain – is falsifiable. Again, as with most religious claims, this claim is very fluid and needs more clarity.
I can call any scientific statement ('Atoms exist' or 'Vaccines save lives') as 'non-falsifiable' because it is very hard or impossible for me to verify (doing Physics PhD/postdoc for 10 years or doing pharmaceutical double blind trials for 20 years).
Pharma companies don’t try to falsify vaccines. They try to find evidence that a vaccine works. The statistical null-hypothesis, in fact assumes that the vaccine has no effect. So the default position is that the vaccine is useless. It is upto the evidence to show that the vaccine has an effect. This is like many scientific theories. These theories need to build their credibility in the eyes of scientists and this process happens by many experimental confirmations.
A point was raised by OP about Arjuna's skill in archery being similar to Shakuntala Devi's skill of being a human computer. Response to this -
Compare apples to apples, not oranges. These two skills require different abilities.
There are examples of other human computers like Alexis Lemaire, Willem Klein.
Comparable to human computers in the field of archery are Olympic archery gold medalists. There exists no such archer with 100% accuracy of hitting a (fixed) target, leave alone hitting a moving arrow coming at you.