The Abhinava Anyatha Khyati is about illusory experience. Let me discuss Madhva’s theory.
Abhinava Anyatha Khyati
This brings us to the problem of illusory experiences. All Indian
systems of philosophy, including that of the Buddhists and Jains, have
their theory of illusions, which is technically called Khyati-vada. We
shall consider here only those that are relevant to the understanding
of Madhva’s theory known as Abhinava-anyatha-khyati, which is a
combination of the Buddhists’ Asat-khyati and the Anyatha-khyati of
the Logical Realists, the Nayyayikas. The Logical Realists explain the
traditional example of the illusory perception of silver in nacre
(mother of pearl) thus: ‘The silver is not in the shell but in the
bazaar (market). That silver of the bazaar is seen in the nacre and
confused with it.’ There are three elements involved in this - the
locus, silveriness or silver of the bazaar and the synthesis of these
due to various environmental causes. The illusion .. is removed when
the sublating perception, ‘This is not silver but nacre’ dawns on the
perceiver. The Nayyayikas maintain that what this sublating knowledge
negates is not the actual silver in the bazaar, but only the
identification of it with the ‘this’ or the locus of the nacre. Thus
they substantiate their realism even in illusory perception by
maintaining that the silver perceived is not false, as it is present
in the bazaar. ….
The Buddhist theory of Asat-khyati is meant not merely to show that
the traditional example of illusory experiences like the silver in the
nacre, the snake in the rope, and the water in the desert (in a
mirage) are illusory perceptions, but to explain on the basis of such
analogy that the whole of our experience of the world as outside is
illusory. …The idealist or Vijnanavadis (they say the the whole of the
world is simply a mental experience) extend this analogically to the
totality of the world experience. The Sunyavadi Buddhists consider all
experience to be like the circle seen when a torch is whirled round
and round at a rapid speed. The circle does not actually exist, yet it
is experienced. Similarly, non-existence is the nature of the world…
It is all Sunya or something falsely perceived as existing.
Now Madhva in the first place agrees with the Idealist Buddhists in
their total denial of actual existence for the object of illusion, say
silver, perceived outside. But he does not deny the locus, the ‘this’
where the false perception manifests. For in a false perception also
there is a real contact of the perceiving faculty, say sight, with a
real object as given in a distorted presentation. That distorted
presentation, which has no actual existence, is the illusion. He thus
agrees with the Nayyayika realists in holding that there is a contact
between the senses and an object, but it is not with the actual silver
of the bazaar but with the distortion of the locus, here in this
instance, nacre.
Bhakti Schools of Vedanta by Swami Tapasyananda
The text below is optional. How does Madhva's position differ from that of the Advaita?
Anirvacaniya khyati
The Advaita Vedantin is one with Madhva in refuting these Buddhist
theories from his Vyavaharika (empirical) point of view, and concurs
partially with them from the Paramarthika (metaphysical) point of
view.
Bhakti Schools of Vedanta by Swami Tapasyananda
The Advaitins say that the silver in the nacre is an illusion. However, the silver itself cannot be wished away as illusion as Buddhists do in the vyavaharika point of view. Another way to think of this is that the universe appears on Nirguna Brahman due to ignorance without affecting the Absolute. Being only an appearance caused by Ignorance, the manifold world is like a rope appearing as a snake or a nacre appearing as silver in defective situations. This means that the universe does not exist at all from the Absolute point of view but exists from the empirical point of view.
This creates a peculiar situation that the universe or the silver in the nacre both exists (vyavaharika level) and does not exist (Paramarthika level)! Thus the entire universe has an indeterminate existence.
He calls this intermediate reality as Sad-asad-anirvachaniya,
indeterminate existent non-existent experience. It is existent,
because it [nacre, universe] is actually experienced; it is also
non-existent because it is sublated. Sublated means that it is
stultified and revealed as having never existed really even when it is
being experienced as real. So it is a category forming a mixture of
both reality and unreality, and for this reason indeterminable
(anirvacaniya) in either way exclusively. This is called
Anirvacaniya-khyati.
Bhakti Schools of Vedanta by Swami Tapasyananda
Madhva’s criticism of Anirvacaniya-Khyati
His basic criticism is that to claim that something is both existent and non-existent at the same time, as Advaitins do, is only a verbal fiction. The existent is the contrary of the non-existent, without any middle ground between them. So they cannot coexist.
An Advaita experience in literature
When they returned to the room and Narendra heard the Master speaking
to others, he was surprised to find in his words an inner logic, a
striking sincerity, and a convincing proof of his spiritual nature. In
answer to Narendra's question, "Sir, have you seen God?" the Master
said: "Yes, I have seen God. I have seen Him more tangibly than I see
you. I have talked to Him more intimately than I am talking to you."
Continuing, the Master said: "But, my child, who wants to see God?
People shed jugs of tears for money, wife, and children. But if they
would weep for God for only one day they would surely see Him."
Narendra was amazed. These words he could not doubt. This was the
first time he had ever heard a man saying that he had seen God. But he
could not reconcile these words of the Master with the scene that had
taken place on the verandah only a few minutes before. He concluded
that Sri Ramakrishna was a monomaniac, and returned home rather
puzzled in mind.
During his second visit, about a month later, suddenly, at the touch
of the Master, Narendra felt overwhelmed and saw the walls of the room
and everything around him whirling and vanishing. "What are you doing
to me?" he cried in terror. "I have my father and mother at home." He
saw his own ego and the whole universe almost swallowed in a nameless
void. With a laugh the Master easily restored him. Narendra thought he
might have been hypnotized, but he could not understand how a
monomaniac could cast a spell over the mind of a strong person like
himself. He returned home more confused than ever, resolved to be
henceforth on his guard before this strange man.
The Gospel of Sri Ramakrishna, Introduction by Swami Nikhilananda
It is a possibility that the Advaita Anirvachaniya Khyati comes from this type of experience.