Your confusion is well warranted since the purpose of both the Yagnyas seems to be the establishment of overlordship over other kingdoms and make them accept the sovereignty of the king performing the ceremony.
However there are two major differences in the two rituals:
FIRST is that while Rajasuya gives access to heaven as a member of Indra's court, a 100 Ashwamedhs can even give you the post of Indra himself!
SECOND and the biggest difference is of the horse-sacrifice that is performed only in Ashwamedh & not Rajasuya.
In the SECTION XII of the Sabha Parva of Mahabharat Narad gives the following aim of the sacrifice when Yuddhisthir asks him about Indra's court:
As regards the Sabha of Sakra, however, thou hast named, O Muni, all the gods, the Gandharvas, and various Rishis. But, O great Muni, thou hast mentioned one and only one king, viz., the royal Rishi Harishchandra as living in the Sabha of the illustrious chief of the gods. What act was performed by that celebrated king, or what ascetic penances with steady vows, in consequence of which he hath been equal to Indra himself? O Brahmana, how didst thou also meet with my father, the exalted Pandu, now a guest in the region of the Pitris? O exalted one of excellent vows hath he told thee anything? O tell me all as I am exceedingly curious to hear all this from thee."
"Narada said,--'O king of kings, I shall tell thee all that thou askest me about Harischandra, I shall presently tell thee of his high excellence. He was a powerful king, in fact, an emperor over all the kings of the earth. Indeed, all the kings of the earth obeyed his sway. O monarch, mounted alone upon a victorious car adorned with gold, that king by the prowess of his weapons brought the whole earth with her seven islands under his sway. And, O monarch, having subjugated the whole earth with her mountains, forests, and woods, he made preparations for the great sacrifice called the Rajasuya.
And all the kings of the earth brought at his command wealth unto that sacrifice. All of them consented to become distributors of food and gifts unto the Brahmanas that were fed on the occasion. At that sacrifice king Harishchandra gave away unto all who asked, wealth that was five times what each had solicited. At the conclusion of the sacrifice, the king gratified the Brahmanas that came from various countries with large presents of various kinds of wealth. The Brahmanas gratified with various kinds of food and enjoyable articles, given away unto them to the extent of their desires, and with the heaps of jewels distributed amongst them, began to say,--King Harischandra is superior to all kings in energy and renown.--And know, O monarch, O bull of the Bharata race, it was for this reason that Harischandra shone more brightly than thousands of other kings. The powerful Harischandra having concluded his great sacrifice,became installed, O king, in the sovereignty of the earth and looked resplendent on his throne.
O bull of the Bharata race, all those monarchs that perform the sacrifice of Rajasuya, (attaining to the region of Indra) pass their time in felicity in Indra's company. And, O bull of the Bharata race, those kings also that yield up their lives without turning their backs on the field of battle attain to the mansion of Indra and live in joy with him. Those again that yield up their bodies after severe ascetic penances also attain to the same region and shine brightly there for ages.
It is interesting that Pandu then asks Narad to give the suggestion of this sacrifice to his son Yuddhishthir so that through the good merits of his son even he may attain heaven:
O king of the Kuru race, O son of Kunti, thy father Pandu, beholding the good fortune of Harischandra and wondering much thereat, hath told thee something. Knowing that I was coming to the world of men, he bowed unto me and said,--Thou shouldst tell Yudhishthira, O Rishi, that he can subjugate the whole Earth inasmuch as his brothers are all obedient to him. And having done this let him commence the grand sacrifice called Rajasuya. He is my son; if he performeth that sacrifice, I may, like Harischandra, soon attain to the region of Indra, and there in his Sabha pass countless years in continuous joy.
Then we have the requirements of the Rajasuya sacrifice mentioned in the SECTION XXXII of the Rajasuyika Parva:
Vaisampayana continued,--"Commanded by Krishna, the son of Pandu along with his brothers set himself upon collecting the materials for the performance of the Rajasuya sacrifice. And that chastiser of all foes, the son of Pandu, then commanded Sahadeva that foremost of all warriors and all ministers also, saying,-Let persons be appointed to collect without loss of time, all those articles which the Brahmanas have directed as necessary for the performance of this sacrifice, and all materials and auspicious necessaries that Dhaumya may order as required for it, each of the kind needed and one after another in due order. Let Indrasena and Visoka and Puru with Arjuna for his charioteer be engaged to collect food if they are to please me. Let these foremost of the Kurus also gather every article of agreeable taste and smell that may delight and attract the hearts of the Brahmanas.'
And Dwaipayana, O king, then appointed as sacrificial priests exalted Brahmanas that were like the Vedas themselves in embodied forms. The son of Satyavati became himself the Brahma of that sacrifice. And that bull of the Dhananjaya race, Susaman, became the chanter of the Vedic (Sama) hymns. Yajnavalkya devoted to Brahma became the Adhyaryu, and Paila--the son of Vasu and Dhaumya became the Hotris. And O bull of the Bharata race, the disciples and the sons of these men, all well-acquainted with the Vedas and the branches of the Vedas, became Hotragts. And all of them, having uttered benedictions and recited the object of the sacrifice, worshipped, according to the ordinance the large sacrificial compound. Commanded by the Brahmanas, builders and artificers erected numerous edifices there that were spacious and well-perfumed like unto the temples of the gods. After these were finished, that best of kings and that bull among men Yudhishthira. commanded his chief adviser Sahadeva, saying,--'Despatch thou, without loss of time, messengers endued with speed to invite all to the sacrifice.
All invited kings were a part of the ceremony and it ended with an Arghya as mentioned in the SECTION XXXV of the same section:
"Then Bhishma, O king, addressing king Yudhisthira the just, said, "O Bharata, let Arghya (an article of respect) be offered unto the kings as each of them deserveth. Listen, O Yudhishthira, the preceptor, the sacrificial priest, the relative, the Snataka, the friend, and the king, it hath been said are the six that deserve Arghya. The wise have said that when any of these dwell with one for full one year he deserveth to be worshipped with Arghya. These kings have been staying with us for some time. Therefore, O king, let Arghyas be procured to be offered unto each of them. And let an Arghya be presented first of all unto him among those present who is the foremost.
It is after this that they decide to honour Krishna and the entire Shishupal episode happens. I shall not go into that here as it would take away from the point under discussion.
As regards the Ashwamedh, the below text from SECTION LXXII of the Ashwamedha Parva give us an introduction to what was required for the ceremony:
"Vyasa said, 'Myself, O son of Kunti, and Paila and Yajnavalkya, shall without doubt, achieve every rite at the proper time. The rite of initiating thee will be performed on the day of full moon belonging to the month of Chaitra. Let all the necessaries of the sacrifice, O foremost of men, be got ready. Let Sutas well-versed in the science of horses, and let Brahmanas also possessed of the same lore, select, after examination, a worthy horse in order that thy sacrifice maybe completed. Loosening the animal according to the injunctions of the scriptures, let him wander over the whole Earth with her belt of seas, displaying thy blazing glory, O king!'
"Yudhishthira said, 'Let arrangements be made by thee, O regenerate one, about loosening this horse for enabling it to wander over the Earth at its will. It behoveth thee, O ascetic, to say who will protect this steed while roaming over the Earth freely according to its will.'
"Vaisampayana continued, 'Thus addressed (by king Yudhishthira), O monarch, the Island-born Krishna said, 'He who is born after Bhimasena, who is the foremost of all bowmen, who is called Jishnu, who is endued with great patience and capable of overcoming all resistance,he will protect the horse. That destroyer of the Nivatakavachas is competent to conquer the whole Earth. In him are all celestial weapons. His body is like that of a celestial in its powers of endurance. His bow and quivers are celestial. Even he will follow this horse. He is well versed in both Religion and wealth. He is a master of all the sciences. O foremost of kings, he will agreeably to the scriptures, cause the steed to roam and graze at its will.
"Yudhishthira said, 'Come, O Arjuna, let the horse, O hero, be protected by thee. Thou alone art competent to protect it, and none else. Those kings, O mighty-armed hero, who will come forward to encounter thee, try, O sinless one, to avoid battles with them to the best of thy power. Thou shouldst also invite them all to this sacrifice of mine. Indeed, O mighty-armed one go forth but try to establish friendly relations with them.'
"Vaisampayana continued, 'The righteous-souled king Yudhishthira, having said so unto his brother Savyasachin, commanded Bhima and Nakula to protect the city. With the permission of king Dhritarashtra, Yudhishthira then set Sahadeva, that foremost of warriors, to wait upon all the invited guests.'"
Hence the entire ritual revolved around the Ashwa or the horse who was to be sacrificed at the end of the ceremony. The other details of the ritual can be checked in the Ashwamedha Prakarana of the Shukla Yajurveda.