TL;DR They basically indicate the occupations that Brahmanas used to have back when they were officiating Vedic Yagnas rather than being temple priests and the like.
In order to understand what it means for someone to be a Rig Veda Brahmana and so on, you need to understand the nature of the Vedas. As I discuss in this answer, how the Vedas originated is that from time immemorial, sages known as Dhristas have heard sacred verses directly from the gods during a state of Tapasya (deep meditation). And then in the Dwapara Yuga, a sage named Krishna Dwaipayana Vedavyasa (or Vyasa for short) compiled these verses into a set of four books we call the Vedas. (Technically Vyasa only compiled the Rig, Yajur, and Sama Veda - the Atharvana Veda was compiled by the sage Atharva and Angiras.)
But Vyasa didn't just divide them in an arbitrary way; he organized the verses according to their function. The verses of the Vedas have a purpose, and that is to be used in the performance of Yagnas (fire rituals). A traditional Vedic Yagna would be conducted by three main priests:
The Hotar, the priest who would chanting Richas, i.e. the Vedic mantras in the original form they had when they were heard from the gods
The Adhvaryu, the priest in charge of carrying out the actual detailed rituals of the Yagna and reciting the appropriate Yajush, a sacred utterance which is a modification of a Richa and which is designed to accompany each ritual action they're performing.
The Udgatri, the priest in charge of singing Samans, or songs which are adaptations of one or more Richas set to music
In earlier Yugas, families just had to struggle to maintain their own traditions to know which verses to say and under what circumstances. So what Vyasa did is he compiled the Vedic verses into books that each kind of priest would have to learn: he compiled all the Richas that a Hotar would use into a book called the Rig Veda Samhita, he compiled the Yajushes for the Adhvaryu into the Yajur Veda Samhita, and he compiled the Samans for the Udgatri into the Sama Veda Samhita. And then he taught these Samhitas to a bunch of different disciples; here is how the Vishnu Purana describes how the Rig Veda was passed down:
In the first place, Paila divided the Rig-veda, and gave the two Sanhitás (or collections of hymns) to Indrapramati and to Báshkali. Báshkali 6 subdivided his Sanhitá into four, which he gave to his disciples Baudhya, Agnimát́hara, Yajnawalka, and Paráśara; and they taught these secondary shoots from the primitive branch. Indrapramati imparted his Sanhitá to his son Mańd́ukeya, and it thence descended through successive generations, as well as disciples 7. Vedamitra, called also Śákalya, studied the same Sanhitá, but he divided it into five Sanhitás, which he distributed amongst as many disciples, named severally Mudgala, Goswalu, Vátsya, Śálíya, and Śiśira 8. Sákapúrńi made a different division of the original Sanhitá into three portions, and added a glossary (Nirukta), constituting a fourth 9. The three Sanhitás were given to his three pupils, Krauncha, Vaitálaki, and Valáka; and a fourth, (thence named) Niruktakrit, had the glossary. In this way branch sprang from branch. Another Báshkali composed three other Sanhitás, which he taught to his disciples Káláyani, Gárgya, and Kathájava. These are they by whom the principal divisions of the Richa’s have been promulgated.
Thus a bunch of parallel schools or Shakhas emerged, each with their own recension of the Rig Veda Samhita, and similarly for the other Samhitas. And then different Brahmana families started affiliating themselves with different Shakhas, sending each generation of children to Gurus following that Shakha. So in a nutshell, the term "Rig Veda Brahmana family" really means "family of Hotar priests", but because (after the time of Vyasa) families of Hotar priests would make their children learn the Rig Veda, they came to be known as Rig Veda Brahmanas, and similarly for the other Vedas.
Now you also asked about the Atharvana Veda. As I mentioned above, the Atharvana Veda Samhita was compiled by the sages Atharva and Angiras, not Vyasa. But Vyasa incorporated the Atharvana Veda into his tradition, by creating a role for the Atharvana Veda in Yagnas: in addition to the Hotar, Adhvaryu, and Udgatri, Vyasa created a new position called by the generic term "Brahmana." Vyasa made the Brahmana responsible for making sure the Yagna is going properly, and for uttering mantras from the Atharvana Veda to correct for mistakes done by the other priests. Here is how the Vishnu Purana describes Vyasa's new four-priest Yagna:
Vyása instituted the sacrificial rite that is administered by four kinds of priests: in which it was the duty of the Adhwaryu to recite the prayers (Yajush) (or direct the ceremony); of the Hotri, to repeat the hymns (Richas); of the Udgátri, to chant other hymns (Sáma); and of the Brahman, to pronounce the formulæ called Atharva. Then the Muni, having collected together the hymns called Richas, compiled the Rigveda; with the prayers and directions termed Yajushas he formed the Yajur-veda; with those called Sáma, Sáma-veda; and with the Atharvas he composed the rules of all the ceremonies suited to kings, and the function of the Brahman agreeably to practice .
Vyasa similarly instituted Shakhas for the Atharvana Veda, and families that used to be Hotars, Adhvaryus, and Udgatris started switching to becoming Atharvana Veda Brahmanas. Unfortunately, most of the Shakhas of the Atharvana Veda have died out, for multiple reasons. First of all, the Brahmana position that Vyasa created was so limited that it was not perceived to be useful. And second of all, the Atharvana Veda acquired a negative stigma, because it contains things like magical spells and the like. As a result, only two Shakhas remain, and even they are dying. So there are few Atharvana Veda Brahmanas left, outside of Odisha, Benares, and Gokarna. But not all hope is lost! In South India the problem of Atharvana Veda extinction has been recognized, and there are efforts to preserve the traditions.
By the way, you may be wondering why most Rig Veda Brahmanas are no longer Hotars, Yajur Veda Brahmanas are no longer Adhvaryus, etc. It's not because people are abandoning their Varna Dharmas due to the evil of the Kali Yuga. It's because Yagnas have become a less popular form of religious activity, in favor of devotional activities like idol worship and the like. This is a scripturally sanctioned move; for instance the Vaikhanasa texts, which I discuss here, describe how worshiping a statue of Vishnu is an acceptable alternative to conducting a Yagna. The Pancharatra texts (which I discuss in my questions here), texts that are fundamental to mainstream Vaishnavism, contain similar statements.
