It's possible that when Baudhāyana Dharma Sūtras were being written, the Kaliṅga region was not ruled or conquered by a Kṣatriya king and cāturvarṇya (system of four castes) was not established there.
Commenting on Manu 2.23 this is what Medhātithi says:
Similarly, if a certain well-behaved king of the Kṣatriya-caste should happen to defeat the mlecchas ...
The same page gives a justification from the Apastamba Dharma Sutram
Let him not visit inferior men (such as Niṣādas), nor countries which are inhabited by them.
Also, Manu Smriti parvam 4:
Let him not dwell in a village where the sacred law is not obeyed, nor (stay) long where diseases are endemic; let him not go alone on a journey, nor reside long on a ...