It is quite possible that Lord Krishna later must have gone to North and thence to Dwarka. Also his color was brown which might indicate a south Indian origin .
Yadu's descendants came to be known as the Yadavas. For more more details see What is the historical origin of Yadavas?
SB 9.19.22 says that
King Yayāti gave the south to his son Yadu.
As Yadu was appointed the king of South so it's oblivious that many Kshatriya Yadav families were/are from the South.
However it's nothing to do with Lord Krishna's birth. Lord Krishna took birth in Mathurā and was raised in Gokul, Vṛndāvana (~ SB 10). Lord Krishna's ancestors (much much earlier; not his father, grand father, ..) might have moved from South to North.
Later Lord Krishna moved to Dvārakā from Mathurā (~SB 10.50).