Arjuna raised this question to Krishna (English translation is of Swami Gambhirananda quoted from Gitasupersite):
अर्जुन उवाच
अयतिः श्रद्धयोपेतो योगाच्चलितमानसः।
अप्राप्य योगसंसिद्धिं कां गतिं कृष्ण गच्छति।।6.37।।
6.37 Arjuna said O krsna, failing to achieve perfection in Yoga, what goal does one attain who, though possessed of faith, is not diligent and whose mind becomes deflected from Yoga?
Krishna answered in four verses of Bhagvad Gita:
श्री भगवानुवाच
पार्थ नैवेह नामुत्र विनाशस्तस्य विद्यते।
नहि कल्याणकृत्कश्चिद्दुर्गतिं तात गच्छति।।6.40।।
6.40 The Blessed Lord said O Partha, there is certainly no ruin for him here or hereafter. For, no one engaged in good meets with a deplorable end, My son!
He either born in family of those who are competent to practise Yoga or in a family of wise Yogins who practise Yoga and are themselves capable of teaching Yoga.
6.41 Attaining the worlds of the righteous, and residing there for eternal years, the man fallen from Yoga is born in the house of the pious and the properous.
6.42 Or he is born in the family of wise yogis [Persons possessing knowledge of Brahman. (S. concedes that some rare householders also can have this knowledge, and he cites the instances of Vasistha, Agastya, Janaka and Asvapati of olden days, and Vacaspati and the author of Khanada of recent times.)] only. Such a birth as is of this kind is surely more difficult to get in the world.
Now, important thing is What happens to spiritual journey traveled in previous birth. This is very important since it's well known that it can take many births for mind to be eligible for spiritual understanding and again it can take many births for attaining the goal in the spiritual journey. Also it is observed that many people who even having birth in the family practicing Yoga, don't follow it. So, only getting birth in the Yogi family doesn't guarantee for spiritual progress. That depends upon his/her previous Samskara.
Here Bhagavan assures that after getting birth in Yogi family, one continues his/her spiritual journey with more success.
तत्र तं बुद्धिसंयोगं लभते पौर्वदेहिकम्।
यतते च ततो भूयः संसिद्धौ कुरुनन्दन।।6.43।।
6.43 There he becomes endowed with that wisdom acquired in the previous body. and he strives more than before for perfection, O scion of the Kuru dynasty.