A tortoise in front of Nandilord Shiva's idol symbolically signifies two reasons :-
As we know that when you try to touch or harm a tortoise, it will take his legs and head inside his shell, hence this signifies that even humans should withdraw all his senses from our material world, which in Hindi we say Moh-Maya nahi rakhni chahiye.
Its the way that tortoise hatches its eggs, they don't sit over their eggs to hatch, instead they constantly look at them, and hence it is kept facing lord shiva which signifies that one should concentrate while praying the god.
Apart from the above twothat, there are many stories behind a turtle facingalso has a remote relation with lord Shiva ..(Not relation to turtles in front of lord Shiva idol)
As we know, lord Shiva is always portrayed in a Yoga pose, so in yoga, we do have Kurmasana (Tortoise Pose)
Image Credits : Yoga JournalTurtle is said to be as one avatar of lord Vishnu - Kurma
According to me, there are any script reference over this, its we people who believe keeping a tortoise in front of lord Shiva's idol.