It does not happen to you, you beget it through your own actions or lack thereof. Such beliefs and traditions serve a more functional purpose, in addition to the divine context they are wrapped in. So, even if one is unable to understand the action of God, one can always understand it in the context of one's own actions or lack thereof.
Such vows, and beliefs and traditions about them, are meant to inculcate two things.
(a) You should understand and hence avoid frivolity of promise .
(b) It helps you inculcate discipline, in some cases, sense-control, dispassion and self-sacrifice.
Frivolity of Promise :
Consider the first. The scriptures abound with instances where X promised something to a Rishi or a God, and failed to do so and met with consequences.
The Kauravas said that the punishment for the Pandavas was staying in the forest and then staying incognito for a year, which Pandavas did, but the Kauravas didn't keep their end of the promise.
In contrast, Sri Rama, fulfilled a promise, that his father made to Kaikeyi, and Dasaratha himself fulfilling a promise that he made to Kaikeyi years back during a war. Such strong adherence to Truth, may be unheard of and laughable in the current times, the sons may not even pay up the credit card debt, but such is the high standard set by the ideals of the seers.
You can see this in everyday life too. Assume this situation. You assure your boss, that : " I shall work 8 hours every day for the next three months and complete this project by the end of February. " . But in the upcoming weeks, say, you spend your time browsing the internet, reading and writing answers on hinduism.stackexchange and a million other websites, spending away your time. Everyday the clock is ticking, you are remembering that, but ignoring it. On the appointed day, the boss calls and fires you. Now, who is responsible for this plight ? You! Boss didn't make special bad things "happen" to you, he is couriering you the result of your own inertia. You dug your own pit, a bit everyday. If anything, bosses have to do that, so as to set a benchmark standard for moral order, so that others know what it means to promise and what it means to breach it knowingly.
God didn't request you to make a deal with Her. You had the need, so you went to God, had faith in Her. She did it for you. That's your belief too. If you had a belief that a certain event will occur because of Her, after the good event occurs, you can't conveniently say, it occurred because of your own effort, right ?
Remember, God is not worried about whether you visit her temple, but you being her son, She may be more worried about the indiscipline, breach and betrayal that you have allowed yourself and those may not work well for you in the world, in your work, in your relationships and even for yourself as a matter of guilt in your mind and conscience.
The mother has to say, 'Write the homework or no ice-cream'. She may occasionally give the ice-cream to you without the homework, but if it's recurring, she will be worried about homework pattern than about icecream liking.
If you make a habit keeping your promise to God, because you know She is all-powerful, one day, we may grow up keeping our promises to men, because God is watching us through them. Swami Sivananda used to say that the karmic effect of breaking one's promise is mental agony and pain, which is true, both for divine promises and those in the material world.
http://www.dlshq.org/download/karmadisease.htm
Inculcation of Discipline :
Setting aside the vow of just visiting a temple, in many cases, the vows are also accompanied by a certain Vratham, a daily code of conduct, abstaining from something impulsive or commonplace. A pilgrimage to Sabarimala, for example, presupposes restrictions on one's behaviour. Sometimes, it even invokes a certain behaviour towards us, from others.
http://www.ayyappaseva.org/Sabarimala/AyyappaVratham
Simple people, avoid smoking, are conscious about hurting others, avoid certain kinds of foods etc.
http://www.dlshq.org/download/20instrch.htm#_VPID_28
Some of them require giving up comforts, it teaches you dispassion about worldly things, for that period. Some people give up things that are dear to them otherwise in a material sense, (such as tonsuring their head), because it helps to know the transience of external appearances and who would do it, if not out of devotion to a higher being ?
It's like jogging in army training, you don't jog all the time after posting, but it keeps you fit and trim for a moment of readiness in uncertain situations. Such periods give you a small window to show you that you can be a master of your senses with conscious application of will power, if you are tuned to a higher ideal.
