We have a very old minhag which dates back at least to the days of the Geonim to begin our observance of Yom Kippur with the reciting of Kol Nidrei. The older text of Kol Nidrei was based on the assumption that the chazzan, together with the two people standing at his sides, constitutes a beis din to be matir the nedarim which we all took over the course of the past year (from last Yom Kippur to this Yom Kippur.)
What is the special connection between hatoras nedarim and Yom Kippur? Several explanations have been given, the most famous of which is as follows: when a husband or a father is maifer the nedarim of his wife or daughter, the neder was in effect up until the time that he declares the ha'forah, and only from that moment and on is the neder is no longer in force (Nazir 21b.) Beis din can be matir a neder, and the Gemorah (Kesubos 74b) asks how are they the baalei batim over my neder to declare it as null and void? The answer is that the beis din paskens that the neder was taken in error (b'ta'us) and therefore was never binding in the first place. The reason for this retroactive uprooting is that the individual never would have made the neder to begin with had he realized how difficult life would be or how uncomfortable things would be because of it, and therefore the beis din has the right to declare the neder to be a neder b'tous - a neder made in error.
The definition of ta'us with respect to nedarim is not the same as the definition of ta'us with respect to a purchase or a marriage. If a couple gets married and after many years realize that they are not for each other, they cannot declare their marriage to have been a kiddushei ta'us. Similarly, if one buys shares in a corporation and the value of the shares goes down, he cannot declare that as a mekach ta'us (see Making a Farce of the Halacha.) Everyone knows that marriages and businesses have their ups and downs. Only with respect to nedarim does the Chumash tell us that we have a different definition of ta'us. The Torah uses the expression "l'chol asher yi'vatei ha'adom b'shvua" which the Gemorah (Shavuos 26a) understands to imply, "h'adom b'shvuah, prat l'anus - to the exclusion of a shevuah or a neder made in error. Regarding neder we work with a different definition of ta'us.
This retroactive uprooting is the connection to Yom Kippur. The Gemorah (Yoma 86b) tells us that a person who does teshuva mai'yirah can accomplish that the aveiros that he violated b'meizid should be considered as if they were only violated b'shogeig, while one who does teshuva mai'ahava will accomplish that the aveiros that he violated b'meizid will be considered as if they were mitzvos. Just like regarding heter nedarim the beis din has the ability to undo the neder such that it is considered as if it was never binding in the first place, teshuva also has the ability to undo aveiros even though the aveiros were done many years before.
How is this possible? Rav Yosef Engel (Otzros Yosef, drush #3) suggests the following explanation: The navi, both in the words of this week's haftorah as well as in other pesukim, tells us, "shuva Yisroel ad Hashem Elokecha", i.e. that one who does teshuva has the ability to come closer to Hakodosh Boruch Hu. Time itself is part of creation so by definition Hashem is above time since he is not part of creation but rather He is The Creator. Once an individual achieves closeness to Hakodosh Boruch Hu, he too, in a certain sense, is above time and therefore can he undo the aveirah today that he violated years ago; he is no longer limited by time! His teshuva makes it is as if that at the time that he did the aveira b'meizid it was really b'shogeig or a mitzva.
The Gemorah (Shabbos 118b) tells us also that one who will be careful to observe Shabbos properly stands a better chance of having his aveiros forgiven if he does teshuva. This idea is conveyed in the Shabbos zemiros when we say "kol shomer Shabbos kados mei'chalilu", where the word mei'chalilo has the connotation of mochul lo, that his sins will be forgiven. What is special about Shabbos that it has the ability to bring about mechilas avonos? Rav Yosef Engel quotes kaballah sources to explain that from the very outset of sheishes yemei bereishis Hashem created the concept of time, but at the conclusion of the six days of creation Hashem instituted his Shabbos which gives us the ability to come closer to Hashem and return to the pre-creation status of lema'alah min ha'zeman. It is that ability to be lemaalah min hazman that enables B'nei Yisroel to undo the aveiros ex post facto. The umos ha'olam also have the ability to do teshuva, as is evidenced from sefer Yonah, but this concept of being okeir aveiros l'mafreiah, which is similar to the beis din's ability to be matir nedarim l'mafreiah, is reserved only for Benei Yisroel.