You do not have to be concerned about the hair you found. A hair on your torso is not considered a
chatzitzah because, when you enter the water, it usually will float away from your body. Further, as your intuition tells you, it is possible that the hair was not even on your body at the time of immersion, and only got there when you dried yourself off afterwards with the towel.
You may want to try using a checklist when doing your
chafifah and iyun. As you go through your preparations, run through the list to make sure you cover everything. Once you complete the list, you can be sure that you prepared thoroughly and properly.
B'Hatzlacha!