There is halachic debate about exactly how deep a bedikah must go, and some authorities do maintain that the entire finger should enter. But the consensus is that the woman's finger should enter as deep as possible for her (including cracks and crevices), as you write. Unfortunately, some women hurt themselves trying to go deeper than they should; halacha clearly does not require a woman to injure herself. (Besides, such injuries can lead to bleeding, further complicating the situation.)

Among the bedikot, the hefsek taharah and one other bedikah of the clean days (usually the first of the first day) should be as thorough as possible. There is a strong halachic basis for the other bedikot to be more shallow.

In cases of dryness, it is a good idea to use a vaginal moisturizer of some sort to facilitate performing a deep bedikah without discomfort. Slightly wetting the bedikah cloth is another, somewhat less effective, option.

Even if a bedikah was shallower than it might have been for a given woman, it is generally accepted after the fact, so you need not be nervous.