Mazal tov on the birth of your baby!
A light brown, or yellow brown (the color of coffee with milk or lighter) with no hint of a reddish tint, like what you describe, is not a niddah color. But postpartum bleeding is unpredictable, and can come and go in waves. So yours might be over, but it also might return (though generally lighter than previously).
Even if relations are off the table, immersion is permissible for the purpose of other contact and to ease caring for the baby. There is no pressure or reason to rush it, but you are free to attempt a hefsek taharah as soon as you feel ready (keeping in mind that bleeding may return).
Start by taking a look in the mirror, to get a sense of where your tears are and how they are healing. Be very gentle around the tears, perhaps lying down to make the bedikah easier, and stop if you feel discomfort. Go at least up to the first knuckle, but don’t force anything deeper than is comfortable. Omit the moch dachuk this first cycle.
Please don’t hesitate to reach out to us with any further questions. You might also find it helpful to read our site’s article on the Postpartum period. Refuah shleimah!