Assuming you have at least one boy and at least one girl in fulfillment of pru urevu, you are permitted to place shalom bayit ahead of having more children. Shalom bayit should be both ensured before bringing a new child into a family and preserved for the sake of each child already in it.
If you and your husband disagree, your best option is to pursue a solution in a way that maintains your shalom bayit. For example, you could consult with a rav whose counsel you both respect, or you could seek guidance from a professional counselor.