Breastmilk is kosher and parve. It is not in any way chametz. Therefore, there is no reason you cannot give expressed milk over Pesach.

Regarding the pump, as long as the parts are not washed together with your regular dishes, they can be used as is. If you wash them together with your other dishes, then it would be best not to use them for 24 hours and then to boil them. This will both sterilize and kasher them.

Note that boiling the pump parts is a stringency, so you can use whichever pot is easiest (Pesach or regular). If boiling the parts is difficult, you can just clean them well with hot water and soap.

Chag kasher v'sameach!