throw out the baby with the bathwater

Throw out the baby with the bathwater is a phrase which is used to suggest that an error could have been avoided if someone good was eliminated while trying to get rid of something bad. In other words, rejecting the essential along with the inessential. This can be used in a variety of circumstances where in order to get through an issue or problem, you must eliminate both the good and the bad.