Does sleeping around matter?

How much does improper conduct in a politician’s personal life (e.g. extramarital affairs) affect their suitability for office?