When you are talking about problems with male sexual health, you usually look at testosterone levels. Estrogen is a female hormone that’s associated with the growth of breasts and secondary sexual characteristics of women. But, estrogen can also play a huge role in maintaining a healthy level of sexual drive in men.