Sometimes, we discriminate “sex differences” from “gender differences” implying that there are two types of differences between women and men: biologically based sex differences and socially constructed gender differences. One may even imply that biologically based sex differences are more “innate” and more difficult to change. In this article, I will argue that biological sex differences are not genetically determined. Rather, biological sex differences are products of genetic, ecological, and social influences. Organisms flexibly and adaptively change their sexual behavior to meet their environmental requirements. More importantly, theoretical biology has produced many hypotheses on how individual organisms adjust their behavior according to both ecological and social environmental changes. Those theories are testable on human behavior and will bring significant insights to “gender” difference studies.