Neural Switch Flips on Aggression in Male Mice

When a male mouse senses a territorial threat from another male, an aggressive reaction toward the invader instinctively kicks in. When the same mouse detects a sexually receptive female, however, his reaction tends to be a mating attempt, not an attack. The messages may both begin with the smell and sight of another mouse. But where does the mouse brain sort out the options for the appropriate social reaction?

Research has already identified two areas in a region called the hypothalamus that are involved in mice’s aggression and mating responses. The hypothalamus, though, is just one stop in a circuit that channels the internal signals that switch on these social behaviors. In results published on July 27 in Nature Neuroscience, investigators show that two other populations of cells in a brain region called the amygdala send out projections, or axons, to two target hypothalamic areas and can switch them on or off, depending on the circumstance.

The study reveals that the amygdala is important in the “social circuit” that regulates aggressive and mating behaviors in the mouse brain. “It’s interesting to see these two populations in the posterior amygdala that seem to play largely nonoverlapping roles in the control of aggression and mating behavior,” says neuroscientist David Anderson, director of the Tianqiao and Chrissy Chen Institute for Neuroscience at the California Institute of Technology, who was not involved in this work. For some of the experiments, however, “the data are much stronger for the projection for controlling aggression than for the projection that’s supposed to control sexual behavior,” he says.

The two hypothalamic regions already had been shown to be involved with aggression and sexual behaviors in both male and female mice. One area, the medial preoptic nucleus (MPN), is linked to sexual behaviors—usually mounting for males and lifting the hindquarters for females. The second area—cumbersomely dubbed the ventrolateral part of the ventromedial hypothalamus, or VMHvl—is associated with aggression.

Inhibiting or fully disabling these hypothalamic areas diminished mice’s sexual behaviors and aggression in the earlier studies. But an open question was the location of the on switch for exciting, rather than inhibiting, these hypothalamic cells.

In the new study, Takashi Yamaguchi, a postdoctoral fellow at NYU Langone Health’s Neuroscience Institute, and his colleagues showed that in the posterior amygdala, two distinct groups of cells pick up incoming sensory signals and excite the appropriate hypothalamic target.

To identify these connections, the researchers first used tracers to backtrack along the neural links between the hypothalamus and the posterior amygdala. They detected VMHvl connections going back to one cell population and MPN connections to a different group of cells. Working in the other direction, the scientists showed that stimulating these separate posterior amygdala cell populations triggered excitatory signals that were sent straight to their separate VMHvl and MPN targets.

In behavioral studies, the team found that this excitation was in full swing in the posterior amygdala neurons projecting into the MPN during mating. And it increased in the neurons leading to the VMHvl during male-male aggression. In addition, stimulating these distinct posterior amygdala populations enhanced the related behaviors: male mice would even attempt sex with nonreceptive females and show aggression to both males and females.

To see how the mice behaved when these cell populations were not functional, the investigators used different techniques to eliminate their activity. Removing posterior amygdala cells’ projections to the MPN meant an almost total loss of sexual activity in the male mice. But taking out the projections to the VMHvl yielded a more complicated result: the mice’s aggression declined, but they also showed a somewhat dampened sexual enthusiasm.

Anderson says one reason for the spillover effect on sexual behavior could be that the posterior amygdala projects to a lot of other structures that, in turn, also connect to the MPN. Whether these effects on mating result from direct or indirect projections to other regional targets is something to tease out, he says.

In addition to this hint of overlapping effects, another feature unites both regions of the posterior amygdala: their high level of a protein that interacts with estrogen. The presence of this estrogen receptor suggests a role for steroid hormones in these communications, although Yamaguchi and his colleagues did not address the hormonal factor.

“It’s a crucial question,” says Yamaguchi, who is co-first author of the new study. These cells also make an enzyme that converts testosterone to estrogen, and he speculates that this conversion could be important. Anderson says it is already linked to some testosterone-related effects on mouse aggression.

An unrelated study by Anderson’s group, released as a preprint and not yet peer-reviewed, also homes in on the VMHvl. The team found that repeatedly switching on these cells in genetically identical male mice could prime most of the animals to show increased aggression. But about a quarter of the mice did not respond to this aggression training. With testosterone treatment, though, these nonaggressive animals became aggressive. This malleability in a trait linked to a brain region thought to be under strong genetic control “gets into the whole issue of individual differences in aggression and nature versus nurture,” Anderson says.

These studies focused on males, but that fact does not necessarily mean that these cell populations will have a different effect in females, says Daphna Joel, a professor in the School of Psychological Sciences at Tel Aviv University and co-author of Gender Mosaic: Beyond the Myth of the Male and Female Brain Casting sexual and aggressive behaviors as “male” or “female” can be problematic because they do not fall along clean lines between the two, she says. Male and female mice can show the same behaviors, “even if they greatly differ in frequency.” Females can also attack other mice or try to mount them, Joel points out. “Both males and females have the ‘neural machinery’ for all behaviors,” she says.

Yamaguchi says that earlier work with female mice showed that in these brain regions, “the cell number and structure are almost the same” as in males. In the females, two areas of the posterior amygdala that project to two areas of the VMHvl influenced aggression and sexual behavior.

These findings also prompt the question of whether any of them apply to humans. One limitation of such studies is that they usually rely on a single mouse strain under controlled laboratory conditions, Joel says. These controls are important for studies of this size and complexity, she adds, but “people are often quick to generalize not only across additional strains and conditions but also across species,” including humans. In fact, Joel says, any sex differences among humans would be even smaller than those in laboratory animals.