One of the key cultural conversations of 2017 has come out of accusations of sexual assault and sexual harassment, made largely by women against powerful men. From sharing their stories in detail and naming accusers in the #MeToo campaign, victims are opening up about the prevalence of sexual violence, leading to more conversation and introspection about sexual violence in our culture.

As a female psychotherapist who has worked with survivors of sexual violence for over a decade, I am thrilled to see so many walls come down, so many myths shattered, and so many violent and misogynistic acts illuminated. I’m for it; let’s continue this progress.

But there is one phrase in particular that gets in the way of that conversation, often stopping it in its tracks. It goes something like this:

Person One: “Man In Power” grabbed my ass/asked me to sleep with him after I’d repeatedly said no/promised I’d get promoted if I slept with him/told me I was wearing too much clothing and should show off my body more, etc.

Person Two: Oh, no, I know him, I’ve met him, I’ve followed his career for years, I’ve seen lots of interviews with him, and he’s really a good guy.

It’s that “good guy” assertion – right after someone has stated that said guy has not actually behaved like a “good guy” – that concerns me. It’s a way to negate the complexity of dealing with the aggressive behavior of the Man in Power and ignore or throw back the burden of proof on the victim. This use of the “good guy” label can be extrapolated for the messages it sends to victims, perpetrators, and those who don’t want to look at the rampant sexual violence in our culture. Here’s what it’s telling people:

Victims: What you are saying can’t be true

I recall facilitating a therapy group for adult, female survivors of incest while I was a social work intern. In one session, the group members talked about their perpetrators, noting how many of them were trusted members of the community who would have been seen in that “good guy” role. These perpetrators were religious leaders, coaches, teachers, patriarchs of the family, mentors, and even elected officials. The women expressed that they would not be believed as adults if they made an outcry against these men, much less as children. Years or decades after their abuse, these group members reflected the added weight of power dynamics as victims thinking about speaking out about their abuse.

When one attempts to negate an uncomfortable accusation, it leaves victims holding the burden of proof and feeling disbelieved again, reinforcing messages that they had already held about their abuse or may have been told by their perpetrator.

Perpetrators: You’ll get away with it

Years later, as a sexual assault counselor at a rape crisis center, it was my job to train agency volunteers and provide outreach and educational events in the community. I was part of a panel on sexual violence prevention for churches and was struck by what another trainer said. She noted that church staff and members often don’t see the use for this type of training, because they don’t think that type of thing could happen in their church. After all, they feel close to other members in the congregation and believe them to be caring and non-harmful. This trainer encouraged church members to face their own disbelief and to never say, “something like that couldn’t happen here,” because it sends a message to sexual predators that they can abuse congregants and no one will believe victims who come forward. This often well-intentioned disbelief leads to a cover for potential abuse to occur.

The “good guy” label also signals to perpetrators that they will be supported because members of their own team want to maintain the status quo. As a recent GQ article points out, Trump often calls a person accused of some type of wrongdoing a “good guy” as a signal that he is on their team and will have their back. He is often referring to men in power when he says this.

While Trump is a bit more tit-for-tat, it is tempting for many in our culture to bring up a “good  guy” image by calling on times when the perpetrator has exhibited good behavior as evidence that the sexual violence could not have occurred. For instance, Nancy Pelosi recently referred to Representative John Conyers Jr. as an “icon” and noted his work for women in a Meet the Press interview. Conyers’ image and previous good acts were not the point—his hidden acts of sexual harassment of women in his office was. Pelosi later clarified her remarks, but her first statement is still an example of the allowances many abusers are given. Being deemed a “good guy” means you get to be part of a protected club, where bad accusations could bounce off the walls of the entry because a “good guy” couldn’t possibly engage in something bad.

(By the way, where is this club for women? I have not heard the same pattern of a woman’s poor behavior being waived away with a “oh, no, she’s really a good girl.” Usually “good girl” is about someone not being sexually promiscuous, and “good woman” is about women fulfilling traditional female roles like working hard in the home and raising children.)

Bystanders/society: It’s okay, you don’t have to face the shit-show of sexual violence

I was recently with my young niece when she visited Santa at a local store. There was much talk afterward about the “naughty” and “nice” lists and how one gets on them. She is 3 years old and is trying to figure out the rules of the world, without a lot of ability to see the grays in behaviors. And we teach kids a lot of all-or-nothings, because we are trying to keep them safe. It’s clear that as we grow up, we learn that even “good people” can be unsafe at times or have bad things happen to them.

The all-or-nothing worldview implied by the “good guy” label plays a number on the rest of us in society. It sends us the message that we can overlook allegations of sexual violence if the perceived sum total of the accused is deemed “good.” This may help us avoid uncomfortable and complex conversations and feelings that come from seeing someone who was in the “safe”—or at least “neutral”—category as someone we should be wary of. But it doesn’t work toward creating the society we want to see: one where victims are believed, perpetrators are held accountable, and “good guys” don’t need to be defended.