An African-American teenager was called racial slurs and attacked by two white men near his home in Virginia as part of an effort to run his interracial family out of town, a federal lawsuit claims.

The lawsuit comes more than a year after the men were convicted of assault and battery of now-18-year-old Mister Frazier, the Associated Press says. But they were cleared of hate crime charges after a judge said it appeared to be just “a neighborhood brawl.”

“That’s like saying that the graffiti painting on LeBron James’ house was just a neighborly dispute,” Frazier’s attorney, Victor Glasberg, told the AP, referring to the report that an unidentified person recently spray-painted the n-word on the front gate of James’ home.

Frazier said that he was playing outside with some kids during an 8-year-old’s birthday party back in 2015, when he heard a man in a neighboring yard of a vacant home calling the children the n-word, the AP reports. Frazier went over to confront the man about his language, according to the lawsuit, which is when the man – identified as Stephen Cooke – came up in Frazier’s face and called him the n-word as well, telling the teen to jump before pushing his shoulder into the teen’s chest.

That’s when the other man, Douglas Clark, hit Frazier in the face, the AP reports, before putting the teen in a headlock and dragging him to the ground, repeatedly punching him in the head.

“The attack fractured feelings of safety and belonging that had allowed them, an interracial family, to move to Gloucester without fear of neighbor’s race-based hostility or violence,” the lawsuit says.

The lawsuit claims the assault was part of a “campaign of race-based hostility, harassment, and violence” aimed at forcing Frazier and his family from the home. There are no other African-Americans on the block, the lawsuit says.

During their trial, the men denied that the attack was racially motivated and claimed that Frazier was the one who started the whole thing. Cooke claimed he didn’t say the n-word, but instead said “jigger”. However, he did acknowledge calling the children “dirt frogs” and “monkeys,” claiming that he was shouting at them because they weren’t supposed to be playing in the yard of the vacant home.

According to the AP, the men were convicted of assault and battery and sentenced to a year in jail but were told they only had to serve a fraction of that as long as they remained on good behavior.