Lionel Richie Revisits ‘Disastrous’ Brawl Between His First and Second Wives

A master of writing hit love songs, Lionel Richie is revisiting a time when his real life love story was anything but harmonious.

The Grammy winner and American Idol judge, 76, opens up about his journey like never before in new memoir Truly, out now.

“There are certain things you realize in your life that you got through them because you tried to forget,” Richie told PEOPLE of his memoir. “That’s where the book took many a turn because it started getting deeper into the emotions of my life. There were some moments when I definitely wanted to remember but there were also others that I had tried to forget and had to dig up.”

One such moment was undoubtedly a shocking brawl that took place between his then-wife Brenda Harvey and Diane Alexander, a woman he was seeing at the time.

Richie and Harvey were college sweethearts at Tuskegee University, the historically Black school where he became a member of Alpha Phi Alpha Fraternity, Inc. and first joined singing group The Commodores. The band went on to become one of the preeminent R&B soul acts of the late ’70s. Meanwhile, Richie and Harvey wed in 1975 and later adopted daughter Nicole Richie, 44.

But amid the height of his fame, their marital bliss wouldn’t last.

“In June of 1988 came the point of no return,” writes Richie of the domestic incident he calls “the scandal of my century.” He shares, “What very few people knew was that Brenda and I were separated. I got a place at the beach, and I was hiding out there, trying to be alone, trying to decompress somehow, because it was killing everyone.”

Richie describes that he’d headed back in to L.A. that morning and stopped by to see Alexander. Then they heard a knock at the door.

“When the door opens, Brenda is standing there,” he writes. “There is a massive confrontation. Diane is in total shock and I’m trying to leave, to draw Brenda away. It’s the most awful screaming match of life, and it’s all in this small entryway.”

Richie reveals that he left in the midst of the chaos, hoping his wife would follow. “But once I was gone, she went back and resumed the argument. The neighbors called the police. Charges were brought and then dropped.”

The scandal played out all over the press, but “the most painful part, the tragedy of our lives,” writes Richie, “was that Brenda was heartbroken. And I get it. Ladies and gentlemen, I get it.”

The entire ordeal was “just raw and ugly and disastrous,” Richie reflects. It was also the final straw in his embattled relationship with Brenda.

“That’s the saddest day in the life of a marriage,” he laments. “When you know you may love each other, still, and you have a history that you will always have, but it’s over. And you know it.”

After a protracted divorce from Brenda, Richie married Diane in 1995. They later welcomed son Miles, 31, and daughter Sofia, 27, but would eventually split in 2004. The star is currently dating Lisa Parigi.

Richie tells PEOPLE that by sharing his life’s ups and downs in Truly, he’s hoping to help others on their journey.

“My old expression is ‘If I can tell you how I got through it, it might remind you of some of the things you’re going through and might help you and guide you through your lives.'”

Truly by Lionel Richie is available now, wherever books are sold.