Tony Danza Recalls Getting Yelled at by Frank Sinatra for ‘Singing to Him’

Tony Danza can’t forget about one fateful interaction he had with Frank Sinatra 30 years ago.

In a new interview with The New York Post, Danza, 74, recalled the moment when he was able to sing in front of the legendary “My Way” artist.

“I was singing to him,” he noted of the time he performed at Sinatra’s 1995 televised 80th birthday tribute. “Back then, I didn’t really sing — except for in the shower. But Frank liked my performance.”

However, Danza added, “Though afterwards it looked like he was losing his balance, because he was perched on these high steps. So I put my hands under him and then he threw his elbow back and almost got me in the chin.”

The motion caused the legendary singer to apparently turn and snarl at Danza, telling him, “Back off! What’s your problem?”

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories.

Though Danza was initially in shock at how Sinatra reacted, the singer’s friend and confidante, Jilly Rizzo, told Danza to not overthink it, as it was apparently a showing of endearment from Sinatra.

“He pulls me aside and said, ‘Don’t worry about it, Tony. That’s how you know Frank likes you,’ ” Danza recalled.

A lifelong Sinatra fan, Danza took a trip down memory lane with The New York Post, saying that, “My mother, when I was a young kid, turned me onto Frank and got me excited about him. And throughout high school and college, I sang along with his records like the rest of us.”

The PEOPLE Puzzler crossword is here! How quickly can you solve it? Play now!

Though a producer initially discovered him for a movie role while he was boxing at Gleason’s Gym in New York City, “By some miracle, when I got out to Hollywood, I ended up being in Frank’s circle,” Danza recalled of what ended up actually happening when he made his way to Hollywood. “It was really kind of crazy.”

Danza, whose film debut in 1983’s Cannonball Run II also just so happened to be Sinatra’s final movie credit ever, quickly became friends with the singer.

“He had an aura when he walked into the room — there was always a hush,” he told The New York Post. “Most times, I just didn’t say anything, you know?”

Nowadays, in honor of Ol’ Blue Eyes, Danza is hosting a continually sold out cabaret show in NYC titled Sinatra and Stories, where he both takes viewers on a journey through similarly interesting stories from his career and interactions with The Chairman of the Board, as well as captivates them with covers of some of Sinatra’s hits.

“The obvious answer is that he’s great,” Danza told The Two River Times about his work on the Sinatra-themed cabaret show during an earlier interview in February. “He really touched people with his singing and acting.”