Robert Downey Jr.Photo:Axelle/Bauer-Griffin/FilmMagic

Robert Downey Jr. attends MAX Original Series “Downey’s Dream Cars” Los Angeles Premiere at Petersen Automotive Museum on June 16, 2023

Axelle/Bauer-Griffin/FilmMagic

Robert Downey Jr.is getting candid about serving time in prison decades ago.

“Two weeks later, I’m in a place called Delano, which is a receiving center where they decide where you’re going to go. Arguably the most dangerous place I’ve ever been in my life because nobody is designated. If they’re a level 1, 2, 3 or 4 criminal, [everyone’s there],” he said. “You could just feel the evil in the air.”

“It was kind of like just being in a really bad neighborhood, and there was no opportunity there; there was only threats,” continued Downey Jr. “So, yes, everyoneisgoing to take your wallet, so watch it.”

Robert Downey Jr.Charley Gallay/Getty

Robert Downey Jr. attends Sr. at FYSEE | Netflix at Red Studios on May 17, 2023

Charley Gallay/Getty

Never miss a story — sign up forPEOPLE’s free daily newsletterto stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

After being transferred, he described what it was like stepping out at the facility for the first time.

“Walking onto the yard that you’re gonna be doing more than a year on for the first time, the closest thing I can associate it to is being sent to a distant planet where there is no way home until the planets align….,” he said.

In December 2015, then-governor of California Jerry Brown pardoned him of the convictions.

HOLLYWOOD, CALIFORNIA - NOVEMBER 04: Robert Downey Jr. attends the 2022 AFI Fest - “Sr.” Special Screening at TCL Chinese Theatre on November 04, 2022 in Hollywood, California. (Photo by Axelle/Bauer-Griffin/FilmMagic)

Though he acknowledged being incarcerated was one of the “worst” things that happened to him, Downey Jr. explained that after two weeks in prison, he found a pattern and somewhat adjusted to his situation.

“We are programmed to, within a short amount of time, be able to adjust to things that are seemingly impossible,” he said. “… Day 15 was a ball. By day 15 I’m playing, literally, I’m dialed in.”

He said of prison culture, “As long as you have a willingness to do harm, it is unlikely that you will be targeted. It really is that thing of what is the difference between acting like you’re willing to do harm and being willing to do harm.”

source: people.com