UPDATED 09/20/12 6:08 a.m.

CHICAGO (CBS) — The former pastor of a huge Baptist church in Hammond, Ind., has pleaded guilty to taking a 17-year-old girl across state lines to have sex.

READ MORE: Chicago Police Union President Urges Aldermen To Repeal Mayor's Vaccine Mandate For City Workers; 'This Tyrant At The Top Needs To Stop What She’s Doing'

Dr. Jack Schaap, who was fired as pastor of First Baptist Church of Hammond this summer, has been charged with transporting a minor across state lines for sexual activity.

He made his initial appearance in federal court in Hammond on Wednesday and was taken into custody by federal marshals. He has agreed to plead guilty, and federal prosecutors will recommend he face the mandatory minimum of 10 years in prison.

Schaap waived his right to have an indictment by a grand jury after having signed a plea agreement in federal court on Tuesday.

LISTEN: WBBM Newsradio’s Mike Krauser reports

Soon after hearing the charges against him Wednesday afternoon, he was taken away by U.S. Marshals, pending his change of plea hearing and sentencing.

Schaap walked into federal court with his supporters and his wife by his side. He held hands with his wife as he entered court.

READ MORE: Chicago Weather: Rain Totals From Sunday And Monday Storms

The judge presiding over the case asked Schaap if he understood the possible sentence. As his wife watched, he said, “Unfortunately, I do, sir.”

He is accused of taking a 17-year-old girl across state lines, with the intent of having sex with her. He had been counseling the girl, who was 17 when the allegations came to light.

Schaap is fired as pastor of First Baptist this summer after admitting to a sexual relationship with the girl.

Prosecutors said Schaap used an iPad, iPhone, digital camera, voice recorder and two computer flash drives in the commission of his crime with the young girl.

In addition to at least 10 years in prison, Schaap faces a $50,000 fine, and at least 5 years of court supervision after his release.

In a statement read by his attorney, Paul Stracci, Schaap admitted he caused harm to the young lady, her family, and his own.

“Rather than cause further harm by dragging all those affected through a long protracted legal practice, I have decided to accept responsibility. I hope that my decision not to contest this matter will allow everyone to move forward with their lives,” Schaap’s statement said. “It’s always distressing when a spiritual leader acts in a way contrary to his beliefs and teachings, but spiritual leaders are men susceptible to sickness, weakness, stumbling and sinning.”

Before his court appearance, Schaap looked at his wife, said, “I love you baby,” gave her a very strong embrace and long lingering kiss.

MORE NEWS: Another Victim Of Serial Killer John Wayne Gacy Identified: Francis Wayne Alexander, Of North Carolina

In the courtroom, upon hearing he would be taken into custody, he winked at his wife, mouthed the words, “I love you,” and walked away.

Suzanne Le Mignot