By Terry Boers-

(CBS) For the last 20 years or so, the vast majority of my weeknights have ended exactly the same way — watching the Late Show with David Letterman.

Here’s how it worked. Give or a take a minute or two either way, I’d stop watching baseball, basketball or whatever right around 11:15 p.m., start the tape of Letterman and make it through the show in 45 minutes, happily zipping through the commercials and getting to the finish line by midnight.

I’m most assuredly one of Letterman’s old guard of fans, a group that seems to be disappearing faster than any new line of Jordan’s.

When he announced his retirement on April 3, Letterman’s late-night audience had shrunk to a puny 2.7 million, placing him right alongside Jimmy Kimmel and well behind the five million-plus being generated by Jimmy Fallon, the newest kid on the late-night block. FYI, you can keep both of the Jimmys.

But it doesn’t appear that the decision had anything to do with numbers. It had more to do with the fact that Letterman turned 67 on April 12, and there has to be an expiration date on even the greatest of jobs unless your name is Vin Scully, who’s still great at 86.

And while Letterman’s official date of departure isn’t going to be announced until sometime next June, he can walk right off the Ed Sullivan Theater stage for the last time and claim his place in the pantheon of the greatest late-night talk show hosts right alongside Johnny Carson, the man he should have replaced long ago instead of the milquetoast Jay Leno, who was finally bus-tossed by NBC earlier this year. Reports are he’s somehow managed to keep his chin up.

There are many reasons Letterman has withstood the test of time, including his keen ability to get out in front of the worst situation of his career, which happened in the fall of 2009.

That’s when it came to light that Letterman had been having sex with more than a few of his employees over a number of years.

He’d been caught kissing his girlfriend, Stephanie Birkitt, who was a Late Show staffer and, by the way, a completely talent-less fool who’d been given an array of appearances on the show. I kind of thought something had to be up. Birkitt had even been dispatched to the Winter Olympics in Utah, where she provided more brain-dead babble.

Keep in mind here that Letterman had married his longtime girlfriend Regina Lasko in March of 2009, well after she’d given birth to his son, Harry.

When a pattern of Letterman’s bad sexual behavior was about to be reported, he offered up a long apology to his wife on a show, vowing that he would do his best to repair their fractured relationship, noting, “I’ve got my work cut out for me.” He also apologized to members of his staff and then revealed he’d been the victim of a $2 million blackmail threat, a case that would later be settled in criminal court.

In the end he somehow managed to crawl out from under the rubble of the horrible mess he’d made. It could have been a career-ending scandal that would have left Letterman and his amazing legacy in ruins. But his own sincerity and the willingness of CBS and his fans to forgive saved him.

It should be noted here that Letterman always seemed to be at his best in the worst of times, especially after the dark days that followed the 9/11 attack on the World Trade Center.

He had no doubt gone through the horrible depression that we all felt when he returned to the air six days after the horrendous events, saying later in an interview with his pal Regis Philbin that he didn’t feel as if he were ready to work again.

In reality, none of us did.

His monologue that night was delivered from behind his desk and there were no jokes, only lavish praise for New York firefighters, police and mayor Rudy Giuliani. He also noted that if he lived a “thousand years,” he would never understand the motivation behind the attack.

It was pitch perfect. So was guest Dan Rather, who fought back tears through most of the interview as he delivered his heartfelt thoughts.

And soon after that night Lettermen went back to being Letterman, and we readily took the cue here at The Score, going back to being Boers and Bernstein, discovering that it was OK to laugh again, to have a good time.

Letterman certainly doesn’t have that edginess that he once did, and his willingness to make nice with Oprah Winfrey a number of years ago was absolutely barf-worthy. All the snide things he’d said about Queen Oprah over the years were spot on. Never has TV seen a more self-indulgent wind machine than Winfrey, who always believed she was saving the world.

But we’ve already noted that Letterman was far from perfect.

This much I can guarantee: When Dave goes, I’m done with late-night talk.

Maybe his replacement, Stephen Colbert, will be great at the job. I know he’s exceedingly bright and funny and has built a loyal following during his years on The Colbert Report.

I’m just not interested.

Letterman has done one other thing for me, setting a sensible path for retirement, something I’ve been mulling over for a while. For some reason I just couldn’t figure out when it was going to be or how I was going to do it.

I’ve got it now, a perfectly clear path that finally makes sense, even though there are several ifs involved.

Funny how things happen that way.

So one more time, thanks Dave. It’s been great.

A longtime sportswriter for the Chicago Sun-Times, Terry Boers now co-hosts The Boers and Bernstein Show, which can be heard Monday-Friday from 1p.m.-6p.m. on 670 The Score.

Watch & Listen LIVE