How Going to Jail Changed This Franchisor's Outlook -- And Helped Him Rake in $42 Million Pat Swisher was a brash young franchise titan until a stretch in federal prison changed the way he thought about business.

Pat Swisher used to carry himself with a cocky self-assurance.

He figured it was his due for being a successful, self-made man. Then he went to jail.

"I'll never forget walking into the dorm," he says. The first person he met was a huge, heavily muscled inmate with gold teeth. Swisher is a former college football player, but he's not an intimidating man -- about 5-foot-9, with a carefully combed coif of reddish-gold hair. As the inmate approached, Swisher remembers thinking, This guy's going to beat the hell out of me.

Instead, "he put his arm around me and said, "I got you, man. If anybody messes with you, tell 'em to come see me.'" Swisher chokes up at the memory. "It turned out that this was the most genuine, sweetest man I ever met. To this day, I love that man. He didn't have a dime. All he wanted to do was get out so he could see his son play basketball. Excuse me." Swisher reaches for a tissue. "So, yeah," he says, composing himself. "That's how you learn who you're dealing with."

Related: 5 Hacks to Keep Your Ego From Squashing Your Success

That moment altered the way Swisher, 62, conducts business. A lifelong entrepreneur, he started Swisher Hygiene in 1983 in Charlotte, N.C., a pioneer in franchised hygiene services for commercial restrooms, mainly in restaurants and gas stations. By the early 2000s, he had shepherded the company into operations in 23 countries, serving more than 100,000 clients and with annual revenue of $22 million from his 140 units. He had a virtual lock on the market, deep-cleaning with proprietary chemicals and treatments to protect surfaces from bacteria, viruses and odor that soap-and-water cleaning and standard disinfectants couldn't kill. He was living large. "Big ego, pursuing a fast life and my own plans -- you know what I'm talking about," he says.

Then came the fall. In the mid-1990s, Swisher Hygiene ran afoul of the Securities and Exchange Commission over alleged accounting fraud (by a firm the company was using to audit its books). Without admitting guilt, Swisher Hygiene paid nearly $400,000 in fines and penalties. Unfortunately, that investigation led indirectly to the IRS probing Swisher personally. Prosecutors accused him of failing to report nearly $2 million from his stock sales and of using an offshore trust to hide them, which Swisher denies. (The account had been established on accountants' advice to protect his children's assets if he became a lawsuit target, Swisher says.) In a bind, Swisher believed he had no choice but to plead guilty to tax evasion and lying about his assets and liabilities on a 2000 home mortgage application. In February 2002, a federal judge sentenced him to two and a half years in prison.

"It was tough. It ripped my family apart. It was the worst thing I've ever been through in my life," he says. "But here I am. I didn't die. If you can survive that, you can pretty much survive anything."

Swisher got out of jail in 2004 and went straight back to his desk. Shortly thereafter he got an offer to sell his share of Swisher Hygiene for $8.1 million to business mogul Wayne Huizenga, of Waste Management, Blockbuster and AutoNation fame. He sold. Swisher spent the next few years working as a franchise consultant, but the experience wasn't satisfying. Commercial hygiene may sound like an unlikely passion, but he was itching for another shot at his old business. In 2009, he founded Enviro-Master, yet another company that specializes in franchised hygiene services for commercial restrooms. The timing was right: His noncompete clause had expired, and the market was ready for him; Huizenga had moved away from hygiene and toward selling chemicals. In 2011, Enviro-Master began franchising.

Swisher was concerned about how his past might come across to investors and potential partners. "I hadn't sold franchises in a long time, and I was very nervous," he says. "And that's when I just decided, It is what it is. I'm going to explain what happened to the best of my ability, and then let the cards fall where they may."

Related: 18 Ways to Bounce Back from Failure

Some banks and franchisees refused to work with him, but for the most part, the cards fell favorably. "I know Pat had some difficulties, but I have found him to be a man of great integrity and honesty," says Bruce Krebs of The Entrepreneur's Source (TES), a franchisor of business ownership and franchise coaching. "He's a guy who'll do whatever it takes to help a franchisee grow his business." Andrew Martin, who co-owned a Swisher Hygiene franchise in Detroit in the 1990s and bought the Orlando-based Enviro-Master franchise for central Florida in 2011, had no reservations, either. "I trust Pat. I trust the system," he says. "He tapped into something a long time ago that still, 25 years later, virtually nobody does."

On the surface, Swisher's two businesses look similar, but at their cores, they're very different. When he was young and brash, Swisher looked only at potential franchisees' cash reserves, business acumen and other quantifiable traits. Now he tries to gauge someone's character, drive and willingness to learn. He stresses the virtues of perseverance, resiliency, transparency and humility. These days, when he's interviewing franchisees, "I'm looking right here," he says, pointing to his heart.

He has even changed his process for granting franchises, with "Discovery Day," the formal visit to headquarters by a franchisee candidate for introductions and interviews. Back at Swisher Hygiene, it used to be a purely transactional enterprise. He would even sell franchises without really knowing the buyers. Now, with Enviro-Master, the night before the official visit, Swisher and often his wife join the candidate -- and even their family -- for dinner. They don't talk business. "We'll get to the business tomorrow. We want to get to know you tonight," he'll say. "I wish I had done that before." He no longer has any interest in someone who wants an Enviro-Master franchise as a side business. Breaking bread helps him weed out people. Recently he and his team met with a prospect in the Northeast who wanted to buy the entire system. He had the money. But over their meal, while Swisher spoke openly about his prison experience and what it taught him, the man was sending text messages. Swisher cut short the visit.

Swisher is also relishing his role as a mentor to his franchisees, sharing the hard lessons he has learned about resiliency and perseverance. Take Clay Whitehurst. He bought his Enviro-Master franchise in early 2012 after spending the previous two decades as a banker. Within a year, he was in deep trouble. Sales were far harder than he'd thought, he had no experience and his hires weren't getting the job done. By summer, he couldn't make payroll. Whitehurst drove to Charlotte for a hard talk with Swisher. "I was in despair," Whitehurst recalls. "I felt like I was trying all the things, but it didn't seem to be happening."

Old Swisher might have told his franchisee to snap out of it. New Swisher decided to dig deeper. "I looked right at Clay," Swisher says, "and at this point, he's crying -- and I said, "Clay, are you going to quit?' And he looked right at me and said, "Never.' I'll never forget that. And I said, "I won't, either. So now we're going to figure it out. Let's go.'" Swisher extended Whitehurst a $35,000 line of credit until he could get on his feet. Whitehurst went through training and began making sales calls himself. By the next year, he was earning more than $12,500 per week in revenue and won the system's Most Improved Franchise award. In 2015, he won Franchisee of the Year. "I had to learn a totally different skill set for a new industry and market. I learned a lot," Whitehurst says. "The fact that he believed I could do it, that he saw something in me, meant that I could and that I just needed to do it."

This new approach appears to be working. In 2016, Swisher says, he pulled in $42 million in system-wide sales across all franchisees and $10.2 million in revenue for the parent company. (Enviro-Master has made Entrepreneur's Franchise 500 list in three of the past four years.) Since the beginning of last year, the company has added 22 franchises in the United States and Canada, pushing its total to 68 in 31 states and two provinces.

Related: 8 Mindsets That Will Set You on the Path to Success

Best of all, last year, Swisher won something he thought he'd lost forever: Swisher Hygiene's old book of business. The brand had foundered under Wayne Huizenga for 11 years before he sold it to Ecolab, a Minnesota-­based water, hygiene and energy technology company. Ecolab, in turn, agreed to sell it back to Swisher in 2016 for an undisclosed amount. This was a circle Swisher had spent more than a decade trying to close. Now the client list was back in his hands, which was all the sweeter because of everything he had been through to get it there again.

"Have you ever put your heart and soul into anything and built something, like a house? It was like I physically built a house, and I sold it, and I lived across the street, and I watched this house slowly degrade -- the roof fall in, the shingles fall off, the grass grow up in the yard," he says, reunited at last. "It's your baby, and you're watching this thing implode. For 20 years, I built that company, nurtured it, babied it, did whatever I had to do to get to that point. So when I got the opportunity to get it back? It was like, "Thank God. Come home to Daddy.'"