But there have been times when he's mulled coming out of retirement.
"I talked to my track coach," Bolt told CNN Sport's Coy Wire. "And he was like, 'No, you're not doing it. People that retire and come back — it doesn't always work out.'
"For me, at the end I knew it was time because the drive wasn't there. But every time I watch track and field I miss it. And every time I go to the track to see my coach and I watch him training I go, 'Did I make the right decision?' ... But every time I train with them I think, 'Ah yeah I made the right decision. I don't miss this.'"
'Sit and watch'
Instead, Bolt will settle for a spot on the sidelines at the upcoming Games, the first for 20 years where he won't be competing.
In particular, he's looking forward to watching the action in the pool.
"I'm going to go and watch, I'm excited for the first time to sit and watch it and see the energy of people," Bolt said.
"I've seen track and field throughout my life, I've always watched track and field and I always know what's going on with track and field. But I didn't see Michael Phelps swim live.
"For me, it's an opportunity to go and see these other swimmers and see something else.
"The only thing I got to see once was when I went to the Commonwealth Games and I went to watch netball. But I've always wanted to go and watch swimming live ... track and field, I love it, but this is what I really want to watch."
Since retiring, Bolt has had a stint as a footballer with Australia's Central Coast Mariners, scoring for the first team but not securing a long-term contract.