My Grandpa walked 18 holes 2-3 times a week into his 70s. He continued to golf, with a cart, into his 80s. My Grandma declined faster, but they swam into their 80s. I saw how their peers aged who didn't exercise. Simply raising/sustaining the heart rate does wonders for maintaining our bodies as we age.

That said, I'm a huge proponent of Yoga. While I've done the athletic/work out styles, I teach a senior men's chair yoga class. It's now my favorite class. Why? I don't hurt myself. Men often don't consider Yoga, it's so female-centric in the US, that's why I teach men's yoga. If interested and if you can't find a men's only class or a male instructor, look for a gentle yoga class or a chair yoga class to get started. I've got a series on you tube if you wanted to try it out. You seem like someone who would try something new a try, even if you were the only guy in the class.

Here's another link on why it's good for men to find a male instructor for yoga.