Here's a few ideas:
*Point things out while you walk - dogs, flowers, birds etc
*What about giving him some books to look at or perhaps go walking when it's his nap time (if he still has one) then you won't need to keep him entertained.
*Try an exercise video - he can copy Mum.
*Chasey or playing ball - where you have to run for ball.
*piggy backing him around your yard (think you might get sick of this before he will!)
When you go swimming try holding him and running around the pool - he'll have a blast and it will be really good for you.
I know you said you don't have anyone to watch him but do you have friends that also walk - I use to do this with my friend and her 2 girls (now 3 girls) and they were busy looking for me, also we used to play 'catch up', I walk a little ahead, Mum and kids would run to catch up - lots of fun - you could swap on the way back so you're not doing all the running.
Hope these help.