I prefer running, but not when I'm naked
Yeah wen you are naked, your dick is just flopping around everywhere. Not a nice feeling.
Yeah, it's not fun
On the contrary, I've never felt uncomfortable running nude? I prefer it over clothed if possible... Could just be me though... 
If we are going to be totally honest here, when running I don't mind what my dick does. But, I hate my balls flopping.
???
Shouldn't there be a physical reaction to retract closer to the body? 
Isn't that a temperature thing?
But also happens during physical excersise.
My balls never retracted during exercise. Those things flopping around make it hard to concentrate.
I once read that there's nerves on your upper legs that, when activated in a specific way, causes your nutsack to tighten up. I find it happens to me when doing high impact exercise like running, but there's no undies in the way, so that could be why.
Also, nude biking is not that bad, I've used the bike trainer at home before (clipping my bike in) and there's no real pain for the first 30 mins, until there's lots of numbness, but it's better than biking in jeans.
Also, I've never experienced any discomfort with flopping, in fact, quite the contrary, even in extreme cases like going to the trampoline park in jogging bottoms. Then again, I've still got another 5 years of development (max.) ahead of me, so that could be why.