Running cap vs. headband: Which to choose?

Anyone who's just started running or has been running for a while probably wonders whether a running cap or a headband is better. Some runners find it comfortable to run with a cap, while others find a headband works just as well. So what's the real difference?