He definitely does come out with a lot of awkward and witty remarks for his guests, I just felt like Ben came off like he was uncomfortable or something. And after he clarified that he was only interested in music Colbert had no idea where to go next.
he ALWAYS seems awkward when he talks. i saw him at the strand bookstore the other day and he did a Q&A and he was so painfully awkward. also, judging from experience as well as hearsay from a few others, he is a bit of a jerk.