Are you familiar with the characters prior to watching this show? With each additional complaint I heard about any given character, I always wonder if the person who said it has the same history with them as I do, having listened to the podcast they've grown on over the past few years. Not trying to sound douche-y, it just seems like they might be a bit of an acquired taste.
I'm familiar with his Bobby B, Oh Hello, El Chupacabra, and fashionista character from his Thank You, Very Cool comedy special. Bobby B is hilarious to me (minus the Ghost Bouncers sketch) but the Oh Hello guys don't do it for me.