Chamber pop quintet Matt Pond PA was led by titular singer/songwriter Pond, who formed the group upon relocating from New England to Philadelphia; also featuring guitarist/bassist Josh Kramer, cellist Jim Hostetter, violinist Rosie McNamara-Jones and drummer Sean Byrne, they debuted in 1998 with Deer Apartments, that same year winning top honors in the Unheard? unsigned bands competition sponsored by online retailer CDnow. Matt Pond PA's sophomore effort, the lovely Measure, followed in 2000. The I Thought You Were Sleeping EP appeared in spring 2001 and dates with Very Secretary, Ted Leo, and Aspera followed that summer. Within the next year, the band shifted around its roster to included Mike Kennedy (drums), Matt Raisch (bass), Jim Kehoe (guitar) and The Rachel's Eve Miller (cello) alongside Pond and Hostetter. A third album, The Green Fury was released in spring 2002 while Pond also took on duties writing music for the Oxygen Network. The Nature of Maps followed several months later.
In 2003, Pond left Philadelphia for Brooklyn, and eventually met the rest of the band that would ultimately become the strongest lineup of Matt Pond PA since its inception. Brian Pearl (guitars/piano), Dan Crowell (drums) and Daniel Mitha (bass) joined Pond for the recording of Emblems, which was released on Altitude in May 2004. Shared dates with Keane and Ted Leo & The Pharmacists followed until the end of the year. Prior to releasing their fifth album, the seasonal EP effort, Winter Songs was issued in January 2005. Several Arrows Later, which also included the debut of cellist Dana Feder, followed in October. Tours dates with Liz Phair and Guster also coincided its release.