ead guitarist Tad Kubler and frontman Craig Finn (both formerly of the band Lifter Puller) first had the idea to form The Hold Steady when sitting at home watching The Last Waltz. Finn asked Kubler, "Dude, why aren't there any bands like this anymore? Let's do this from now on."[2]

The band's 2004 debut album Almost Killed Me, released on Frenchkiss Records, was a sleeper hit among music critics, reaching #31 in the Pazz & Jop critics poll that year - an achievement which Finn described as "like being nominated for an Oscar or something."[3] In May 2005, they were the first rock band to grace the front page of The Village Voice in 15 years. The band was Blender's 2006 Band of the Year.

The band released its second album, Separation Sunday, on May 3, 2005. The album was a loose concept album following the exploits of a character named "Holly" (short for Hallelujah) and her quest to find a balance between Catholicism and rock and roll through drugs and an apparent resurrection. The album was critically lauded, and its religious overtones were picked up by many critics, such as the Village Voice who dubbed it "the most egregiously American Catholic album since X's Under the Big Black Sun, Springsteen's Tunnel of Love, or that Jewish new waver Billy Joel's The Stranger. [3]

After Separation Sunday, for its third album the band moved from the Frenchkiss Records imprint to Vagrant Records. Released October 3, 2006, Boys and Girls in America was ranked #8 on the Rolling Stone Best Albums of 2006 list, and first by The A.V. Club. Pitchfork Media's Scott Plagenhoff wrote of the album, "[Craig Finn] not only has a commanding, rousing voice but he also says something worth hearing, displaying gifts for both scope and depth that are all too rare in contemporary rock - indie or mainstream."[4] The band recorded a modified version of "Take Me Out to the Ballgame" to be played during the seventh-inning stretch at Minnesota Twins home games during the 2007 season. [5]