Jo Tae-gyu made his acting debut in musical theater in 2001, in the Korean staging of popular musicals The Rocky Horror Show and Grease. From 2003 to 2004, using the stage name Kang Ji-hwan, he began appearing in small roles on television, which included the sitcom Nonstop 4, Summer Scent, and Save the Last Dance for Me.

Kang shot to fame in 2005 with Be Strong, Geum-soon!, in his first onscreen leading role as a fastidious doctor who falls for a widowed hairdresser (played by Han Hye-jin). The drama increased his pan-Asian popularity, particularly in Japan, China, and Taiwan. He then made his big screen debut in the role of a Jehovah’s Witness who befriends an unemployed professor in the independent film Host and Guest, which traveled the international film festival circuit.

2008 became a successful year for Kang. He was cast as the titular Joseon folk hero in Hong Gil-dong, written by the Hong Sisters. He said his portrayal was inspired by Stephen Chow, and the fushion-period-dramedy became a TV cult hit. Later that year, he also starred in Rough Cut, a low-budget film directed by Jang Hoon and written by Kim Ki-duk about a gangster who wants to become an actor and a hot-tempered star who acts like a thug. Kang and costar So Ji-sub were praised for their performances, and both won several Best New Actor awards.