Harnoncourt was born in Berlin, was brought up in Graz and studied music in Vienna. He founded Concentus Musicus Wien with his wife, Alice, in 1953 while playing cello with the Vienna Symphony Orchestra. That group was dedicated to authentic performances on period instruments, and by the 1970s his work with it had made him quite well known.
He subsequently performed with many other orchestras using modern instruments, but still with an eye on historical authenticity in terms of tempi, dynamics and so on. He also expanded his repertoire somewhat, continuing to play the baroque works which had made him famous, but also championing the Viennese operetta repertoire.
In 1971, Harnoncourt started a joint project with the conductor Gustav Leonhardt to record all of Johann Sebastian Bach's cantatas. The project was eventually completed in 1990. In 2001 a critically acclaimed and Grammy Award winning recording of Bach's St. Matthew Passion conducted by Harnoncourt was released, which included the entire score of the piece in Bach's own hand on a CD-ROM.