It's a mix of British rock and a score by Mark Mothersbaugh
Wes Anderson actually had the soundtrack finished before the he started working on the movie.
https://en.wikipedia.org/wiki/Rushmore_(soundtrack)
"Hardest Geometry Problem in the World" – Mark Mothersbaugh
"Making Time" – The Creation
"Concrete and Clay" – Unit 4 + 2
"Nothin' in the World Can Stop Me Worryin' 'Bout That Girl" – The Kinks
"Sharp Little Guy" – Mark Mothersbaugh
"The Lad With the Silver Button" – Mark Mothersbaugh
"A Summer Song" – Chad & Jeremy
"Edward Appleby (In Memoriam)" – Mark Mothersbaugh
"Here Comes My Baby" – Cat Stevens
"A Quick One, While He's Away" – The Who
"Snowflake Music" (from Bottle Rocket) – Mark Mothersbaugh
"Piranhas Are a Very Tricky Species" – Mark Mothersbaugh
"Blinuet" – Zoot Sims
"Friends Like You, Who Needs Friends" – Mark Mothersbaugh
"Rue St. Vincent" – Yves Montand
"Kite Flying Society" – Mark Mothersbaugh
"The Wind" – Cat Stevens
"Oh Yoko!" – John Lennon
""Ooh La La"" – Faces
"Margaret Yang's Theme" – Mark Mothersbaugh