Terim gained hero status by guiding Galatasaray to a UEFA Cup and UEFA Super Cup double in 2000, making them the first Turkish side to win European trophies.
The 68-year-old former Galatasaray defender rejoined the club as manager for a third time in 2017 after spells coaching the national side and in Serie A with AC Milan and Fiorentina.
He led Galatasaray to two more championship titles, increasing their record haul to 23.
The club finished an agonising second last season, level on points with eternal Istanbul rivals Besiktas, but losing the title on goal difference.
But their form has been dire this season, with three losses in the last five matches leaving them stranded in 12th place and closer to the relegation zone than another title.
The team were whistled off the pitch after losing 1-0 at home to Giresunspor on Saturday, prompting Terim to lash out to fans for failing to stand by the club.
“The fans may be angry, they may react, and they may be right. But that is no reason to behave like that,” Terim said after the match.
“Unfortunately, we are going through a season where even on good days, we do not win.”
The TRT state broadcaster said Terim resigned after a meeting with the club president.
The club said in a statement Terim was leaving his duties affective immediately, without providing further details.
Turkish media reports said Galatasaray have already started the search for Terim’s successor, with Spaniard Domenec Torrent expected in Istanbul for preliminary discussions on Tuesday.
The 59-year-old has worked as an assistant coach at Barcelona, Bayern Munich and Manchester City between 2008 and 2018, also briefly managing Brazilian giants Flamengo in 2020.