Thilo Sarrazin, a board member of the Bundesbank, will leave his post at the end of this month.
He has said that Jews "share a particular gene" and has accused Muslims of failing to integrate.
Chancellor Angela Merkel was among several leaders who called for his removal from the board.
Mr Sarrazin, in his book entitled Germany Abolishes Itself, says that Muslim immigrants are a drain on German society.
"Most of the cultural and economic problems are concentrated in a group of the five to six million immigrants from Muslim countries," he stated in the book.
The issue has proved divisive in Germany, with right-wing groups claiming his views vindicate their own stances.
But advocates of improving integration say he has made it harder to hold an objective debate by polarising opinion and obscuring the facts.
Mrs Merkel's office said his controversial remarks were damaging the reputation of the Bundesbank.
And in a brief statement on Thursday, the Central Bank said: "With a view to the public discussions, both sides agreed to end their work together at the end of the month."
BBC News
