Lutfur Rahman was removed as mayor of Tower Hamlets in 2015 after being found guilty of electoral fraud.
However, standing for the Aspire party on his first return to politics he beat incumbent Labour mayor John Biggs.
Mr Rahman said he was "so grateful" to be given "another chance... to rebuild and to serve" the borough.
Mr Biggs had held the post of mayor since Mr Rahman had been found responsible for electoral wrongdoing at an Election Court in 2015.
However, he was beaten after second preference votes were counted, taking 33,487 votes compared to the 40,804 won by the Aspire candidate.
Following his victory, Mr Rahman told the BBC he was very pleased voters had given him "another chance to serve them" and "deliver my progressive agenda which I started in 2010".
"The people of the borough gave a verdict today. I was in the court of the people and they said in a loud voice that they wanted Lutfur Rahman and his team to serve them for the next four years," he added.
Mr Rahman suggested that one of his first acts would be to scrap low-traffic neighbourhoods (LTNs), which limit traffic in residential areas and have proved controversial with some voters.
"We're going to look at our roads, we're going to consult and reopen our roads," he said.
Mr Biggs called the result "decisive", adding that while he was disappointed "I must accept it graciously".
"I am proud of our many achievements but clearly this reflects a desire for change.
"I worry about divisive community politics, however, the new mayor must seize the opportunity and deliver on the promises he made," he said.
Reacting on Twitter, Conservative government minister James Cleverly called the result: "Unbelievable, absolutely unbelievable."