Labour's left and right wings both treated the issue of anti-Semitism as a "factional weapon" when Jeremy Corbyn was leader, a report says.
Martin Forde QC's inquiry finds general "toxicity" existed between Mr Corbyn's office and staff at party HQ.
And he says the opposing groups used the issue of anti-Semitism as a weapon during their arguments, rather than confront the issue.
Complaints procedures have improved since but more work is needed, he adds.
Mr Forde's inquiry began in April 2020 after an 860-page dossier - dated March 2020 - was leaked.
It contained private WhatsApp messages and claimed some Labour workers had not wanted Mr Corbyn, on the party's left, to win the 2017 general election and had hindered efforts to tackle anti-Semitism.
The dossier found "no evidence" of anti-Semitism being handled differently from other complaints and blamed "factional opposition" towards Mr Corbyn.
But Mr Forde's report says factionalism was "endemic" within Labour and the issue of anti-Semitism was weaponised by both sides, not just the party's right.
"The evidence clearly demonstrated that a vociferous faction in the party sees any issues regarding anti-Semitism as exaggerated by the right to embarrass the left," it says.
"It was of course also true that some opponents of Jeremy Corbyn saw the issue of anti-Semitism as a means of attacking him.
"Thus, rather than confront the paramount need to deal with the profoundly serious issue of anti-Semitism in the party, both factions treated it as a factional weapon."
'Steps in right direction'
The report also says Labour's disciplinary process was "not fit for purpose" and "potentially prone to factional interference".
However, it adds that "many aspects of the party's recent reforms of disciplinary procedures" are a positive, and changes have been "generally steps in the right direction", although further work is needed.
The report also criticises a "culture of intellectual smugness which exists at the extremes of the political spectrum" of Labour opinion.
Responding to the findings, a Labour spokesperson said: "The Forde report details a party that was out of control.
"Keir Starmer is now in control and has made real progress in ridding the party of the destructive factionalism and unacceptable culture that did so much damage previously and contributed to our [general election] defeat in 2019."
Jeremy Corbyn said the Forde report raised questions about Labour's future direction
But, in a statement, Mr Corbyn said that many in Labour had found it "hard to come to terms" with his "overwhelming" election as leader in 2015.
"In any party there are groups and factions, but the resistance we were faced with went far beyond that," he added.
Mr Corbyn also said the Forde report showed Labour needed to "decide what it is for".
"Are we a democratic socialist party, run by members and affiliated unions, that aims for a fundamental transfer of wealth and power from the few to the many?" he asked. "Or are we something else?"
Hilary Schan, co-chair of the Momentum Group, which supports Mr Corbyn, said the report showed that "right-wing Labour staff members worked to undermine the party's general election chances and its own complaints system, including on anti-Semitism".
"Disgracefully, while tens of thousands of Labour members were pounding the streets to kick the Tories out in favour of a socialist Labour government, these right-wing factional operators were wreaking havoc on the party from within," she added.
In October 2020, a report by the Equality and Human Rights Commission's found Labour to have been responsible for "unlawful" acts of harassment and discrimination during Mr Corbyn's four-and a-half years as party leader.
Its investigation identified serious failings in leadership and an inadequate process of handling anti-Semitism complaints.
Mr Corbyn said the scale of anti-Semitism within Labour had been "dramatically overstated" by his opponents and that he had always been "determined to eliminate all forms of racism".
He was suspended from the party and was readmitted a month later.
But Mr Corbyn was not readmitted to Labour's parliamentary party and continues to sit in the House of Commons as an independent MP.