Consider this. The Temple was destroyed not because the good, kosher, Torah True Jews hated one another, as most presume, but because the various different Jewish sects were always and forever at each others throats.
Take for instance, the Hellenists, who sparked the Great Revolt by defiling a shul in Ceasrea. Or, the Zealots who murdered the entire leadership of the southern command, and terrorized all of Jerusalem. Or, the Scarii who tried to provoke the Jews to fight by burning all the food in Jerusalem, and so terrified Yochana ben Zackai he was forced to escape Jerusalem in a coffin. Or the Jewish-Christians, who had a strong Jerusalem community (led, at one time by James, brother of Jesus) and were already circulating the books that would become the New Testament. In particular, John, the most anti-Semitic of the gospels, is believed by conservative scholars to have been published between 65 and 90, by proto-Christians who were still marginally a Jewish sect.
In short, Jerusalem, in the late 60s, was a wreck. I have very little doubt that all of this - and more - is what the rabbis meant when they said that baseless hatred destroyed the Temple.