|Michelangelo's version; I assume Canaan is the one outside|
How did the Sages know what they "knew"? Often via close examination of the text. Here's BT Sanhedrim 70a on the crime committed against Noah:
And Noah began to be an husbandman, and he planted a vineyard: And he drank of the wine and was drunken; and he was uncovered within his tent. And Ham the father of Canaan, saw the nakedness of his father, and told his two brethren without. And Shem and Japheth took a garment, and laid it upon their shoulders, and went backward and covered the nakedness of their father, and their faces were backward, and they saw not their father's nakedness. And Noah awoke from his wine, and knew what his younger son had done unto him. [Until here is a quote from Gen 9 20-24' [With respect to what was done] Rab and Samuel [differ,] one maintaining that he castrated him, whilst the other says that he sexually abused him. He who maintains that he castrated him, [reasons thus;] Since he cursed him by his fourth son, he must have injured him with respect to a fourth son But he who says that he sexually abused him, draws an analogy between 'and he saw' written twice. Here it is written, And Ham the father of Canaan saw the nakedness of his father; whilst elsewhere it is written, And when Shechem the son of Hamor saw her [he took her and lay with her and defiled her].Now, on the view that he emasculated him, it is right that he cursed him by his fourth son; but on the view that he abused him, why did he curse his fourth son; he should have cursed him himself? — Both indignities were perpetrated. [Soncino translation]To summarize:
Everyone agrees something bad happened, because we're told Noah "knew" what was done to him. This makes it clear that the crime was something other than being gazed upon.
Rav is directed to castration [blue above] because of the Mida Kneged Mida, suggesting that Noah was prevented from having a fourth child so he avenged himself on the perpetrator's fourth child.
Shmuel [red above] thinks rape is more likely because we're told that Ham SAW his father and there are many more places where the victim of a rape or sexual abuse is "seen" prior to the attack. Along with the example given of Shechem seeing Dina, the Sons of God in Genesis 6 see the Daughters of Man; Sara is seen by Pharoh; and Potiphar's wife sees Josef.
Additionally there are at least 5 other clues that the crime was something sexual:
1) It says Vayedah Noach, and Noah knew. All over the bible vayedah is used as a euphamism for sex
2) Quite unecessarily, we are told the brothers carried the blanket on their shoulders, וַיָּשִׂימוּ עַל-שְׁכֶם שְׁנֵיהֶם. The seemingly gratuitous mention of Shchem reminds us of the crime committed their against Dina.
3) Canan is everywhere in Scripture used as a representative for wantan immorality. The fact that he is singled out here suggests that the crime he committed was from that category of offenses.
4) There are many, many parallels between the story of Lot and the story of Noah. To make the paralel complete, we must assume that Noah's episode of excessive drinking was followed by an act of immorality, just as Lot's episode of excessive drinking led to an illicit act.
5) Noah was given an express command to procreate. He was also told to leave the ark with his wife (Gen 8:16) but instead leaves with his sons (Gen 8:18) in defiance of the order. Rather than make babies, he made wine, and lounged around in his birthday suit showing open contempt for his obligation. Casteration is the appropriate mida kneged mida,