I've done posts with sources on this topic before, but here's an update. Keep it handy and be sure to use it the next time someone insists that Orthodox Jews are required to treat midrash as infallible.
1) Or HaChaim (Bereishis 1:1): You should know that we have permission to explain the implication of the verses after careful study--even when our conclusions differ from the explanation of our Sages. That is because there are 70 faces to Torah. There is no prohibition against differing from the words of our Sages except if it changes the Halacha.
DB: Ayin panim l'Torah means that you can chose one that you prefer and disregard [="pay no attention to; ignore.] the one the Sages liked.
2) Ohr Hachaim Bereishis 46:8: Don't be bothered by the fact that our explanation is the opposite of what our Sages stated. We have already asserted that concerning the understanding of the non-halachic verses of the Torah --permission is given to the diligent student to innovate
DB:Any interpretation that Sages presented of a non-halachic verse can be put aside. You're free to replace it with an interpretation of your own if you wish.
3) Ohr Hachaim Devarim 32:1 Concerning Aggada it is permitted to offer explanations even if they contradict those of our Sages as long as they don't contradict the Halacha.
DB: When it comes to aggada, (ie any non-legal statement Chazal made about any topic, including science and theology, which anyway were not considered separate categories until recently) you are free to contradict [="deny the truth of (a statement)"] Chazal.
4) Ramban on Vayikrah 12:2. He brings the Chazal that the gender of a child depends on which parent is "mazriah" first. He then quotes the Greek philosophers that everything in the baby comes from the mother except for hyly, the generative force. He then explains the pasuk according to the opinion of the Greek philosophers, and against Chazal.
DB: Here Ramban demonstrates that he's not unwilling to use the principle he set down during his Disputation with Pablo Christiani. Though Chzal had a POV, expressed in their aggada, he rejects it ["dismiss as inadequate"] in favor of another idea.
5) Rashbam Shemos 4:10: It is impossible that a prophet who spoke with G-d face to face and received the Torah from Him should have a speech impediment. Such an assertion is not found in the words of the Tanayim and Amoraim. We don't concern ourselves with what is written in sefer chitzonim (eg shmos raba where it says Moshe stuttered)
DB: If its in a Midrash we don't have to concern ourselves with it!
6) Ramban Genesis 8:4: This [Rashi] wrote on the basis of his calculations and its found in Berashis Raba. However, Rashi in some places minutely examine the Midrash and on the same verses provides a [contradictory] interpretative, so he has given us permission to do the same.
DB: Once again Ramban is happy to show that he eats his own cooking. The Midrash says one thing, but we have permission to contradict it [="deny the truth of (a statement)"]
7) R. Yehudah ben Bilam of Toledo: ...if the words of the ancient [Sages] contradict the intellect, we are not obliged to accept them.
DB: Use your brains people! You're not required to blindly follow Chazal where their words are self-evidently wrong.
8) R. Shemu’el ha-Naggid’s Mevo ha-Talmud: Aggadah is every explanation that comes in the Talmud regarding any matter that is not a mizvah. This is Aggadah; and you should learn from it only what arises in the mind.… What [the Sages] interpreted in [Scriptural] verses is [for] each one according to what occurred to him and what he saw in his mind. And according to what arises in the mind from these interpretations, one learns it; and one does not rely upon the rest.
DB: The Sages interpreted the the Torah [ie created Midrash] according to whatever ideas struck them as most true; you're allowed to do the same.
9) Sefer Haeshkol: One should not rely upon Aggadah and Midrash, even though they are written in the Talmud, if they are unattuned or erroneous. For our principle is: One does not rely upon the Aggadah.
DB: Does it get any clearer? One does not rely on the Aggadah.
10) Chai Gaon: Words of Aggadah are not like a tradition; rather, everyone expounds what arises in his heart — like ‘it is possible’ and ‘one may say’ — not a decisive statement. Therefore, one does not rely upon them.
DB: Absolutely crystal! Aggada != tradition and you don't have to rely on it!
11) R. Yehִ i’el ben Yosef of Paris: Words of Aggadah, to draw the heart of man ... if you desire — believe them; and if you do not desire — do not believe them, for no law is determined based upon them.
DB: Unless the man was lying this seems dead obvious: If you don't want to believe aggadah, then don't believe aggadah. It's fine!
12) R. Samson Rephael Hirsch: It is possible that some of [the stories in the Talmud and Midrash] were stated only in the manner of rhetorical invention [Heb. hamzִ a’at melizah], for some ethical or educational goal. And even if one says that the [apparently historical] stories of Avraham’s life with Terahִ and Nimrod in Ur Kasdim were a rhetorical invention — [we] should not reject [that position] with two hands.
DB: Even the seemingly historical story of Abraham in the furnace might be an invention!
Search for more information about ### at4torah.com