I found this tidbit from the Catholic Answers Forum (give credit where credit is due). Cardinal Ratzinger discussed briefly Luther's attitude toward the church fathers and allegory in his book, Principles of Catholic Theology (pp. 141-142):
The quote is said to come from Luther's Sermons on the Second Book of Moses (Alleg. I, Wittenberger Ausgabe, 16.67). I don't doubt the validity of the quote being used (which was taken by Ratzinger from a secondary source. Luther preached on Exodus in 1524, and I think the reference is to a sermon on Exodus 1). While Luther said both positive and negative things about the church fathers throughout his career, this description of Luther's view is accurate.
As to discrediting the fathers because of their use of allegory, Luther likewise would often provide an allegorical interpretation of a passage of Scripture. Examples of this can be found in his sermons. I can recall my surprise reading a sermon from Luther's Church Postil and coming upon a section entitled, "The spiritual interpretation of this gospel." It wasn't that allegory automatically discredited the church fathers. Using allegorical interpretations as primary proofs was the issue. Luther complained that the papists and fanatics did this. Luther also fought against allegorizing as an accepted and recommended method of biblical interpretation prevalent in his day and previously. The primary proofs of the faith must be based on the clear exposition of scripture. Luther goes on in the very sermon Ratizinger cited to say of allegorical interpretation:
I too know well that hidden meanings do not have convincing power and should not be the ground on which we base ourselves. For we should and must rely on the clear, express, and plain word of God, such as that referring us to faith in Christ and love toward our neighbor. Thus one is saved. Other teachings and allegories you disregard. Such is also the allegory of St. Paul concerning Abraham, according to which his two sons signify the two testaments. For if this allegory were not grounded on this passage in Paul, my heart would doubt and constantly ask how I may be sure of it. For one would say: Who knows whether it be so? The heart must doubt in that case and cannot be sure. It dare not rest or rely on allegories. I must have the plain text and page of Holy Scripture (WA 16:72 cited in What Luther Says, 1:100-101).
Ratzinger then goes to make quite an interesting claim a short while later on the use of the church fathers as biblical interpreters and his earlier summation of Luther's view, that Luther's "historical instinct is clearly proving itself right" and also that "there is nothing to be proved or disproved" by referring to their interpretations of Scripture: