Augustine wrote The Retractations late in his life to correct points expressed in his own writings. Here, Augustine corrects his earlier opinion that Peter was the rock of Matthew 16:18. According to Augustine the rock is Christ or Peter’s confession which pointed to the person of Christ:
"In a passage in this book, I said about the Apostle Peter: ‘On him as on a rock the Church was built’...But I know that very frequently at a later time, I so explained what the Lord said: ‘Thou art Peter, and upon this rock I will build my Church,’ that it be understood as built upon Him whom Peter confessed saying: ‘Thou art the Christ, the Son of the living God,’ and so Peter, called after this rock, represented the person of the Church which is built upon this rock, and has received ‘the keys of the kingdom of heaven.’ For, ‘Thou art Peter’ and not ‘Thou art the rock’ was said to him. But ‘the rock was Christ,’ in confessing whom, as also the whole Church confesses, Simon was called Peter. But let the reader decide which of these two opinions is the more probable."
Source:The Fathers of the Church (Washington D.C., Catholic University, 1968), Saint Augustine, The Retractations Chapter 20.1.
Here's another few interesting Augustine quotes on this subject:
"And I tell you...‘You are Peter, Rocky, and on this rock I shall build my Church, and the gates of the underworld will not conquer her. To you shall I give the keys of the kingdom. Whatever you bind on earth shall also be bound in heaven; whatever you loose on earth shall also be loosed in heaven’ (Mt 16:15-19). In Peter, Rocky, we see our attention drawn to the rock. Now the apostle Paul says about the former people, ‘They drank from the spiritual rock that was following them; but the rock was Christ’ (1 Cor 10:4). So this disciple is called Rocky from the rock, like Christian from Christ...Why have I wanted to make this little introduction? In order to suggest to you that in Peter the Church is to be recognized. Christ, you see, built his Church not on a man but on Peter’s confession. What is Peter’s confession? ‘You are the Christ, the Son of the living God.’ There’s the rock for you, there’s the foundation, there’s where the Church has been built, which the gates of the underworld cannot conquer."
Source: John Rotelle, Ed., The Works of Saint Augustine (New Rochelle: New City Press, 1993), Sermons, Vol. 6, Sermon 229P.1, p. 327.
"But whom say ye that I am? Peter answered, ‘Thou art the Christ, The Son of the living God.’ One for many gave the answer, Unity in many. Then said the Lord to him, ‘Blessed art thou, Simon Barjonas: for flesh and blood hath not revealed it unto thee, but My Father which is in heaven.’ Then He added, ‘and I say unto thee.’ As if He had said, ‘Because thou hast said unto Me, “Thou art the Christ the Son of the living God;” I also say unto thee, “Thou art Peter.” ’ For before he was called Simon. Now this name of Peter was given him by the Lord, and in a figure, that he should signify the Church. For seeing that Christ is the rock (Petra), Peter is the Christian people. For the rock (Petra) is the original name. Therefore Peter is so called from the rock; not the rock from Peter; as Christ is not called Christ from the Christian, but the Christian from Christ. ‘Therefore,’ he saith, ‘Thou art Peter; and upon this Rock’ which Thou hast confessed, upon this rock which Thou hast acknowledged, saying, ‘Thou art the Christ, the Son of the living God, will I build My Church;’ that is upon Myself, the Son of the living God, ‘will I build My Church.’ I will build thee upon Myself, not Myself upon Thee.For men who wished to be built upon men, said, ‘I am of Paul; and I of Apollos; and I of Cephas,’ who is Peter. But others who did not wish to built upon Peter, but upon the Rock, said, ‘But I am of Christ.’ And when the Apostle Paul ascertained that he was chosen, and Christ despised, he said, ‘Is Christ divided? was Paul crucified for you? or were ye baptized in the name of Paul?’ And, as not in the name of Paul, so neither in the name of Peter; but in the name of Christ: that Peter might be built upon the Rock, not the Rock upon Peter. This same Peter therefore who had been by the Rock pronounced ‘blessed,’ bearing the figure of the Church."
Source: Philip Schaff, Nicene and Post-Nicene Fathers (Grand Rapids: Eerdmans, 1956), Volume VI, St. Augustin, Sermon XXVI.1-4, pp. 340-341).
"And this Church, symbolized in its generality, was personified in the Apostle Peter, on account of the primacy of his apostleship. For, as regards his proper personality, he was by nature one man, by grace one Christian, by still more abounding grace one, and yet also, the first apostle; but when it was said to him, ‘I will give unto thee the keys of the kingdom of heaven, and whatsoever thou shalt bind on earth, shall be bound in heaven; and whatsoever thou shalt loose on earth, shall be loosed in heaven,’ he represented the universal Church, which in this world is shaken by divers temptations, that come upon it like torrents of rain, floods and tempests, and falleth not, because it is founded upon a rock (petra), from which Peter received his name. For petra (rock) is not derived from Peter, but Peter from petra; just as Christ is not called so from the Christian, but the Christian from Christ. For on this very account the Lord said, ‘On this rock will I build my Church,’ because Peter had said, ‘Thou art the Christ, the Son of the living God.’ On this rock, therefore, He said, which thou hast confessed, I will build my Church. For the Rock (Petra) was Christ; and on this foundation was Peter himself built. For other foundation can no man lay than that is laid, which is Christ Jesus. The Church, therefore, which is founded in Christ received from Him the keys of the kingdom of heaven in the person of Peter, that is to say, the power of binding and loosing sins. For what the Church is essentially in Christ, such representatively is Peter in the rock (petra); and in this representation Christ is to be understood as the Rock, Peter as the Church."
Source: Philip Schaff, Nicene and Post-Nicene Fathers (Grand Rapids: Eerdmans, 1956), Volume VII, St. Augustin, On the Gospel of John, Tractate 124.5.
"Before his passion the Lord Jesus, as you know, chose those disciples of his, whom he called apostles. Among these it was only Peter who almost everywhere was given the privilege of representing the whole Church. It was in the person of the whole Church, which he alone represented, that he was privileged to hear, ‘To you will I give the keys of the kingdom of heaven’ (Mt 16:19). After all, it isn’t just one man that received these keys, but the Church in its unity. So this is the reason for Peter’s acknowledged pre–eminence, that he stood for the Church’s universality and unity, when he was told, ‘To you I am entrusting,’ what has in fact been entrusted to all.I mean, to show you that it is the Church which has received the keys of the kingdom of heaven, listen to what the Lord says in another place to all his apostles: ‘Receive the Holy Spirit;’ and straightway, ‘Whose sins you forgive, they will be forgiven them; whose sins you retain, they will be retained’ (Jn 20:22-23). This refers to the keys, about which it is said, ‘whatever you loose on earth shall be loosed in heaven, and whatever you bind on earth shall be bound in heaven’ (Mt 16:19). But that was said to Peter. To show you that Peter at that time stood for the universal Church, listen to what is said to him, what is said to all the faithful, the saints: ‘If your brother sins against you, correct him between you and himself alone’"
Source: John Rotelle, Ed., The Works of Saint Augustine (Hyde Park: New City, 1994), Sermons, III/8 (273-305A), On the Saints, Sermon 295.1-3, pp. 197-198).
For further reading: The Church Fathers' Interpretation of the Rock of Matthew 16:18 An Historical Refutation of the Claims of Roman Catholicism Includes a Critique of Jesus, Peter and the Keys- By William Webster