Luther said of Pighius:
Christ Himself complains of these matters in Ps. 109:2 ff.: “Wicked and deceitful mouths are opened against Me, speaking against Me with lying tongues. They beset Me with words of hate and attack Me without cause. In return for My love they accuse Me.” For thus the enemies of the truth are accustomed to obscure, traduce, and corrupt the fruits and gains of the Gospel and of salvation among simple and godly hearers. Eck, Cochlaeus, Pighius, and many others are the best contrivers of such calumnies. They adorn themselves with false and counterfeit praises; but they defame us, in order to make us more obnoxious to those who are strangers to our doctrine. Accordingly, they secretly take away what is most beautiful and best for winning over the hearts of simple men, namely, the favor and goodwill of men, by which we could gain and educate many through the Word. We have to be befouled in order that they may be beautiful. “But God will finally cut off all deceitful lips, the tongue that makes great boasts” (Ps. 12:3). Thus Eck recently found an end worthy of his deeds and words when, after losing his reason, he died miserably without acknowledging God and without calling upon Him. The same end also awaits the others. (LW 7:92)Here's a sample of what Calvin wrote in response to Pighius on his attacks against Luther:
It seemed right to make this short response to Pighius's preface quickly, lest readers should be prejudiced with regard to the matter itself through being preoccupied with his charges, and in this I am sure that I have been successful. For it is not part of my present purpose to refute the insults with which he censures Luther's character and way of life, since they are not of much importance for the matter in hand, nor is Luther in need of defence by me, nor finally is Pighius behaving in any way differently from a ravenous starving dog that takes revenge by barking when he finds nothing to bite! For the most severe of his accusations against Luther amounts to no more than his being a hellish monster because he has often been troubled by grave struggies of the conscience equal to the pains and torments of hell.
But really, if this idiot could imagine even in dream what this means, what it implies, he would either be struck dumb or he would rather be changed into an admirer and a praiser of Luther. For it is the common lot of the devout to endure from time to time awful tortures to the conscience, so that taught by these they may become more accustomed to true humility and fear of God. Therefore as each is endowed with particular excellence of character in excess of others, so he is sometimes afflicted in strange and unfamiliar ways, so that he can say that he has been not only surrounded and besieged by the pains of death but devoured by hell itself. So it is necessary for the most excellent of the saints to be as it were choice workshops of God for him marvelously to carry out his judgments beyond all fleshly feeling. Lest I be delayed by citing numerous examples,such was the wrestling of Jacob, in which that celebrated wrestler had no human opponent, but encountered God himself. Now if you want to understand how much he toiled in that arena, consider first the strength of God, and then draw your conclusion also from the fact that though victorious he nevertheless was lame to the end of his life. Let these words be for the devout. For what can you do with Pighius and those irreligious people who have no conscience or feeling of devotion, and who, if anything is said about the judgments of God, receive it as if they were hearing Homer's stories about the banquets of the gods?John Calvin, The Bondage and Liberation of the Will (Grand Rapids: Baker Books, 1996), 21-22.
Reformation scholar Brian Gerrish commented on this section from Calvin saying,
Calvin must be acknowledged as a vigorous and skillful polemicist, with a flair for a telling phrase. More important, no doubt, he displays a sensitive appreciation of Luther's thought and personality. He has grasped the meaning of the famous Anfechtungen (or "spiritual assaults"), and he recognizes in the admitted extravagance of Luther's style precisely the kind of hero for whom the times cried out... In any case, the worst that Pighius could find to say against Luther's character was that he must have been a very monster from hell, since he was often tormented with oppressive struggles of conscience that were like the anguish of the damned. Calvin retorts that if only the windbag Pighius had the least inkling of what those struggles meant, he would either have held his peace or become Luther's admirer. It is the common lot of the godly to undergo 'fearful tortures of conscience' by which they are made familiar with true humility. a man may even say, in times of unusual testing, that he is not only surrounded and beset by the agonies of death, but swallowed up by hell itself. For among the saints there are certain exceptional men whom God has chosen to be the special objects of his strange judgments. The echoes of characteristically Lutheran language in this passage are unmistakeable, and they testify to Calvin's insight into the religious struggles of the German Reformer. (source).The unfortunate thing about both Calvin's and Luther's comments is that (as far as I know) what Pighius actually wrote isn't available (at least in English). It appears that at least one of the charges made by Pighius against Luther was that Luther advocated polygamy. It would be interesting to see what Pighius actually stated. The 16th Century defenders of Romanism were not typically ecumenical, so it wouldn't be surprising to find that Pighius thought of Luther as heretic on his way to hell.