Keith B. Davis, proceeding pro se, appeals from the Tax Court’s dismissal for lack of prosecution and its refusal to vacate that order. We affirm.

Davis practiced general and patent law in Minnesota for eleven years before he was suspended in 1978 for misconduct, incompetence, failure to refund a retainer, and failure to represent a client zealously. As a result of his suspension, Davis claimed on his 1977 income tax return a $120,000 “loss of intangible assets” (his law practice). The IRS disallowed his loss and notified Davis of his tax deficiency of $1,018.31. The parties were unable to agree on a stipulation of facts for trial. Therefore, pursuant to Tax Court Rule 91(f), the court ordered the IRS to show cause why Davis’s stipulation was unacceptable. At the show cause hearing, Davis presented no evidence to rebut the showing made by the IRS, but intimated that his tax records had been tampered with “by the U.S. mail” and that he was “a victim.” The Tax Court discharged the order to show cause, denied Davis’s motion to compel stipulation, and scheduled a trial on the merits. Because he viewed the court as being biased, Davis failed to attend the trial and the court granted the government’s motion to dismiss. Davis subsequently filed an affidavit of prejudice and a motion to vacate, both of which were denied.

Under the statutory standard for judicial disqualification, ^[1] judges are charged with an affirmative duty to probe the legal sufficiency of petitioner’s affidavit of prejudice and not to disqualify themselves unnecessarily. National Auto Brokers Corp. v. General Motors Corp., 572 F.2d 953, 958 (2d Cir.1978), cert. denied, 439 U.S. 1072, 99 S.Ct. 844, 59 L.Ed.2d 38 (1979). Affidavits based on conclusions, opinions, and rumors are an insufficient basis for recusal. City of Cleveland v. Krupansky, 619 F.2d 576, 578 (6th Cir.), cert. denied, 449 U.S. 834, 101 S.Ct. 106, 66 L.Ed.2d 40 (1980). For recusal to be necessary, the bias must be personal and extrajudicial. United States v. Phillips, 664 F.2d 971, 1002 (5th Cir.1981), ce rt. denied, 457 U.S. 1136, 102 S.Ct. 2965, 73 L.Ed.2d 1354 (1982). The courts have recognized an exception to this rule in extreme eases of pervasive personal bias and prejudice. Id. 664 F.2d at 1003.

[*1304] The Tax Court was correct in rejecting Davis’s affidavit of prejudice. When the judge asked if the proposed stipulation on file was Davis’s, Davis suggested it appeared to be his, but that it might have been altered “by the U.S. mail,” to which the judge replied, “Oh, come on.” On this exchange, Davis bases his claim concerning the judge’s “insensitive demeanor.” We have reviewed the record and we do not see any impropriety or any reflection of bias in the judge’s remark. Moreover, whatever bias might inhere in the judge’s remark is not extrajudicial and does not appear personal. Certainly his comment does not reflect pervasive bias and prejudice.

Accordingly, the judgment of the Tax Court is affirmed.