Petition for review granted; order vacated and remanded by published opinion. Judge AGEE wrote the opinion, in which Judge NIEMEYER and Judge DUNCAN joined.

Jerry Addison applied for financial assistance under the Black Lung Benefits Act, 30 U.S.C. §§ 901-944 (“the Act”), claiming that he suffered from coal-dust induced pneumoconiosis as a result of his prior work as a coal miner. Over conflicting medical evidence, an Administrative Law Judge (“ALJ”) found that Addison was entitled to benefits under the Act because he had established the existence of clinical and legal pneumoconiosis that resulted in a total respiratory disability. Addison’s former employer, Sea B Mining Co. (“Sea-B”), filed a petition for review, arguing the ALJ erred in several ways which were not harmless. For the reasons described below, we grant the petition for review, vacate the order awarding benefits, and remand for further proceedings. ^[1]

I.

We begin with a brief discussion of the statutory and regulatory framework, which provides context for the events of this case. The Act creates an adversarial, administrative procedure designed to determine whether miners (or their surviving dependents) qualify for compensatory benefits because they suffer from coal dust-related pulmonary injuries, commonly categorized as pneumoconiosis. See 30 U.S.C. §§ 901-944. The implementing regulations define pneumoconiosis as a “chronic dust disease of the lung and its sequelae, including respiratory and pulmonary impairments, arising out of coal mine employment.” 20 C.F.R. § 718.201(a).

Courts recognize two forms of pneumo-coniosis: “clinical” and “legal.” See Clinch-field Coal Co. v. Fuller, 180 F.3d 622, 625 (4th Cir. 1999). ^[2] Clinical pneumoconiosis “consists of those diseases recognized by the medical community as pneumoconiosis, 1.e., the conditions characterized by permanent deposition of substantial amounts of particulate matter in the lungs and the fibrotic reaction of the lung tissue to that deposition caused by dust exposure in coal mine employment.” • 20 C.F.R. § 718.201(a)(1). Legal pneumoconiosis, by contrast, “encompasses a wide variety of conditions ... whose etiology is not the inhalation of coal dust, but whose respiratory and pulmonary symptomatology have nonetheless been made worse by coal dust exposure.” Clinchfield, 180 F.3d at 625. The regulations thus define legal pneumo-coniosis as “any chronic lung disease or impairment and its sequelae arising out of coal mine employment.” 20 C.F.R. § 718.201(a)(2).

To obtain black lung benefits under the Act, a claimant must prove by a preponderance of the evidence that: “(1) he has [either kind of] pneumoconiosis; (2) the pneumoconiosis arose out of his coal mine employment; (3) he has a totally disabling respiratory or pulmonary condition; and (4) pneumoconiosis is a contributing cause to his total respiratory disability.” Milburn Colliery Co. v. Hicks, 138 F.3d 524, 529 (4th Cir. 1998). The parties agreed that Addison suffered from a disabling respiratory condition that prevénted[*249] further employment. The issue below, and on review, is whether Addison’s disability was the result of pneumoconiosis arising out of his coal mine employment. The dispute centers around the exclusion and consideration of certain medical evidence and the ALJ’s conclusions in evaluating the expert medical opinions.

A claimant may establish the existence of pneumoconiosis by, among other means, chest x-rays and medical opinion evidence. See 20 C.F.R. § 718.202(a). In addition, “[t]he results of any medically acceptable test or procedure ..., which tends to demonstrate the presence or absence of pneumoconiosis ... may be submitted in connection with a claim and shall be given appropriate consideration.” 20 C.F.R. § 718.107(a). Although the regulations group the forms of permissible evidence into discrete categories, an ALJ must weigh all of the evidence together when determining whether the miner has established the presence of pneumoconiosis. See Island Creek Coal Co. v. Compton, 211 F.3d 203, 208-09 (4th Cir. 2000).

II.

A.

Addison worked in the coal industry for approximately 12 years. ^[3] Prior to abandoning this line of work in 1981 due to a neck fracture and arthritis, his employment consisted of stints as a general laborer, scoop operator, and finally foreman with Sea-B. As often occurs in these cases, Addison was a cigarette smoker, and his smoking history far exceeds the length of his mining career. Addison began his pack-a-day smoking habit in 1956 and stopped sometime between 2001 and 2012. The evidence is clear that Addison suffered from a myriad of ailments during the latter part of his life that, if not caused by smoking, were certainly amplified by this activity. Among other things, he had a history of arthritis, coronary artery disease, hypertension, and diabetes.

In March 2011, Addison filed the present claim for living miner benefits. ^[4] His case was referred to a claims manager, who found that Addison was entitled to benefits due to his prior coal employment. Sea-B disputed the award and sought administrative review before an ALJ.

At the ensuing hearing, Addison testified about his employment history, explaining that he worked in “very thick dust” while at the mines. J.A. 52. He also testified about his decade of breathing problems, for which he had been prescribed oxygen and other pulmonary medications.Apart from Addison’s testimony, the parties introduced various medical evidence concerning his condition, including (1) conflicting interpretations of several chest x-rays; (2) three CT scans which all read negative for pneumoconiosis; (3) the results from pulmonary function tests and arterial blood gas studies; (4) hospitalization and treatment records; and (5) conflicting medical opinions from three physicians, Dr. J. Randolph Forehand, Dr. Gregory J. Fino, and Dr. James R. Castle, all of whom agreed that Addison was total[*250] ly disabled by a respiratory impairment but differed as to its cause and type.

Dr. Forehand, who performed the Department of Labor sponsored examination, diagnosed Addison as having both pneumo-coniosis and a non-disabling ventilatory impairment caused by cigarette smoking. His opinion was based on an arterial blood gas study showing impaired gas exchange during exercise, a single 2011 chest x-ray, and Addison’s history of coal dust exposure. Had Addison not worked in the mines, Dr. Forehand opined, “his. arterial blood gas would no doubt be normal and his chest x-ray clean.” J.A. 104.

Dr. Fino diagnosed Addison with “idiopathic interstitial fibrosis” that, although disabling, is “unrelated to coal dust inhalation.” J.A. 154. As support for this opinion, Dr. Fino cited the “marked progression” of Addison’s lung condition between 2008 and 2011, as evidenced by the photographic progression in the CT scans and x-rays. J.A. 153. He explained that the worsening of Addison’s illness “occurred far too rapidly to be consistent with coal-mine-dust inhalation.” J.A. 205. Dr. Fino further testified that although coal workers’ pneumo-coniosis can cause pulmonary fibrosis, the medical evidence did not support such a diagnosis here. “Coal dust causes nodular fibrosis,” Dr. Fino explained, “[wjhereas this fibrosis [Addison] has is a diffuse type” which is “completely different pathologically and radiographically.” J.A. 213. Dr. Fino also noted that Addison’s fibrosis was restrictive in nature, which is not characteristic of pneumoconiosis. Finally, Dr. Fino averred that he was in the best position to assess Addison’s condition because he had the benefit of reviewing lung imagery over time, whereas Dr. Forehand had only conducted “a one-time review of a chest x-ray.” J.A. 207.

Dr. Castle reached a similar conclusion as Dr. Fino, opining that Addison suffered from idiopathic pulmonary fibrosis. After reviewing essentially the same evidence, Dr. Castle explained that these tests revealed “linear, irregular type opacities which are not typical of coal workers’ pneumoconiosis.” J.A. 267-68. Dr. Castle further noted that idiopathic pulmonary fibrosis is a disease of unknown cause but is associated with heavy cigarette smoking and not coal dust exposure.

B.

In deciding that Addison established the existence of pneumoconiosis, the ALJ evaluated several items of conflicting medical evidence. He first considered the x-ray evidence, which consisted of three chest images dated January 2009, February 2011, and May 2011. The ALJ found the first two x-rays in equipoise as to the existence of pneumoconiosis because similarly qualified doctors rendered conflicting interpretations for each. As to the May 2011 x-ray, the ALJ noted that “Dr. Forehand and Dr. Miller interpreted it as positive for pneumoconiosis ..., while Dr. Scott interpreted the same x-ray as negative for pneumoconiosis.” J.A. 12. ^[5] Observing that Drs. Miller and Scott were both equally qualified “B-readers and board-certified radiologists,” the ALJ determined that the “May 20, 2011 chest x-ray is overall positive for clinical pneumoconiosis” because Dr. Miller’s positive reading was “supported by Dr. Forehand’s opinion.” Id. Dr. Forehand is a certified B-reader but not a radiologist. ^[6]

[*251] The ALJ next considered the CT scan evidence and noted such scans do not fall within the category of traditional x-ray evidence, and consequently they “must be weighed with other acceptable medical evidence under 20 C.F.R. § 718.107.” J.A. 13-14. The ALJ further interpreted the regulations as allowing “only one reading of ‘other evidence’ such as CT scans.” J.A. 14. Accordingly, although Sea-B had offered three CT scans spanning 2008 to 2012, all with negative readings for pneumoconiosis, the ALJ determined Sea-B was entitled to admit only one scan. Without explanation, the ALJ picked Addison’s July 2012 CT scan, which showed “no changes consistent with a coal mine dust associated occupational disease,” as evaluated by Dr. Fino. Id. Because Dr. Fino’s testimony was undisputed, the ALJ concluded that “the CT scan evidence d[id] not support a finding of clinical or legal pneumoconiosis.” Id.

After discounting Addison’s treatment and hospital records as non-probative, the ALJ lastly turned to the conflicting medical opinions from Drs. Forehand, Fino, and Castle. The ALJ accorded the most weight to Dr. Forehand, finding his view consistent with the Department" of Labor’s position that coal dust exposure and cigarette smoking are additive in producing significant airway obstruction. J.A. 20. On this point, the ALJ referenced 65 Fed. Reg. 79,940 (Dec. 20, 2000), which notes that “[c]oal dust exposure is additive with smoking in causing clinically significant airways obstruction and chronic bronchitis.” The ALJ further found Dr. Forehand’s opinion supported by unidentified “diagnostic testing.” J.A. 20.

The ALJ discredited the opinions of Drs. Fino and Castle for several reasons. He found, among other things, that their diagnoses overemphasized the fact that Addison’s pulmonary impairment was restrictive in nature, rather than obstructive, when evaluating the existence of pneumo-coniosis. The regulations, on the other hand, state that legal pneumoconiosis includes “any chronic restrictive or obstructive pulmonary disease arising out of coal mine employment.” 20 C.F.R. § 718.201(a)(2). The ALJ also found the opinions “based on generalities, rather than focusing on [Addison’s] condition.” J.A. 19. In the end, the ALJ concluded that the medical opinion evidence weighed in Addison’s favor.

Specifically crediting his determination of the x-ray evidence and Dr. Forehand’s report over the remaining record, the ALJ found that Addison had established the existence of both clinical and legal pneu-moconiosis by a preponderance of the evidence. After further finding that Addison’s pneumoconiosis arose out of his prior employment and was a substantially contributing cause of his disability, the ALJ awarded benefits under the Act.

C.

Sea-B filed an administrative appeal with the Benefits Review Board (“Board”), specifically disputing the ALJ’s finding that the medical evidence established the existence of pneumoconiosis. ^[7] In a split decision, the Board affirmed.

[*252] Agreeing with Sea-B, the majority conceded that the ALJ “erred in considering only one of the three CT scans [Sea-B] submitted in its affirmative case.” J.A. 30. Ultimately, however, the majority declined to vacate the award, finding that Sea-B failed to show this error was harmful. Id. They faulted Sea-B for not proffering a “specific explanation of how the [ALJ’s] error could have made a difference.” Id. The dissent took issue with this conclusion, explaining that, “[b]y the majority’s reasoning, improper exclusion of evidence would always be harmless error because it is not possible to determine with certainty its effect on the trier-of fact’s ultimate determination.” J.A. 34. Given this error, the dissent explained, the ALJ’s “overall finding of the existence of pneumoconiosis ... is tainted” and should be sent back. J.A. 35.

The majority also rejected Sea-B’s arguments against the ALJ’s method of weighing the medical evidence. In response to Sea-B’s contention that the ALJ impermis-sibly resolved the conflicting x-ray evidence by resorting to a headcount, the Board concluded that he “properly considered the weight of the positive x-ray readings in light of the readers’ qualifications.” J.A. 28. The Board further sustained the ALJ’s decision to accord “less weight to [Dr. Fino’s and Dr. Castle’s] opinions because the[se] physicians were not able to adequately explain the bases for their conclusions.” J.A. 32.

Following the Board’s unfavorable decision, Sea-B timely filed the instant petition for review. We have jurisdiction under 33 U.S.C. § 921(e).

III.

Our review of a decision awarding black lung benefits is “limited.” Harman Mining Co. v. Dir., OWCP, 678 F.3d 305, 310 (4th Cir. 2012). We evaluate the legal conclusions of the Board and ALJ de novo but defer to the ALJ’s factual findings if supported by substantial evidence. See Hobet Mining, LLC v. Epling, 783 F.3d 498, 504 (4th Cir. 2015) (“We ask only whether substantial evidence supports the factual findings of the ALJ and whether the legal conclusions of the Board and ALJ are rational and consistent with applicable law.”). “Substantial evidence is more than a mere scintilla. It means such relevant evidence as a reasonable mind might accept as adequate to support a conclusion.” Consol. Edison Co. of N.Y.’v. NLRB, 305 U.S. 197, 229, 59 S.Ct. 206, 83 L.Ed. 126 (1938).

Applying this standard, we do not undertake to reweigh contradictory medical evidence, make credibility determinations, or substitute our judgment for that reached below. Rather, the duty to resolve conflicts in the evidence rests with the ALJ as factfinder. And when conflicting evidence allows reasonable minds to differ as to whether a claimant is disabled or has pneumoconiosis, the responsibility for that decision falls on the ALJ. See Harman Mining Co., 678 F.3d at 310.

That said, our deference to an ALJ’s factual findings is not unlimited. An ALJ must still conduct “an appropriate analysis of the evidence to support his conclusion.” Milburn Colliery Co., 138 F.3d at 529. As this Court has previously explained, “[u]nless the [ALJ] has analyzed all evidence and has sufficiently explained the weight he has given to [the] exhibits, to say that his decision is supported by substantial evidence approaches an abdication of the court’s duty to scrutinize the record as a whole to determine whether the conclusions reached are rational.” Sterling Smokeless Coal Co. v. Akers, 131 F.3d 438, 439-40 (4th Cir. 1997). Thus, “[e]ven if legitimate reasons exist for rejecting [or[*253] crediting] certain evidence, the [ALJ] cannot do so for no reason or for the wrong reason.” King v. Califano, 615 F.2d 1018, 1020 (4th Cir. 1980). Where an ALJ has incorrectly weighed the evidence or failed to account for relevant record evidence, deference is not warranted and remand is frequently required. See Island ’Creek Coal Co. v. Compton, 211 F.3d at 213.

Furthermore, as a condition to appellate review, an ALJ must “adequately explain why he credited certain evidence and discredited other evidence.” Milburn Colliery Co., 138 F.3d at 533; see also Consolidation Coal Co. v. Filer, No. 95-1270, 1996 WL 139196, at *5 (4th Cir. Mar. 26, 1996) (“Decisions on conflicting evidence ... must be addressed and explained at the administrative level before judicial review under the substantial evidence standard can be accomplished meaningfully.”). Although this requirement “is not intended to be a mandate for administrative verbosity,” a reviewing court must be able to “discern what the ALJ did and why he did it.” Piney Mountain Coal Co. v. Mays, 176 F.3d 753, 762 n. 10 (4th Cir. 1999).

With this standard of review as the backdrop, we turn to Sea-B’s arguments.

IV.

A.

Sea-B initially contends that the ALJ’s decision to consider only one of the three CT scans included in the record was error requiring reversal of the ALJ’s judgment. Addison does not argue in favor of the ALJ’s evidentiary ruling; rather, like the Board, he contends Sea-B failed to show this error was prejudicial.

We agree with the parties and the Board that the ALJ erred: Sea-B was entitled to submit, and the ALJ was required to consider, one reading of each CT scan under 20 C.F.R. § 718.107. The remaining issue is whether Sea-B is entitled to any relief for that error.

Sea-B appears to contend that this evidentiary error requires remand without further inquiry. On brief, Sea-B argued that when an ALJ fails to review all relevant evidence, “[a]ppellate review is impossible.” Opening Br. 21. This argument sweeps too broadly. Administrative adjudications are subject to the same harmless error rule that generally applies to civil cases. Reversal on account of error is not automatic but requires a showing of prejudice. See Consolidation Coal Co. v. Williams, 453 F.3d 609, 621-22 (4th Cir. 2006). The harmless error rule applies to agency action because if the agency’s mistake did not affect the outcome, it would be senseless to vacate and remand for reconsideration. The rule of prejudicial error further prevents reviewing courts from becoming “impregnable citadels of technicality” and preserves the relative roles of courts and agencies in implementing substantive policy. Shinseki v. Sanders, 556 U.S. 396, 407, 129 S.Ct. 1696, 173 L.Ed.2d 532 (2009).

The burden to demonstrate prejudicial error is on Sea-B, the party challenging the agency action. Id. at 409, 129 S.Ct. 1696. However, the harmless error rule is “not ... a particularly onerous requirement.” Id. at 410, 129 S.Ct. 1696. As the Supreme Court has explained, “[o]f-ten the circumstances of the case will make clear to the appellate judge that the ruling, if erroneous, was harmful and nothing further need be said.” Id. Our determination of prejudice ultimately requires “case-specific application of judgment, based upon examination of the record,” not “mandatory presumptions and rigid rules.” Id. at 407, 129 S.Ct. 1696. In each case, an appellate court must consider “the likeli[*254] hood that the result would have been different,” as well as how the error might impact the public perception of such proceedings. Id. at 411, 129 S.Ct. 1696.

In claiming that the error here was not harmless, Sea-B relies on our decision in Island Creek Coal Co. v. Compton,, where we held "all relevant evidence is to be considered” in evaluating a claim for black lung benefits. 211 F.3d at 208. Selecting this language out of context, Sea-B posits that the ALJ’s failure to evaluate the full spectrum of CT scan evidence is per se prejudicial. We cannot agree. Sea-B’s proposed per se rule is contrary to the Supreme Court’s direction not to determine prejudice through “mandatory presumptions and rigid rules.” Shinseki, 556 U.S. at 398, 129 S.Ct. 1696. The use of such presumptions, the Court explained, “exhibit[s] the very characteristics that Congress sought to discourage,” because it prevents the court “from resting its conclusion on the facts and circumstances of the particular case.” Id. at 408, 129 S.Ct. 1696. Furthermore, Sea-B has not suggested any basis for concluding that an ALJ’s failure to evaluate every piece of relevant evidence will always shape the outcome. For example, we could hardly find prejudice where the excluded evidence was merely cumulative or concerned an uncontested point. See Hall v. Arthur, 141 F.3d 844, 850 (8th Cir. 1998) (“The exclusion of cumulative evidence, of course, is merely harmless error.”).

Although we reject Sea-B’s proposed per se rule, we agree that the ALJ’s decision to exclude the additional CT scan evidence was not harmless. This error affects the determination of both clinical and legal pneumoconiosis and impacts the ALJ’s consideration of the other evidence in this case.

The omitted CT scan evidence is unquestionably probative of the central issue in dispute: whether Addison suffered from pneumoconiosis. Considered in aggregate, the scans show a timeline of the progression of Addison’s condition that neither Dr. Forehand nor the ALJ addressed. As explained by Dr. Fino, this progression “occurred far too rapidly to be consistent with coal-mine-dust inhalation.” J.A. 205. While the ALJ was not bound to accept this conclusion, he was required to consider it and explain why he found other evidence more persuasive. Although the ALJ acknowledged that such evidence “must be weighed with other acceptable medical evidence,” J.A. 13-14, he failed to undertake that task.

When the record contains the results of a medically acceptable test that is probative of pneumoconiosis, particularly where it is uncontested, the factfinder must consider that evidence in his analysis. See Dixie Fuel Co., LLC v. Dir., OWCP, 700 F.3d 878, 880 (6th Cir. 2012); Shelton v. Old Ben Coal Co., 933 F.2d 504, 507 (7th Cir. 1991). That obligation went unfulfilled here, as the ALJ failed altogether to weigh the CT scan evidence against the remaining record in his decision. And this error had multiple ramifications. By excluding the CT scans, the ALJ was unable to properly weigh the CT scan evidence as a whole, particularly as probative of idiopathic pulmonary fibrosis and exclusive of pneumoconiosis. Further, the CT scans would have contradicted the ALJ’s findings as to the x-ray evidence. Consequently, the ALJ never considered, much less explained, how the CT scan evidence would weigh against the x-ray evidence or impact his consideration of the overall record. See Milburn Colliery Co., 138 F.3d at 531 (explaining the general rule that an ALJ must “consider all the relevant evidence presented”).

In a related way, the exclusion of the CT scan evidence rendered the ALJ’s consid[*255] eration of the medical opinions of Drs. Fino and Castle inadequate. The ALJ correctly noted that “[w]hen CT scans are evaluated by qualified experts ... they are important diagnostic tools that have resulted in major improvements in the assessment of occupational lung disease.” J.A. 14. And the ALJ credited both Drs. Fino and Castle as such experts. However, the ALJ never considered the importance of the CT scan timeline to the opinions of Drs. Fino and Castle. See J.A. 153, 267-69. The two doctors tied their progression diagnosis to extensive support in the medical literature and other physical tests, none of which the ALJ addressed. Again, the ALJ is not required to accept the medical diagnosis that is shown by the CT scans and their analysis in the medical opinions. But he is required to consider that evidence and explain the reasons for finding another analysis entitled to more weight. See King, 615 F.2d at 1020 (“Even if legitimate reasons exist for rejecting or discounting certain evidence, the [ALJ] cannot do so for no reason or for the wrong reason.”). Thus, by erroneously excluding the CT scan evidence, the ALJ’s opinion was significantly flawed on all these fronts to the prejudice of Sea-B.

We have previously recognized that prejudice is a natural effect of an error of this kind. See Consolidation Coal Co. v. Brown, 230 F.3d 1351 (4th Cir. 2000); fed land Creek Coal Co. v. Groves, 246 Fed.Appx. 842, 846 (4th Cir. 2007); -see also Eastover Mining Co. v. Williams, 338 F.3d 501, 508 (6th Cir. 2003) (stating where “an ALJ has improperly characterized the evidence or failed to account [for] relevant record material, deference is inappropriate and remand is required”). And here, that prejudice is magnified because it is intertwined with the ALJ’s findings as to the x-ray evidence and the medical opinions, not just the CT scan evidence.

Given that the record is otherwise comprised of contradictory evidence regarding Addison’s diagnosis, this excluded evidence could have materially affected the ALJ’s decision. See, e.g., Carnevale v. Gardner, 393 F.2d 889, 891 (2d Cir. 1968) (“[I]t is clear that in summarizing and sifting the evidence in this case, the Hearing Examiner totally ignored a major piece of evidence which might well have influenced his decision. We cannot fulfill the duty entrusted to us ... if we cannot be sure that he considered some of the more important evidence presented!.]”). Rather than assessing and rejecting a single negative CT scan, the ALJ should have weighed all three negative CT scans along with the other credited evidence.

The error was not harmless and warrants remand to ensure the ALJ fully considers .the entire record, particularly in relation to the opinions of Drs. Fino and Castle, who, as discussed in more detail below, both found that these scans affected the evaluation of the x-ray evidence and discredited a diagnosis of pneumoconiosis. See Stout v. Comm’r Soc. Sec. Admin., 454 F.3d 1050, 1056 (9th Cir. 2006) (explaining that an error is harmless only if a court can conclude with confidence that “no reasonable ALJ, when fully crediting the testimony, could have reached a different [result]”).

B.

Sea-B next argues that the ALJ erred by utilizing a headcount of the x-ray readings to conclude Addison suffered from pneumoconiosis. Because the record is insufficiently developed to permit appellate review of this issue, we must vacate and remand for the ALJ to provide an explanation for his decision. See Consolidation Coal Co., 1996 WL 139196, at *5 (“Decisions on conflicting evidence ... must be addressed and explained at the administra[*256] tive level before judicial review under the substantial evidence standard can be accomplished meaningfully.”).

As noted, the ALJ considered three chest x-rays dated January 12, 2009, February 28, 2011, and May 20, 2011, in his pneumoconiosis analysis. He found the first two “in equipoise as to the existence of pneumoconiosis” because “equally qualified” B-reader radiologists had rendered contradictory opinions on each image. See J.A. 12. Turning to the May 20, 2011 x-ray, however, the ALJ found it positive for pneumoconiosis and stated the following:

There were three readings of the most recent x-ray taken on May 20, 2011. Dr. Forehand and Dr. Miller interpreted it as positive for pneumoconiosis ..., while Dr. Scott interpreted the same x-ray as negative for pneumoconiosis. Dr. Forehand is a B reader but not board certified in radiology. Drs. Scott and Miller are both dually qualified as B-readers and board-certified radiologists. Dr. Miller’s opinion that the x-ray is positive for clinical pneumoconiosis is supported by Dr. Forehand’s opinion. Consequently, I find that the May 20, 2011, chest x-ray is overall positive for clinical pneumoconio-sis.

J.A. 12.

Sea-B asserted before the Board that the ALJ did not weigh this evidence on a valid basis, but instead resolved the issue by a headcount of expert witnesses. The Board disagreed and upheld the ALJ’s conclusion, stating that he “properly considered the weight of the positive x-ray readings in light of the readers’ qualifications.” J.A. 28. The record basis the Board referenced was its statement “that the May 20, 2011 x-ray evidence was positive for pneumoconiosis, as it was read as positive by both Dr. Miller and Dr. Forehand, and as negative only by Dr. Scott.” J.A. 28.

When engaged in fact finding, administrative agencies may not base a decision on the numerical superiority of the same items of evidence. See Sterling Smokeless Coal Co., 131 F.3d at 441 (“By resolving the conflict of medical opinion solely on the basis of the number of physicians supporting the respective parties, the ALJ below committed ... error[.]”). In assessing such evidence, the ALJ must articulate specific reasons and provide support for favoring one medical reading over another. See Milburn Colliery Co., 138 F.3d at 536; see also Mitchell v. OWCP, 25 F.3d 500, 508 (7th Cir. 1994) (observing that an ALJ may not substitute his judgment for that of the medical evidence). We have rejected the practice of simply resorting to a numerical headcount as “hollow” and not consistent with an ALJ’s duties in making a reasoned decision. Adkins v. Dir., OWCP, 958 F.2d 49, 52 (4th Cir. 1992); see also Mullins Coal Co., Inc, of Va. v. Dir., OWCP, 484 U.S. 135, 149 n. 23, 108 S.Ct. 427, 98 L.Ed.2d 450 (1987) (explaining that an ALJ must “weigh the quality, and not just the quantity, of the evidence”).

We cannot decipher from the ALJ’s sparse explanation how, or if, he weighed the x-ray readings in light of the readers’ qualifications. To conduct appellate review, we must be able to identify that the ALJ “has analyzed all evidence and has sufficiently explained the weight he has given to [the] exhibits.” Sterling Smokeless Coal Co., 131 F.3d at 439. Without a more specific record of the ALJ’s rationale for reaching his decision as to the May 20 x-ray, we are unable to adequately perform our judicial review function to assure that the ALJ’s decision is based on a “reasoned explanation.” Adkins, 958 F.2d at 52. We cannot guess at what the ALJ meant to say, but didn’t because “[established precedent dictates that a court may[*257] not guess at what an agency meant to say, but must instead restrict itself to what the agency actually did say.” Nken v. Holder, 585 F.3d 818, 822 (4th Cir. 2009).

Consequently, on remand, the ALJ should provide an explanation for his decision concerning the May 20 x-ray by explaining how he weighed the evidence “in light of the readers’ qualifications” and whether his conclusion was based on a numerical headcount of experts. With a “reasoned explanation” in the record, the court would then be in a position to conduct appellate review should that issue arise again. Adkins, 958 F.2d at 52.

C.

Lastly, we turn to the ALJ’s consideration of the medical opinion evidence, particularly in view of the disposition of the issues involving the CT scan and x-ray evidence. As noted, three physicians submitted reports regarding the cause of Addison’s disability. Dr. Forehand opined that Addison suffered from pneumoconio-sis based on a physical examination in combination with Addison’s prior occupational exposure to coal dust. Drs. Fino and Castle, on the other hand, agreed that Addison’s respiratory troubles were unrelated to coal dust exposure. The ALJ sided with Dr. Forehand, finding his diagnosis worthy of the greatest weight.

Although it is within the ALJ’s statutory authority to evaluate and weigh medical opinion evidence, an ALJ may not credit or discredit expert testimony “for no reason or for the wrong reason.” King, 615 F.2d at 1020; see also Island Creek Coal Co. v. Compton, 211 F.3d at 211 (“The ALJ must examine the reasoning employed in a medical opinion in light of the objective material supporting that opinion, and also must take into account any contrary test results or diagnoses.”). In the absence of an adequate explanation supporting the ALJ’s evaluation, “to say that his decision is supported by substantial evidence approaches an abdication of the court’s duty to scrutinize the record as a whole to determine whether the conclusions reached are rational.” Sterling Smokeless Coal Co., 131 F.3d at 439-40; see also Milburn Colliery Co., 138 F.3d at 533 (finding error where “the ALJ failed to adequately explain why he credited certain evidence and discredited other evidence”).

The ALJ credited Dr. Forehand’s diagnosis based on its purported consistency with the following passage from the preamble to the amended regulations: “Coal dust exposure is additive with smoking in causing clinically significant airways obstruction and chronic bronchitis.” J.A. 20 (citing 65 Fed. Reg. 79,940 (Dec. 20, 2000)). It is well settled that a factfinder may consult the Act’s preamble in assessing medical opinion, evidence. See Harman Mining Co., 678 F.3d at 314-15. Nevertheless, the ALJ erred in relying on this passage here because it has no .bearing on Dr. Forehand’s pneumoconiosis opinion.

Although Dr. Forehand diagnosed Addison with an obstructive impairment, he attributed that impairment solely to cigarette smoking and found it non-disabling. See J.A. 104. Dr. Forehand’s diagnosis of legal pneumoconiosis, instead, was based on an arterial blood gas study showing weakened gas exchange and the May 2011 x-ray, which he concluded would have been different had Addison never been exposed to coal dust. However, Dr. Forehand never says why he reached that conclusion, particularly since he never found coal dust exposure related to Addison’s obstructive impairment. Quite the opposite, Dr. Forehand’s opinion contradicts the preamble text, as he found the obstructive respiratory impairment was attributed entirely to[*258] smoking without any aggravation from coal dust exposure.

Because this proffered explanation for elevating Dr. Forehand’s diagnosis is not supported, the ALJ must reevaluate that opinion to determine the proper weight it should be given. See Island Creek Coal Co. v. Compton, 211 F.3d at 211-12; King, 615 F.2d at 1020 (“Even if legitimate reasons exist for rejecting or discounting certain evidence, the [ALJ] cannot do so for no reason or for the wrong reason.”).

As noted earlier, the ALJ’s error as to the CT scan evidence (and the uncertainty as to the validity of the ALJ’s determination on the x-ray evidence) render his consideration of the opinions of Drs. Fino and Castle infirm. Their opinions explain in detail, with extensive test and medical literature support, why they conclude Addison had idiopathic pulmonary fibrosis instead of coal workers’ pneumoconiosis. A substantial basis for those opinions was the progressive timeline of Addison’s disease, proven by the chronology of CT scans and x-rays, that established idiopathic pulmonary fibrosis. While the ALJ was not required to accept their opinions, he could not have made a reasoned decision evaluating the opinions in view of the foundational errors regarding the medical evidence.

Finally, we note the ALJ ignored the respective qualifications of these physicians in reaching his decision. Dr. Forehand is a board-certified pediatrician and allergist, whereas Drs. Fino and Castle are both board-certified in internal medicine and pulmonary disease. “A primary method of evaluating the reliability of an expert’s opinion is of course his expertise^]” Adkins, 958 F.2d at 52. The ALJ should have given some reasoned explanation as to why Dr. Fino’s and Dr. Castle’s superi- or qualifications did not carry any weight in his evaluation. See Milburn Colliery Co., 138 F.3d at 536.

The ALJ’s finding that Addison suffered from legal pneumoconiosis relied heavily on the weight given to Dr. Forehand’s opinion over that of Drs. Castle and Fino. As explained several ways above, however, the error as to the CT scan evidence fundamentally affected the ALJ’s capacity to reach that conclusion. The ALJ failed to analyze all of the relevant evidence and give a reasoned explanation for how it was weighed. Sterling, 131 F.3d at 439^10 (“Unless the [ALJ] has analyzed all evidence and has sufficiently explained the weight he has given to [the] exhibits, to say that his decision is supported by substantial evidence approaches an abdication of the court’s duty to scrutinize the record as a whole to determine whether the conclusions reached are rational.”). Consequently, we conclude these errors were prejudicial to Sea-B because without them the likelihood that the result would have been different is significant. See Shinseki, 556 U.S. at 410, 129 S.Ct. 1696 (“Often the circumstances of the case will make clear to the appellate judge that the ruling, if erroneous, was harmful and nothing further need be said.”).

V.

For all these reasons, we grant the petition for review, vacate the Board’s decision, and remand with instructions for the Board to return Addison’s case to the ALJ for reconsideration consistent with this opinion.

PETITION FOR REVIEW GRANTED; ORDER VACATED AND REMANDED