v.
Robert E. HAMPTON, Chairman of Civil Service Commission, Et Al.
The named appellants in this class action are eight black college graduates who were recruited by the Chicago Regional Office of the Department of Housing and Urban Development (HUD) for participation in HUD’s Urban Intern Program. The program was designed to bring qualified minority group citizens into responsible positions within the agency. Three appellants [1] were hired as temporary employees, with permanent employment contingent on passing the Federal Service Entrance Examination (FSEE). All three failed the FSEE and were discharged solely because of this failure. The other named appellants [2] qualified for permanent appointments because of either an outstanding scholastic record or prior work experience, but took the FSEE to qualify for higher ratings or entrance into HUD’s Management Intern Program. All five failed to obtain the necessary scores and thus were ineligible for permanent employment in higher positions.
Despite appellants’ lack of success on the FSEE, they apparently performed their jobs in a highly satisfactory manner. A HUD official [3] stated that appellants were “extremely well qualified for Federal employment . . . [and] highly qualified for the positions to which they were assigned.” [4] Appellants “received satisfactory evaluations and commentaries from their supervisors,” he said, “and proved . . . their ability to progress to a higher and more responsible position within [HUD].” [5] The official further stated his opinion, based on personal observation, that some employees who performed poorly on the FSEE were as qualified as other employees who did well on the examination. [6]
On August 25, 1970, appellants filed a complaint with the Chicago Interagency Board of the United States Civil Service Commission charging that the FSEE unlawfully discriminates on the basis of race. [7] Appellants asked that they be given jobs and ratings commensurate with their job performance, and that the FSEE be revised or abandoned. The Board denied this relief by letter of September 8. [8] Appellants then filed a petition for an appeal with the Chicago Regional Office and the Commission’s Bureau of Research and Examination. [9] A letter accompanying the petition sought data disclosing results on the FSEE according to race and any data bearing on Commission attempts to “validate” [10] the FSEE. The appeal was denied on December 10, 1970, and the Commission did not supply any of the requested information. [11]
On February 4, 1971, appellants filed this action in the District Court, alleging that use of the FSEE in the employment and promotion of federal employees violates the Fifth Amendment, [12] the Civil[*980] Rights Act of 1870, [13] and statutes relating to the federal civil service. [14] On August 5, appellants filed a motion for a preliminary injunction restraining the Commission from using the FSEE for purposes of hiring and rating federal employees, prohibiting the Commission from discharging any employee solely because of failure to pass the FSEE, and ordering reinstatement of any employee so terminated. The Commission promulgated new regulations on August 14 governing administrative challenges to employment practices in the federal service, including administration of examinations like the FSEE. [15] Appellees [16] then moved for a remand of the case to the Commission for reconsideration of appellants’ claims under these regulations.
The District Court held that appellants had not demonstrated a sufficient likelihood of success on the merits, because they had failed to make a showing that the FSEE has a racially disproportionate impact. [17] Further, the court found, although it was unnecessary to its decision, that appellees had “demonstrated that the FSEE is a reasonable measure of job performance in those occupations where it is generally used as the entry requirement.” [18] The court also granted appellees’ motion to remand, “[bjecause of the complex factual determinations which will have to be made in this case, the particular expertise and statutory jurisdiction of the Commission as to the issues involved, and in- the interest of judicial economy.” [19]
In this opinion, we discuss the likelihood of proof that the FSEE has a racially disproportionate impact, [20] the proper legal standard for determining the validity of the FSEE, [21] and the propriety of the District Court’s remand to the Commission. [22] In the end, we vacate the denial of the preliminary injunction and affirm the remand order. [23]
I. RACIALLY DISPROPORTIONATE IMPACT
A. The Federal Service Entrance Examination
The FSEE is the “primary avenue of entry” into managerial and professional positions in the federal civil service. [24] The examination was developed by the Commission for this purpose and was first used in 1955. [25] It is administered to approximately 150,000 applicants annually, and the results are used to fill about .10,000 positions in over 200 job categories throughout the Federal Government. [26] The jobs are widely varied, including those of computer specialist, customs inspector, economist, psychologist,[*981] social service representative and many more. [27]
It goes without saying that the Commission is prohibited from discriminating on the basis of race in the hiring or rating of federal employees. The major differences between the parties on this appeal concern interpretation of the standard by which the Commission’s employment practices are to be measured. Numerous cases in the federal courts have involved challenges to standardized aptitude tests on both constitutional [28] and statutory [29] grounds. No court has distinguished the standard mandated by the Fifth and Fourteenth Amendments from that specified by Title VII of the Civil Rights Act of 1964. [30] In 1971 the Supreme Court defined the Title VII standard for private employers in Griggs v. Duke Power Company, [31] and since that decision Congress has extended the reach of Title VII to public employers, including the Federal Government. [32] Congress clearly intended to give public employees the same substantive rights and remedies that had previously been provided for employees in the private sector; [33] beyond that, the applicability of the Griggs standard has also been recognized in numerous cases involving pub-lie employees not grounded on Title VII. [34] So, notwithstanding the several equal protection guarantees implicated in this litigation, the Griggs standard is the measure of the rights and liabilities of the parties. [35]
By that standard, “[i]f an employment practice which operates to exclude Negroes cannot be shown to be related to job performance, the practice is prohibited.” [36] Once it is shown that a practice has a racially disproportionate impact, the employer must meet the heavy burden of proving that the practice “bear[s] a demonstrable relationship to successful performance of the jobs for which it was used.” [37] The task at hand is to determine whether, on the evidentiary showings viewed in light of these principles, the District Court properly denied appellants’ motion for a preliminary injunction and properly granted appellees’ motion for a remand to the Commission.
B. Racial Data Regarding the FSEE
The Commission does not maintain pass-fail data on the FSEE by race. As a result, any evaluation of the racial impact of the FSEE must be based on data[*982] approximating direct evidence of black performance on the FSEE. Appellants have produced two statistical analyses, of information furnished by the Commission, clearly establishing that whites perform much better than blacks.
In discovery proceedings in the District Court, the Commission furnished pass-fail rates at 1149 colleges from October, 1969 to June, 1970. One study [38] designated 50 of these colleges predominantly black and 835 predominantly white, based on enrollments of at least 95% black or white respectively. [39] The total number of people included in this sample appears to be over one-third of the applicants taking the FSEE during that year. [40] The passing rate on black campuses was 12.4% and on white campuses was 60.2%. When the minimum racial percentage was increased, much the same result was found; the passing rate at 99% black colleges was 11.5%, and at 99% white colleges was 57.8%. [41] Because entrants into HUD’s Management Intern positions must score at least 95 on the FSEE, the study also investigated the frequencies of scores at that level for blacks and whites. The percentage scoring 95 or above on the FSEE was 0.6 at the predominantly black schools and 17.4 at the predominantly white colleges; at the 99% colleges, the percentages were 0.7 for blacks and 14.4 for whites. An independent study reached similar results through a different technique. [42] This study paired 50 predominantly black schools with 50 predominantly white schools on the basis of several factors. [43] At the paired schools for which the Commission had provided data, the study found the passing rate to be 8.6% for seniors and graduate students at black schools and 42.1% at the white schools. [44]
Appellees challenge both studies on the ground that the sample was not random because it disregarded colleges for which racial data were unavailable, ignored black students who do not attend predominantly black institutions, and did not make allowances for applicants taking the FSEE more than once. These criticisms, however, do not question the validity of the sample; they merely point out differences between the sample and the actual group of applicants. The sample from which the statistics were derived was random, in the statistical sense, and large enough to be significant. [45]
Data like those presented in these studies have been accepted in at least one other case to support a finding that an aptitude test has a racially dispropor[*983] tionate impact. [46] The court there accepted “the scores reported to be from predominately white institutions as being the scores of white students and the scores reported to be from the predominately black institutions as being scores of black students.” [47] The racial populations of the schools in that case were determined from precisely the same source used in the two studies in this case. [48] Moreover, the disparity indicated for the FSEE is significantly greater than disparities held disproportionate in other cases. [49]
The factual demonstration made by appellants is corroborated by the “substantial body of evidence that black persons and other disadvantaged groups perform on the average far below the norm for whites on generalized intelligence or aptitude tests.” [50] Judicial decisions in the “ever-extending series of challenges to civil service examinations” [51] unequivocally establish that blacks are test-rejected more frequently than whites [52] and that this phenomenon is the result of a long history of educational and cultural deprivation rather than an innate lack of qualifications. [53]
These data have great importance in any determination of the legality of the FSEE. In considering the 1972 amendments to Title VII, [54] Congress recognized that standardized tests in hiring for the federal service may operate to the detriment of disadvantaged minorities:
Civil Service selection and promotion requirements are replete with artificial selection and promotion requirements that place a premium on “paper” credentials which frequently prove of questionable value as a means of predicting actual job performance. The problem is further aggravated by the [Civil Service Commission’s] use of general ability tests which are not aimed at any direct relationship to specific jobs. The inevitable consequence of this, as demonstrated by similar practices in the private sector, and, found unlawful by the Supreme Court, is that classes of persons who are culturally or educationally disadvantaged are subjected to a heavier burden in seeking employment. [55]
The responsibility of the courts to give the data their just due is thus clear.
II. THE VALIDITY OF THE FSEE
In determining the likelihood of success on the issue of validity of the FSEE, the controlling legal principles are those expressed by the Supreme Court in Griggs and applied with remarkable consistency by other federal[*984] courts. [56] The evidence above discussed would impose a heavy burden on appellees to show, at a trial on the merits, that the FSEE bears a demonstrable relationship to successful performance of the jobs for which it is used. [57] Appellees agree that if there is a sufficient showing of racially disproportionate impact, Griggs requires them to prove that the FSEE is a reasonable measure of job performance. [58]
Three techniques for proving the validity of testing procedures have been judicially and professionally [59] noted. “Empirical” validity is demonstrated by identifying criteria that indicate successful job performance and then showing a correlation between test scores and those criteria. [60] “Construct” validity is proven when an examination is structured to determine the degree to which applicants possess identifiable characteristics that have been determined to be important to successful job performance. [61] “Content” validity is established when the content of the test closely approximates the tasks to be performed on the job by the applicant. [62]
When a showing of racially disproportionate impact has been made, the courts have required employers to prove the validity of the challenged employment practice conformably with these accepted standards. [63] As has been well[*985] said, the law “does not permit an employer to engage in unsubstantiated speculation at the expense of Negro workers.” [64] Relying on Griggs, the First Circuit has stated that employers “may not . . . rely on any reasonable version of the facts, but must come forward with convincing facts establishing a fit between the qualification and the job.” [65] The courts that have expressed a view on the relative merits of these techniques have uniformly manifested a preference for proof of empirical validity. [66]
Appellees do not contend that the FSEE has empirical validity. Indeed, two investigations of the empirical validity of the FSEE have been made, and neither would support a claim for empirical validity. In a 1970 study conducted by the Civil Service Commission, an “intensive effort was made to obtain job performance measures on employees in seven of the major occupations filled through the FSEE program and to compare these data with written test scores,” [67] but no meaningful results were achieved. [68] A more recent study by the Department of Labor [69] also failed to demonstrate the empirical validity of the FSEE; indeed, it reached some conclusions that could be regarded as proving that the FSEE is not empirically valid. [70]
Appellees maintain that they have shown the FSEE to be job related through construct validity. [71] The Commission, they say, conducted extensive job analyses and concluded that verbal and quantitative abilities were significantly related to ability to perform the jobs for which the FSEE is used, and the FSEE was then designed to measure these abilities. Appellants argue that empirical validity is required in all cases and that appellees’ assertion of construct validity will not fulfill their heavy burden of proving job relatedness.
Although construct validity is a professionally recognized technique for proving validity, [72] we note that no court has found a demonstration of construct validity sufficient to satisfy Griggs. [73] It[*986] may be that in a proper case a convincing demonstration of construct validity will suffice. It should be noted, however, that construct validity does not conclusively establish that test results are directly related to job performance. It merely means that the test accurately measures certain constructs; in determining whether a showing of construct validity satisfies Griggs, the court must also determine whether the constructs are themselves related to job performance. In light of the strong preference expressed in reported opinions for empirical validity and the greater reliability of empirical validity, we think construct validity may be considered, if at all, only in certain circumstances.
The guidelines promulgated by the Equal Employment Opportunity Commission, [74] which incoporate the professional standards stated by the American Psychological Association, [75] recognize the value of construct validity only when proof of empirical validity is not feasible. [76] These guidelines have been cited with approval by the Supreme Court, [77] followed by all courts dealing with these issues, [78] and recognized as controlling in at least one circuit. [79] We think it unwise to depart from these accepted principles at this stage in the development of the law concerning equal employment opportunity. We hold that construct validity may be considered only after a showing that it is infeasible to undertake proof of empirical validity. [80]
Although the District Court did not need to examine the validity of the FSEE, [81] the court made findings that might support a conclusion that the FSEE has construct validity. [82] The[*987] court made no finding, however, on the feasibility of proving empirical validity. [83] Some statements in the record do concern feasibility, [84] but the record is not sufficiently developed to determine the probable outcome at trial on this issue for the purpose of reviewing the denial of the preliminary injunction. Moreover, the parties seem to regard the question as still open; counsel for appellees informed us at oral argument that feasibility would be an issue in the Commission hearing on remand. [85]
We conclude, therefore, that the denial of the preliminary injunction by the District Court was based on the erroneous premise [86] that proof of construct validity is acceptable without a demonstration that proof of empirical validity is not feasible. Despite appellants’ strong showing that the FSEE has a racially disproportionate impact, the insufficiency of the record on the question of validity leaves us unable to conclude, as we must to reverse a denial of a preliminary injunction, [87] that appellants are likely to succeed on the merits. Moreover, the paucity of the record leaves no satisfactory basis for making other determinations prerequisite to a reversal— irreparable injury to appellants, [88] the balance of harm to appellees against harm to appellants, [89] and the effect on the public interest of a grant or a denial of injunctive relief. [90]
In this kind of situation, we normally set aside the denial of the preliminary injunction [91] and remand the case to the[*988] District Court for reconsideration in light of our opinion. We adhere to this practice and direct the court not to delay fulfillment of this responsibility pending the results of Commission determinations on the remand, discussed below. The necessary initial conclusions need not await the results of a detailed agency inquiry.
III. THE REMAND TO THE COMMISSION
Appellants contend that the District Court erred in remanding the case to the Commission. They argue that they should not be required to exhaust administrative remedies created after the filing of their suit. They further maintain that remand would be futile, because the Commission is a party to the litigation and has prejudged the dispositive issues therein. We do not agree.
We are clearly bound to apply the law as it stands today, rather than the law in effect when appellants filed their complaint. The rule is that a federal appellate court must abide by judicial decisions and statutory enactments subsequent to the judgment of the court under review. [92] And “[sjurely it applies with equal force when the change[s] [are] made by an administrative agency acting pursuant to legislative authorization.” [93] To be sure, “[e]xceptions have been made to prevent manifest injustice, but this is not such a case.” [94]
We do not summon these principles in support of any proposition that the District Court was required to remand the case to the Commission; it had jurisdiction, and the availability of new administrative remedies did not deprive the court of that jurisdiction. [95] Nevertheless, jurisdiction aside, judicial relief should ordinarily be postponed until the litigant seeking it has fully pursued all available administrative remedies. [96] Beyond that, “[j]udicial and administrative agencies ‘are to be deemed collaborative instrumentalities of justice,’ ” [97] and “[c]ourts have frequently called upon administrative bodies for assistance in connection with issues falling within an area of administrative competence.” [98] The District Court stated that it was exercising its discretion in remanding the case in the interest of sound judicial administration, [99] and we cannot say that the decision to do so was erroneous. The Commission has the authority to grant the[*989] relief requested by appellants. [100] The Commission might grant the relief or alter the issues in the case. [101] The case involves complex factual determinations within the expertise of the Commission, [102] and the agency may narrow the issues to be decided by the court. In these and perhaps other ways the Commission may aid the District Court in the decision it must ultimately make. [103] We perceive no sound reason why, in the circumstances of this case, the District Court could not utilize this opportunity for administrative reappraisal of the problem as a forerunner to its judicial resolution.
We are mindful of appellants’ bare allegation that the result of any further administrative proceedings has been predetermined by the Commission. Very recently we were confronted by a similar claim that remand to the Commission would be futile because the Commission was a party and had prejudged the dis-positive issues in the case. We there rejected the claim as unsubstantiated by the facts, [104] and we do the same here. As the District Court pointed out, “[tjhere has . . . been no showing that the Commission will act other than in accordance with law in developing the factual foundation upon which its decision should be based.” [105]
Appellants are also concerned that the standard for establishing test validity set out in the Commission’s guidelines are weaker than those imposed by the EEOC guidelines and the legal precedents. [106] Primarily because the Commission apparently has never interpreted its guidelines in an adjudicative proceeding, we are uncertain as to what standards are incorporated therein. The guidelines require a “rational relationship” between the tests and job performance, [107] but we do not understand this term to carry the same meaning as in equal protection cases that decline to apply a “strict scrutiny” standard. [108] Nor do we think the meaning of the term in the context of test validity is clear and unequivocal. The Commission’s regulations are sufficiently flexible to permit an interpretation that will comport with present statutory and constitutional standards of[*990] equal employment opportunity. [109] Certainly the uncertainty of the standard to be applied will not justify reversing a discretionary decision to seek the aid of agency expertise and remand the case to the Commission.
Accordingly, the denial of the motion for a preliminary injunction is vacated and the judgment granting appellees’ motion for remand is affirmed. The case is remanded for further proceedings consistent with this opinion.
. Jesse Douglas, Everett Rand and Fred Ruck-er. See Joint App. at 133-34, 139^12.
. Brenda Head, Barbara Jones, Lillian Smith, Joyce Wade and Errol Ware. See Joint App. at 135-48, 143 — 46.
. Robert L. Tucker, Assistant Regional Administrator for Equal Opportunity in the Chicago Regional Office.
. Joint App. at 126.
. Id.
. Id.
. This administrative remedy is provided in the Federal Personnel Manual, ch. 337, § 2-7, at 337-9.
. Joint App. at 294-97.
. This procedure is also provided in the Federal Personnel Manual. See note 7, supra.
. See notes 54r-60, infra. This letter also asked the Commission to advise appellants of any further available administrative remedy. Joint App. at 298-300. The denial of the appeal petition did not specify any further remedy.
. Joint App. at 309.
. U.S.Const. Amend. V. Appellants claimed denials of equal protection of the laws, a guarantee applicable to actions by the Federal Government through the Due Process Clause of the Fifth Amendment. Bolling v. Sharpe,[*980] 347 U.S. 497, 74 S.Ct. 693, 98 L.Ed. 884 (1957); Bolton v. Harris, 130 U.S.App.D.C. 1, 4 n. 3, 395 F.2d 642, 645 n. 3 (1968).
. Act of May 31, 1870, ch. 114, § 16, 16 Stat. 144, 42 U.S.C. § 1981 (1970).
. 5 U.S.C. §§ 3304, 7751, 7754 (1970).
. 5 C.F.R. §§ 300.101 et seq. (1974).
. Appellees are the Commissioners of the Civil Service Commission, the Director of the Commission’s Bureau of Recruiting and Examining and the Secretary of HUD.
. Douglas v. Hampton, 338 F.Supp. 18, 22 (D.D.C.1972). The finding of the court was that appellants had not shown that blacks are “disproportionately disadvantaged.” We prefer, however, to use the phrase in the text. See Davis v. Washington, 168 U.S. App.D.C. —- -n. 6. 512 F.2d 956, 958 n. 6 (D.C.Cir.), decided today.
. Douglas v. Hampton, supra note 17, 338 F.Supp. at 22.
. See Part I, infra.
. See Part II, infra.
. See Part III, infra.
. See Parts II, III, infra.
. Federal Service Entrance Examination, Civil Serv. Comm’n Announcement No. 410, at 2 (1970).
. Until 1971, the Commission accepted scores on the Graduate Record Examination (GRE), a general aptitude test not designed for use in nonacademic settings, as a substitute for the FSEE. The Educational Testing Service, which administers the GRE, has refused to provide GRE scores to anyone other than accredited higher education institutions, because the scores had been improperly used by employers. See Armstead v. Starkeville Mun. Separate School Dist., 325 F.Supp. 560 (D.Miss.1971), aff’d in part and rev’d in part, 461 F.2d 276 (5th Cir. 1972).
. Douglas v. Hampton, supra note 17, 338 F.Supp. at 20.
. Federal Service Entrance Examination, Civil Serv. Comm’n Announcement No. 410, at 3-5 (1970).
. E. g., Davis v. Washington, supra note 17; Bridgeport Guardians, Inc. v. Civil Serv. Comm’n, 482 F.2d 1333 (2d Cir. 1973); Allen v. Mobile, 466 F.2d 122 (5th Cir. 1972), cert. denied, 412 U.S. 909, 93 S.Ct. 2292, 36 L.Ed.2d 975 (1973); Carter v. Gallagher, 452 F.2d 315 (8th Cir. 1971), cert. denied, 406 U.S. 950, 92 S.Ct. 2045, 32 L.Ed.2d 338 (1972); Fowler v. Schwarzwalder, 351 F.Supp. 721 (D.Minn.1972).
. E. g., Griggs v. Duke Power Co., 401 U.S. 424, 91 S.Ct. 849, 28 L.Ed.2d 158 (1971); United States v. Jacksonville Terminal Co., 451 F.2d 418 (5th Cir. 1971), cert. denied, 406 U.S. 906, 92 S.Ct. 1607, 31 L.Ed.2d 815 (1972).
. 42 U.S.C. § 2000e et seq. (1970).
. Supra note 29.
. Equal Employment Opportunity Act of 1972, Pub.L. No. 92-261, § 717, 86 Stat. 211, 42 U.S.C. § 2000e — 16 (Supp. II 1972).
. See S.Rep.No.92-A15, 92d Cong., 1st Sess. 16 (1972); Womack v. Lynn, 165 U.S.App. D.C. -, 504 F.2d 267 (1974); Henderson v. Defense Contract Admin. Serv., 370 F.Supp. 180, 183 (S.D.N.Y.1973); Hackley v. Johnson, 360 F.Supp. 1247, 1251 (D.D.C.1973); Thompson v. United States Dep’t of Justice, 360 F.Supp. 255, 257 (N.D.Cal.1973).
. See cases cited note 28, supra.
. Davis v. Washington, supra note 17, 168 U.S.App.D.C. at-n. 2, 512 F.2d at 957 n. 2.
. Griggs v. Duke Power Co., supra note 29, 401 U.S. at 431, 91 S.Ct. at 853, 28 L.Ed.2d at 164.
. Id. See McDonnell Douglas Corp. v. Green, 411 U.S. 792, 802 & n. 14, 93 S.Ct. 1817, 1824 & n. 14, 36 L.Ed.2d 668, 678 & n. 14, (1973); Davis v. Washington, supra note 17, 168 U.S.App.D.C. at-n. 15, 512 F.2d at 959 n. 15; Bridgeport Guardians, Inc. v. Civil Serv. Comm’n, supra note 28, 482 F.2d at 1337; Officers for Justice v. Civil Serv. Comm’n, 371 F.Supp. 1328, 1336 (N.D.Cal. 1973); Smith v. East Cleveland, 363 F.Supp. 1131, 1147 (N.D.Ohio 1973).
. The study was conducted by Dr. Kurt Bayer of the University of Maryland.
. The enrollment data were obtained from a document published by the Office of Civil Rights of the United States Department of Health, Education and Welfare.
. We base this estimate on the fact that 150,000 people take the FSEE each year and the inclusion of over 50,000 students in the sample.
. la addition, Dr. Bayer found a correlation between black enrollment and the FSEE passing rate of -0.34. Thus, as black enrollment increased, the passing rate declined significantly. See Boston Chapter, NAACP, Inc. v. Beecher, 504 F.2d 1017, 1024 n. 13 (1st Cir. 1974).
. This study was conducted by the Urban Institute, Washington, D. C.
. The factors were location, size of enrollment, number of professors with advanced degrees, endowment per student and type of school.
. Joint App. at 340.
. Boston Chapter, NAACP, Inc. v. Beecher, supra note 41, 504 F.2d at 1021 n. 6. The balance of appellees’ criticisms similarly fail to raise any doubt about the demonstration of a racially disproportionate impact. For example, they maintain that the studies used data in which the race of the applicants cannot be determined. Even if we make the questionable assumption that, at the 99% black colleges in the Bayer study, 10% of those taking the FSEE were white and all of these whites failed, the passing rate for blacks at these schools is increased from 11.5% only to 12.8%. The questions posed by appellees concerning appellants’ statistical evidence in some ways merely explain the disparate performances of blacks and whites, rather than contradict the proof. See Kirkland v. State Dep’t of Correctional Serv., 374 F.Supp. 1361, 1366 (S.D.N.Y.1974).
. Armstead v. Starkeville Mun. Separate School Dist., supra note 25, 325 F.Supp. at 567.
. Compare Castro v. Beecher, 459 F.2d 725, 729 (1st Cir. 1972) (passing rate 25% for blacks and 65% for whites); Bridgeport Guardians, Inc. v. Civil Serv. Comm’n, supra note 28, 482 F.2d at 1335 (passing rate for whites 3‘/2 times the black rate); Chance v. Board of Examiners, 458 F.2d 1167, 1171 (2d Cir. 1972) (passing rate for whites l'A times greater than for blacks); United States v. Chicago, 385 F.Supp. 543, 550 (N.D.Ill.1974) (failure rate for blacks twice the rate for whites); Pennsylvania v. O’Neill, 348 F.Supp. 1084, 1089-90 (E.D.Pa.1972), aff’d in part and vacated in part, 473 F.2d 1029 (3d Cir. 1973) (1.82 times as many whites passed as blacks).
. Arrington v. Massachusetts Bay Transp. Auth., 306 F.Supp. 1355, 1358 (D.Mass.1969).
. Kirkland v. State Dep’t of Correctional Serv., supra note 45, 374 F.Supp. at 1364.
. In every one of the more than twenty reported opinions involving standardized aptitude tests, cited herein and in Davis v. Washington, supra note 17, the final disposition has included a conclusion of racially disproportionate impact.
. Griggs v. Duke Power Co., supra note 29, 401 U.S. at 430, 91 S.Ct. at 853, 28 L.Ed.2d at 163-164. Gaston County v. United States, 395 U.S. 285, 89 S.Ct. 1720, 23 L.Ed.2d 309 (1969); Hobson v. Hansen, 269 F.Supp. 401, 419-21 (D.D.C.1967), aff’d sub nom., Smuck v. Hobson, 132 U.S.App.D.C. 372, 408 F.2d 175 (en banc 1969).
. See note 32, supra.
. H.R.Rep.No.238, 92d Cong., 1st Sess. 24 (1971), 1972 U.S.Code Cong. & Admin.News, p. 2159.
. See text supra at notes 28-35.
. See note 37, supra, and accompanying text. It is possible to read the civil service laws as imposing this burden on appellees even apart from the racially disproportionate impact of the FSEE. See 5 U.S.C. § 3304(a) (1970). We do not pass on that question, never squarely addressed by any court, because of the strong tendency of appellants’ evidence to prove the racially disproportionate impact of the FSEE. Cf. Note, Legal Implications of the Use of Standardized Ability Tests in Employment and Education, 68 Colum.L.Rev. 691, 729 (1968). In any event, § 3304(a) could not authorize imposition on the Civil Service Commission of a standard less stringent than that mandated by Griggs. See text supra at notes 28-34.
. Appellees’ Brief at 13-18. A similar concession was made at oral argument.
. The universally recognized professional authority is American Psychological Association, Standards for Educational and Psychological Tests and Manuals (1966) (hereinafter cited as APA Standards).
. See Vulcan Soc’y v. Civil Serv. Comm’n, 360 F.Supp. 1265, 1273 (S.D.N.Y.), aff’d in part and remanded in part, 490 F.2d 387 (1973); Kirkland v. State Dep’t of Correctional Serv., supra note 45, 374 F.Supp. at 1370. Some courts have referred to this technique as “criterion” or “criterion-related” validity. See APA Standards, supra note 59, at 13; Kirkland v. State Dep’t of Correctional Serv., supra note 45, 374 F.Supp. at 1370-71; Smith v. East Cleveland, supra note 37, 363 F.Supp. at 1148; Davis v. Washington, 352 F.Supp. 187, 191 (D.D.C.1972). Other courts have also referred to it as “predictive validity.” Chance v. Board of Examiners, supra note 49, 458 F.2d at 1174; Bridgeport Guardians, Inc. v. Civil Serv. Comm’n, supra note 28, 482 F.2d at 1337; Walston v. County School Bd., 492 F.2d 919, 926 (4th Cir. 1974); Allen v. Mobile, supra note 28, 466 F.2d at 127 (Goldberg, J., dissenting); Officers for Justice v. Civil Serv. Comm’n, supra note 37, 371 F.Supp. at 1336.
. See APA Standards, supra note 59, at 13; Bridgeport Guardians, Inc. v. Civil Serv. Comm’n, supra note 28, 482 F.2d at 1337-38; Kirkland v. State Dep’t of Correctional Serv., supra note 45, 374 F.Supp. at 1370; Smith v. East Cleveland, supra note 37, 363 F.Supp. at 1148; Davis v. Washington, supra note 60, 352 F.Supp. at 191.
. See APA Standards, supra note 59, at 12-13; Vulcan Soc’y v. Civil Serv. Comm’n, supra note 60, 360 F.Supp. at 1274; Bridgeport Guardians, Inc. v. Civil Serv. Comm’n, supra note 28, 482 F.2d at 1337-38; Kirkland v. State Dep’t of Correctional Serv., supra note 45, 374 F.Supp. at 1372; Walston v. County School Bd., supra note 60, 492 F.2d at 926; United States v. Chicago, supra note 49, 385 F.Supp. at 559; Davis v. Washington, supra note 60, 352 F.Supp. at 191; Fowler v. Schwarzwalder, supra note 28, 351 F.Supp. at 725. The concepts of content validity and construct validity have often been confused; a number of courts have apparently referred to construct validity as content validity. Vulcan Soc’y y. Civil Serv. Comm’n, supra note 60, 490 F.2d at 395; Chance v. Board of Examiners, supra note 49, 458 F.2d at 1174; Officers for Justice v. Civil Serv. Comm’n, supra note 37, 371 F.Supp. at 1336-37.
. E. g„ Vulcan Soc’y v. Civil Serv. Comm’n, supra note 60, 490 F.2d at 395; Bridgeport Guardians, Inc. v. Civil Serv. Comm’n, supra note 28, 482 F.2d at 1337; Officers for Justice v. Civil Serv. Comm’n, supra note 37, 371[*985] F.Supp. at 1337; Fowler v. Schwarzwalder, supra note 28, 351 F.Supp. at 725.
. Hicks v. Crown Zellerbach Corp., 319 F.Supp. 314, 319 (E.D.La.1970).
. Castro v. Beecher, supra note 49, 459 F.2d at 732.
. Bridgeport Guardians, Inc. v. Civil Serv. Comm’n supra note 28, 482 F.2d at 1337; Kirkland v. State Dep’t of Correctional Serv., supra note 45, 374 F.Supp. at 1370; Officers for Justice v. Civil Serv. Comm’n, supra note 37, 371 F.Supp. at 1337; Smith v. East Cleveland, supra note 37, 363 F.Supp. at 1148; Vulcan Soc’y v. Civil Serv. Comm’n, supra note 60, 360 F.Supp. at 1273; Fowler v. Schwarzwalder, supra note 28, 351 F.Supp. at 725.
. Joint App. at 437.
. Id. Scores in the seven job categories were compared with four measures of job performance. Of the 28 resulting correlations, only one was positive and over 0.3, and only seven were over 0.2. Such data, in this context, cannot be regarded as significant. See Davis v. Washington, supra note 17, 168 U.S.App.D.C. at-n. 38, 512 F.2d at 962 n. 38; Boston Chapter, NAACP, Inc. v. Beecher, supra note 41, 504 F.2d at 1024 n. 13.
. Ellison, Fox & Taylor, An Examination of the Federal Service Entrance Examination, Mimeograph, Jan. 31, 1972.
. “These essentially zero correlations indicate that the FSEE had no practical or statistically significant relationship with performance . . . The scores which were used as part of the selection process to identify those individuals who could more adequately learn to effectively perform their jobs were not promoted any more rapidly than those who had obtained somewhat lower scores.” Id. at 6, 9.
. They make no claim of content validity. As a matter of logic, it is impossible to show that a test used in filling over 200 diverse jobs has content validity. See text supra at note 62. It is likely that such a showing could be made only in regard to simple, narrowly-defined jobs; perhaps the best example is stenographic jobs. Cf. Developments in the Law — Employment Discrimination and Title VII of the Civil Rights Act of 1964, 84 Harv.L.Rev. 1109, 1128 n. 90 (1971).
. APA Standards, supra note 59, at 13.
. See Vulcan Soc’y v. Civil Serv. Comm’n, supra note 60, 490 F.2d at 395; Bridgeport Guardians, Inc. v. Civil Serv. Comm’n, supra [*986] note 28, 482 F.2d at 1337-38; United States v. Chicago, supra note 49, 385 F.Supp. at 555; Kirkland v. State Dept. of Correctional Serv., supra note 45, 374 F.Supp. at 1370; Smith v. East Cleveland, supra note 37, 363 F.Supp. at 1148. See also Chance v. Board of Examiners, supra note 49, 458 F.2d at 1174; Officers for Justice v. Civil Serv. Comm’n, supra note 37, 371 F.Supp. at 1336-37. Compare Allen v. Mobile, supra note 28, dealing with an achievement test for police officers.
. 29 C.F.R. §§ 1607.1 et seq. (1973). These guidelines were substantially adopted by the Office of Federal Contract Compliance. 41 C.F.R. §§ 60-3.1 et seq. (1974).
. APA Standards, supra note 59.
. 29 C.F.R. § 1607.5(a). (1973).
. Griggs v. Duke Power Co., supra note 29, 401 U.S. at 433-34, 91 S.Ct. at 853, 28 L.Ed.2d at 165-166.
. E. g., Davis v. Washington, supra note 17, 168 U.S.App.D.C. at-n. 37, 512 F.2d at 962 n. 37; Castro v. Beecher, supra note 39, 459 F.2d at 729; Vulcan Soc’y v. Civil Serv. Comm’n, supra note 60, 490 F.2d at 394 n. 8; Moody v. Albemarle Paper Co., 474 F.2d 134, 139 (4th Cir. 1973), cert. granted, 419 U.S. 1068, 95 S.Ct. 654,-42 L.Ed.2d 664 (1974); Carter v. Gallagher, supra note 28, 452 F.2d at 327; Pennsylvania v. O’Neill, supra note 49, 348 F.Supp. at 1103.
. United States v. Georgia Power Co., 474 F.2d 906, 913 (5th Cir. 1973). Cf. Officers for Justice v. Civil Serv. Comm’n, supra note 37, 371 F.Supp. at 1337.
. See 29 C.F.R. § 1607.5(a). It may not be feasible to establish empirical validity at the time a test is initiated, but in that event at that stage the test can only be regarded as experimental. APA Standards, supra note 59, at 14; 29 C.F.R. § 1607.9 (1973). After some period of time empirical studies should be required unless the employer can establish the infeasibility of such studies. The FSEE has been used since 1955, and we have no doubt that after nearly 20 years the best evidence of job relatedness is empirical validity. In these circumstances, more than intuition of job relatedness — no matter how scientific or carefully arrived at it may be thought to be— should be required if possible. Since we are not presented with the questions, we express no opinion as to the legality of using a test without first establishing empirical validity, nor do we pass on appellees’ claims of construct validity if they can prove infeasibility. We hold only that absent proof that empirical validity is not feasible, other techniques for establishing validity are not acceptable.
. See text supra at notes 17-18.
. “[Appellees] have demonstrated that the FSEE measures verbal and quantitative abilities and that those abilities are necessary for learning the duties and successfully reaching full performance levels within those occupational fields for which it is used.” Douglas v. Hampton, supra note 17, 338 F.Supp. at 21. It is unclear whether the court had an adequate opportunity to consider issues such as the nature of the job analysis, see 29 C.F.R. § 1607.5(a) (1973), and the relationship of the constructs to job performance, see text supra following note 73. In light of our disposition[*987] of this appeal, it is unnecessary for us to pass on these questions.
. Appellees do assert that proof of empirical validity is not feasible, because of the undue burden of performing a study for each FSEE position and because of problems such as the narrow range of test scores for various FSEE positions. Appellees’ Brief at 28 n. 27. These claims, even if true, do not conclusively establish that proof of empirical validity is not feasible. See 29 C.F.R. § 1607.4(c)(2) (1973).
. Joint App. at 350 (supplemental affidavit of Dr. Philip Ash); Joint App. at 436-37 (affidavit of Dr. Albert Maslow).
. See text supra at note 19; Part III, infra.
. See Delaware & Hudson Ry. v. United Transp. Union, 146 U.S.App.D.C. 142, 158, 450 F.2d 603, 619, cer. denied, 403 U.S. 911, 91 S.Ct. 2209, 29 L.Ed.2d 689 (1971); A Quaker Action Group v. Hickel, 137 U.S.App.D.C. 176, 180, 421 F.2d 1111, 1115 (1969); District 50, UMW v. International Union, UMW, 134 U.S.App.D.C. 34, 35, 412 F.2d 165, 166 (1969); Udall v. D.C. Transit System, Inc., 131 U.S.App.D.C. 381, 384, 404 F.2d 1358, 1361 (1968).
. Delaware & Hudson Ry. v. United Transp. Union, supra note 86, 146 U.S.App.D.C. at 158, 450 F.2d at 619; A Quaker Action Group v. Hickel, supra note 86, 137 U.S.App.D.C. at 181, 421 F.2d at 1116; Industrial Bank v. Tobriner, 132 U.S.App.D.C. 51, 54, 405 F.2d 1321, 1324 (1968).
. One court has held that a showing of racially disproportionate impact, not met with convincing proof of validity, is sufficient to support a finding of irreparable injury. United States v. Chicago, supra note 49, 385 F.Supp. at 547.
. Delaware & Hudson Ry. v. United Transp. Union, supra note 86, 146 U.S.App.D.C. at 158, 450 F.2d at 619; A Quaker Action Group v. Hickel, supra note 86, 137 U.S.App.D.C. at 181, 421 F.2d at 1116; Dorfmann v. Boozer, 134 U.S.App.D.C. 272, 277, 414 F.2d 1168, 1173 (1969); District 50, UMW v. International Union, UMW, supra note 86, 134 U.S.App. D.C. at 32, 412 F.2d at 168.
. Yakus v. United States, 321 U.S. 414, 440-41, 64 S.Ct. 660, 88 L.Ed. 834 (1944); Delaware & Hudson Ry. v. United Transp. Union, supra note 86, 146 U.S.App.D.C. at 158, 450 F.2d at 619; Businessmen Affected by the Second Year Action Plan v. Redevelopment Land Agency, 143 U.S.App.D.C. 161, 162, 442 F.2d 883, 884 (1971); A Quaker Action Group v. Hickel, supra note 86, 137 U.S.App.D.C. at 181, 421 F.2d at 1116. Appellees argue that an injunction restraining the use of the FSEE would not serve the public interest because it would paralyze hiring for the federal service. Although this contention has some logical appeal, it is seriously questioned by the facts of the case at bar. Appellants were hired and began working for HUD before taking the FSEE, and they were performing their jobs well. See text supra at notes 1-6.
. It is arguable that we should affirm the denial of the preliminary injunction because appellants have failed to make the necessary proof. The burden of the issue of validity, however, rests with appellees. See Johnson v. Goodyear Tire & Rubber Co., 491 F.2d 1364, 1373 (5th Cir. 1974). Moreover, the inadequate development of other issues, see text, supra, at notes 88-90, is likely due to[*988] the court’s determination that appellants had not shown a substantial likelihood of success on the merits.
. Thorpe v. Housing Authority, 393 U.S. 268, 281, 89 S.Ct. 518, 21 L.Ed.2d 474 (1969); Wilson v. Shultz, 155 U.S.App.D.C. 4, 475 F.2d 997 (1973); De Rodulfa v. United States, 149 U.S.App.D.C. 154, 164, 461 F.2d 1240, 1250, cert. denied, 409 U.S. 949, 93 S.Ct. 270, 34 L.Ed.2d 220 (1972). This principle was clearly enunciated by Chief Justice Marshall over 170 years ago. United States v. Schooner Peggy, 5 U.S. (1 Cranch) 103, 110, 2 L.Ed. 49 (1801).
. Thorpe v. Housing Authority, supra note 92, 393 U.S. at 282, 89 S.Ct. at 526, 21 L.Ed.2d at 484.
. E. g., McKart v. United States, 395 U.S. 185, 193-95, 89 S.Ct. 1657, 23 L.Ed.2d 194 (1969).
. FCC v. Schreiber, 381 U.S. 279, 296-97, 85 S.Ct. 1459, 14 L.Ed.2d 383 (1965); Myers v. Bethlehem Shipbuilding Corp., 303 U.S. 41, 50-51, 58 S.Ct. 459, 82 L.Ed. 638 (1938); Lodge 1858, AFGE v. Paine, 141 U.S.App.D.C. 152, 166, 436 F.2d 882, 896 (1972). This general rule has been applied to claims of racial discrimination. Freeman v. Shultz, 152 U.S.App.D.C. 16, 468 F.2d 120 (1972); Hadnott v. Laird, 149 U.S.App.D.C. 358, 463 F.2d 304 (1972).
. Bethlehem Steel Corp. v. Grace Line, Inc., 135 U.S.App.D.C. 81, 93, 416 F.2d 1096, 1108 (1969), quoting United States v. Morgan, 313 U.S. 409, 422, 61 S.Ct. 999, 85 L.Ed. 1429 (1941), and cases cited. See also Democratic Cent. Comm. v. Washington Metropolitan Area Transit Comm’n, 158 U.S.App.D.C. 7, 48, 485 F.2d 786, 817 (1973).
. Bethlehem Steel Corp. v. Grace Line, Inc., supra note 97, 135 U.S.App.D.C. at 93, 416 F.2d at 1108. See also Democratic Cent. Comm. v. Washington Metropolitan Area Transit Comm’n, supra note 97, 158 U.S.App.D.C. at 48, 485 F.2d at 817.
. Douglas v. Hampton, supra note 17, 338 F.Supp. at 23.
. No question is raised regarding the jurisdiction of the Commission over claims of discrimination against the Federal Government. The Commission’s authority in this case is clearly established by 5 U.S.C. §§ 3301, 3304 and Exec. Order No. 10577, 3 C.F.R. 218 (1954-58 Comp.). See League of United Latin American Citizens v. Hampton, 163 U.S.App.D.C. 283, 287 n. 1, 501 F.2d 843, 847 n. 1 (1974); Douglas v. Hampton, supra note 17, 338 F.Supp. at 23.
. Sohm v. Fowler, 124 U.S.App.D.C. 382, 385, 365 F.2d 915, 918 (1966).
. Appellants are fearful that the Commission will adjudge their case by an erroneous legal standard. They point out that the Commission has no expertise in the area of determining legal standards. We must assume that the Commission will act in accordance with the appropriate legal precedents as enunciated in this opinion. See text infra at note 105. We must also assume that, as the District Court apparently felt, the Commission’s expertise in the area of factfinding will be of great assistance to the court in ultimately resolving the issues in this case.
. See League of United Latin American Citizens v. Hampton, supra note 100, 501 F.2d at 847; Sohm v. Fowler, supra note 101, 124 U.S.App.D.C. at 385, 365 F.2d at 918.
. League of United Latin American Citizens v. Hampton, supra note 100, 501 F.2d at 846. Appellants cite Glover v. St. Louis-San Francisco Ry., 393 U.S. 324, 89 S.Ct. 548, 21 L.Ed.2d 519 (1969), but we would distinguish that case from the case at bar for the reasons stated in League of United Latin American Citizens v. Hampton, supra note 100, 501 F.2d at 845-46.
. Douglas v. Hampton, supra note 17, 338 F.Supp. at 23.
. There is some support for appellants’ view in the amicus brief filed by the EEOC and the letters from the Department of Justice and the EEOC to the Civil Service Commission. See note 109, infra.
. 5 C.F.R. § 300.103(b)(1) (1974).
. See Chance v. Board of Examiners, supra note 49, 458 F.2d at 1177-78. Application of the constitutional rational relationship standard to these cases has been rejected by several courts. E. g., Castro v. Beecher, supra note 49, 459 F.2d at 733; Bridgeport Guardians, Inc. v. Civil Serv. Comm’n, supra note 28, 482 F.2d at 1337; Allen v. Mobile, supra note 28, 466 F.2d at 126 (Goldberg, J., dissenting); Officers for Justice v. Civil Serv. Comm’n, supra note 37, 371 F.Supp. at 1335.
. While the Department of Justice and the EEOC expressed concern that the Commission’s regulations could be read to impose a standard that does not comport with Griggs, each agency based that view on the ambiguity of the “rational relationship” test stated in the regulations.
The term ‘rational relationship’ is unnecessarily vague and is, therefore, subject to a multitude of diverse interpretations, many of which we probably could not accept under administrative and judicial interpretations of Title VII.
Letter from William H. Brown, III, Chairman, EEOC, to Robert E. Hampton, Chairman, Civil Service Commission, undated. Joint App. at 421.
[W]e cannot determine whether the standards to be applied by the Commission in establishing the validity of Federal employment selection tests and criteria are in conformity with the holding of the Supreme Court in the Griggs case.
However, we are concerned that in such guidelines the term ‘rational relationship’ as used in Section 300.103(b), and that portion of Section 300.103(c) which provides that the nondiscriminatory requirement of the regulations ‘is generally met when an employment practice is relevant,’ be interpreted to require a degree and type of validity comparable to that required by the EEOC Guidelines.
Letter from David L. Norman, Acting Assistant Attorney General, Civil Rights Division, Department of Justice, to Robert E. Hampton, Chairman, Civil Service Commission, August 5, 1971. Joint App. at 424-25.