FACT SHEET

"Above the Line, Below the Line":
A Selective Look at the State-Constructed
Dental Licensing Examination

  1. Highlights

    • Test Scoring; Appropriateness. A (compensatory) test scoring system that: (1) makes it mathematically impossible to score greater than 2.000 but less than 3.000 on a one-part test or any part of a two-part test; and (2) lowers marginally unsatisfactory scores and raises marginally satisfactory scores; does not seem an appropriate tool for measuring minimal competency. This increases inter-examiner reliability at the expense of a candidate.

    • Test Scoring; Fairness. The suppression of a critical difference grade to increase inter-examiner reliability does not seem fair because such a test scoring system helps some candidates and hurts others in borderline cases.

    • Test Grading; Appropriateness. If there is a root cause to the low pass rate on the state-constructed dental licensing examination that does not involve candidate or examiner behavior, it may be the practice of requiring the lowest rating earned by a candidate to control the grade awarded by an examiner.

    • The Bureau's Observations.

      1. Candidate examination materials. The examination materials provided to each candidate do not disclose all the criteria that are used by an examiner. Giving candidates the same information as the examiners might largely eliminate the claim that the candidate failed the dental licensing examination because the candidate did not know what the examiner was looking for.

      2. Examiner standardization session. According to the calibrator for the August 1999 dental licensing examination, the calibration examination was re-scored because not enough potential examiners had initially passed the calibration examination and because chapter 16-79, Hawaii Administrative Rules, does not contain specific procedures for handling situations where there are not enough qualified examiners to administer a dental licensing examination. According to the calibrator, if the Board had accepted the impartial observer's 1996 recommendation to change the critical difference grade cut-off from thirteen percent to eighteen percent, there initially would have been enough qualified examiners to administer the August 1999 dental licensing examination.

      3. Anonymity between grading examiners and candidates. The procedures for ensuring anonymity between the grading examiners and the candidates seemed to be well thought out and executed by the Board, the Department, the examiners, and the many volunteer and paid personnel involved with the administration of the examinations.

      4. Assignment of examiners to grading teams. In February 1997, the Examination Branch began randomly assigning examiners to grading teams in order to guard against the possible introduction of bias-or systematic error-caused by examiners assigning themselves to grading teams. This kind of error would have been difficult to detect in the post-examination analysis conducted by the Examination Branch.

      5. Anonymity of foreign dental school graduates. Because the restorative technique examination is conducted on ceramic teeth mounted in a mannequin, grading examiners almost certainly know whether a candidate is a United States or foreign dental school graduate. The Bureau, however, did not observe any discussions between examiners regarding the quality of work performed by foreign dental school graduates.

      6. Status of the impartial observer's recommendations. Although the Board voted to (1) explore the addition of a nonpatient examination of periodontal diagnosis and treatment planning to the dental licensing examination, and (2) eliminate the class II amalgam preparation test on the mannequin, neither of these decisions has been incorporated into the Board's rules (Chapter 16-79, Hawaii Administrative Rules), to date.

      7. Use of errata sheet to purportedly amend rules. Several errors in the grading (rating) criteria for the dental licensing examination have not been corrected through the Administrative Procedure Act. Instead, corrections are listed on a separate errata sheet, which is included in the candidate handbooks. As the so-called "corrections" have never been adopted through the rulemaking process, it would appear that the Board cannot validly apply them.

      8. Substitution of grades if anonymity has been breached. The data seem to suggest that substituting (a) the grades obtained by a candidate for the preparation part of the class II amalgam test on the mannequin for (b) the grades obtained by the candidate for the preparation part of the class II amalgam test on the patient, may not be appropriate.

    • United States dental school graduates pass the dental licensing examination at a higher rate than foreign dental school graduates-53.66 percent versus 41.30 percent. The pass rate on the restorative technique examination for this cohort of foreign dental school graduates was 26.73 percent. If the pass rate of foreign dental school graduates on the dental licensing examination is made a function of their pass rate on the restorative technique examination, then the pass rate for foreign dental school graduates on the dental licensing examination was 18.81 percent instead of 41.30 percent.

      If the dental licensing examination was scored using a conjunctive model rather than a compensatory model, then the pass rate for United States dental school graduates would be less than or equal to 25.56 percent instead of 53.66 percent. A conjunctive scoring model places substantial demands on foreign dental school graduates taking the restorative technique examination. Conversely, a compensatory scoring model gives substantial allowances to United States dental school graduates and foreign dental school graduates taking the dental licensing examination.

      At the legislative level, the ultimate policy issue that the Legislature must decide is whether to: (1) retain the state-constructed dental examination (the status quo), (2) abolish the state-constructed examination and allow candidates to pass any one of the four regional examinations, or (3) allow candidates to pass either the state-constructed examination or a regional examination. This legislative policy decision is separate and distinct from anyone's opinion on the technical merits or defects of the existing state-constructed dental licensing examination.

      The question of whether to authorize the Board of Dental Examiners to accept the results of all four regional testing agencies' examinations does not depend on a determination that the existing state-constructed dental licensing examination is an undue exclusionary barrier to entry into the profession. Instead, this question depends on whether the Legislature's desire for local control of the dental licensing examination, if any, outweighs its concerns about state liability for alleged or proven injustices, convenience and affordability for candidates, and ongoing program costs to the State.

  2. Frequently Asked Questions

    • What was the Pekarsky litigation about? Why does it figure so prominently in past and present discussions about the dental licensing examination?

      This federal class action, civil rights lawsuit was brought in 1976 against the Board of Dental Examiners, and contended that the Board had utilized the dental licensing examination to illegally discriminate against candidates for a dental license based on their racial (Caucasian) and nonresident status.

      The Board denied Pekarsky's and Koch's allegations of wrongdoing but agreed to settle the lawsuit in 1980. The settlement agreement included the appointment of a court master in 1987-one Dr. A. Lewis Leo, Associate Dean of the University of Florida College of Dentistry and a national dental expert who had been involved in prior phases of the litigation-to modify the dental licensing examination, including the rules relating to its administration.

      In 1991, the United States District Court modified the settlement agreement to eliminate the requirement that the court continue to supervise and monitor the dental licensing examination, and discharged Dr. Leo as the court's master. Paragraph 3 of the settlement agreement was modified to read as follows:

      3. The requirements and procedures by which candidates for dental licensure are examined and licensed in the State of Hawaii ["licensure requirements"] [sic] shall at all times now and in the future conform to the requirements of the Constitution and laws of the United States and the State of Hawaii and shall (a) be based only upon the present state of dental practice and current trends in dental education; (b) utilize only valid and reliable test and measurement procedures; (c) employ only comprehensive objective criteria for measuring the competence of candidates; and, (d) assure that the Hawaii dental licensure examination is anonymous in all respects. . . . [emphasis added]

      The United States District Court did not retain continuing jurisdiction to enforce or amend the settlement agreement after 1991.

    • Why didn't the Bureau draw any definitive conclusions about the appropriateness of the dental licensing examination and its effect on entry into the profession?

      This report reviews selected issues concerning the dental licensing examination. Even with the assistance of the Board of Dental Examiners and the Department of Commerce and Consumer Affairs, the Bureau does not possess adequate expertise in the fields of dentistry or psychometry, much less both, to conduct an expert level of review of this or any other professional licensing examination.

      The Bureau cannot conclude that the dental licensing examination, as a technical matter, is either an appropriate test for professional competency or an undue exclusionary barrier to entry into the profession. Some of the Bureau's observations, as noted above, however, indicate that the examination process is less than perfect.

      On the other hand, a court-appointed master and an impartial observer hired as a consultant have reviewed the dental licensing examination very favorably. The Bureau has no basis to doubt the findings of the impartial observer, and finds no compelling evidence to conclude that the examination is defective in any significant way.

    Back