The opinion of the court was delivered by: O'hern, J.
On review of the Report of Special Master.
When the United States Supreme Court restored the constitutionality of the death penalty, it imposed a concomitant obligation on states to provide "the further safeguard of meaningful appellate review" of every death sentence. Gregg v. Georgia, 428 U.S. 153, 195, 96 S. Ct. 2909, 2935, 49 L. Ed. 2d 859, 887 (1976). This matter arises out of our exercise of that function and concerns specifically our system for proportionality review of death sentences. By that we mean not the review of any legal error in the imposition of the sentence but, rather, the review of the sentence itself. We seek to determine "[w]hether the sentence of death is excessive or disproportionate to the penalty imposed in similar cases, considering both the crime and the defendant." Id. at 167, 96 S. Ct. at 2922, 49 L. Ed. 2d at 871. Proportionality review seeks "to ensure that the death penalty is being administered in a rational, non-arbitrary, and evenhanded manner, fairly and with reasonable consistency." State v. Marshall, 130 N.J. 109, 131 (1992), cert. denied, 507 U.S. 929, 113 S. Ct. 1306, 122 L. Ed. 2d 694 (1993) (Marshall II). That review serves as "a means through which to monitor the imposition of death sentences and thereby to prevent any impermissible discrimination in imposing the death penalty." State v. Ramseur, 106 N.J. 123, 327 (1987), denial of habeas corpus aff'd sub nom., Ramseur v. Beyer, 983 F.2d 1215 (3d Cir. 1992), cert. denied, 508 U.S. 947, 113 S. Ct. 2433, 124 L. Ed. 2d 653 (1993). "Proportionality review seeks to determine only whether a particular death sentence is aberrational, not whether it compares perfectly with other sentences." State v. Bey, 137 N.J. 334, 352 (1994) (Bey IV), cert. denied, 513 U.S. 1164, 115 S. Ct. 1131, 130 L. Ed. 2d 1093 (1995) (citing Marshall II, supra, 130 N.J. at 131). Our goal is to prevent the death penalty from being imposed "capriciously or in a freakish manner." Gregg, supra, 428 U.S. at 195, 95 S. Ct. at 2935, 49 L. Ed. 2d at 887. Our Dissenting member argues that we should insist that death sentences be "generally imposed" in similar cases for a sentence of death to be found to be proportional. Post at ___ (slip op. at 11).
"Because New Jersey jurors have been sparing in their imposition of the death sentence, it will never be the case that death would be "generally received" or "received in a defined preponderance of cases." Because juries impose death infrequently, we have recognized that "death need not be normal or general to be a licit sentence."" [State v. Loftin, 157 N.J. 253, 322 (1999) (Loftin II) (quoting Marshall II, supra, 130 N.J. at 153).]
Our Dissenting member asks for more. He "would have us find that death is the normal sentence [for similar cases] when that can never be so." Ibid.
In Loftin II, supra, 157 N.J. at 279, we considered our existing system of proportionality review and the effect of N.J.S.A. 2C:11-3e, which limited proportionality review "to a comparison of similar cases in which a sentence of death has been imposed." The details of our system of proportionality review are fully described in that opinion and need not be repeated here in any detail. We give this brief summary.
The system consists of two parts. The first part is frequency analysis, a statistical measure of the numerical frequency with which similar cases have resulted in sentences of death. The second part is precedent-seeking review, a traditional judicial way of comparing the files in similar cases to determine whether a defendant's death sentence is freakish or aberrational or the result of impermissible influences.
Until recently, similar cases were identified for purposes of frequency analysis by: (1) their salient factors (for example, cases involving prior murders or a sexual assault); (2) the raw numbers of statutory aggravating and mitigating factors (aggravating factors are those that make a murder death-eligible, such as the murder of a public official or murder in the course of committing a felony, and mitigating factors are those that may be weighed by a jury in determining whether to impose a sentence of death, such as extreme mental disturbance); and (3) an index of outcomes, a composite statistical test incorporating various statutory and non-statutory factors (such as motive or extent of premeditation) that sought to rank cases by the presence or absence of factors that appear to influence prosecutorial and jury decision-making.
These three statistical methods were applied to those cases that were clearly death-eligible, including cases in which the State had not sought the death penalty or defendants had obtained non-capital pleas. We refer to this as the universe of similar cases for purposes of comparison. The Administrative Office of the Courts (AOC) collects and organizes the data. Thus, each proportionality review involves an examination of prior cases through frequency analysis and precedent- seeking review. In Loftin II, supra, we found that because the raw numbers failed to account for the qualitative nature of aggravating and mitigating factors, the numerical preponderance test had not contributed to the Court's proportionality reviews and in light of its inherent flaws could not be expected to do so in the future. 157 N.J. at 295. We determined that test should be abandoned. Ibid.
In Loftin II we also expressed concern that the statistical methods used in the index-of-outcomes test to predict the probability of any defendant receiving a sentence may lack sufficient reliability. Id. at 295-96. We also examined the data in support of a contention that there had been "impermissible discrimination in imposing the death penalty." Loftin II, supra, 157 N.J. at 275 (quoting Ramseur, supra, 106 N.J. at 327). We referred to this as an inquiry into systemic proportionality review as opposed to individual proportionality review. In connection with that inquiry, we appointed retired Appellate Division Judge Richard S. Cohen to conduct a review and make findings and recommendations concerning whether a defendant's race or the race of the victim possibly affected prosecutorial decisions to seek, and jury decisions to impose, the death penalty. Following the receipt of Special Master Cohen's report and supplemental report, we concluded that the data did not demonstrate racial disparity in the imposition of the death penalty. Nevertheless, because many questions had been raised by the parties, by Special Master Cohen, and by the AOC, about our present systems of both individual and systemic proportionality review, we decided to remand the contested issues to a Special Master appointed to hear and take testimony and to report to the Court on the efficacy of the system.
Finally, in Loftin II we considered, but did not decide, whether N.J.S.A. 2C:11-3e would impermissibly infringe on our exercise of appellate review. We summed up our review of the system of proportionality review as follows:
"Our experience teaches us that the proportionality review methodologies we use are not without substantial shortcomings and, accordingly, warrant careful reconsideration. Our goal is to retain those elements of the present system that provide useful information, to refine and improve that which we retain, if appropriate, and to reject methods that have proved unhelpful. We seek a practical approach that ensures every defendant before us a rigorous and complete review of his or her sentence of death.
Our reconsideration extends to four discrete areas of concern: the size of the universe of comparison cases; particular issues in respect of individual proportionality review; questions relating to the statistical models used in both individual and systemic proportionality review, and the status of proportionality review as a separate proceeding in death penalty appeals. . . . Because these issues, with one exception, cannot be resolved on the record before us, we are appointing a Special Master to conduct additional fact-finding and make recommendations to the Court. On our receipt of his report, we will be in a position to determine whether the statutory limitation on the proportionality review universe prevents meaningful appellate review." [157 N.J. at 286-87 (footnote omitted).]
On February 1, 1999, this Court appointed Superior Court Judge David S. Baime, a Presiding Judge in the Appellate Division, as Special Master to evaluate the Court's proportionality review methodology, which was modeled after a proposal by the first Special Master, see David C. Baldus, Death Penalty Proportionality Review Project: Final Report to the New Jersey Supreme Court (Sept. 24, 1991), and first used in Marshall II, supra. See Loftin II, supra, 157 N.J. at 453-57. Generally, the Special Master was assigned to conduct a review, perform analyses, and make findings and recommendations relating to the discrete areas of concern that we had identified. Id. at 454-55.
Specifically, the Court ordered the following:
"(1) The Special Master shall conduct additional fact-finding concerning the proper scope of the proportionality review universe. The Special Master shall make an independent evaluation of the deathworthiness of a sample of cases previously classified by the Administrative Office of the Courts (AOC) as either death-eligible or death-ineligible. The "provability" of the selected cases and the presence or absence of aggravating and mitigating factors shall be considered and the results compared to the data-coding decisions made by the AOC. If there is a variance between the survey results and the AOC data-coding decisions, possible causes of the variance shall be identified along with recommendations for improved data- coding procedures. The Special Master shall consider whether a questionnaire should be filled out by the Judge in each case and used to improve both the data-collection and data-coding process. Alternatively, if the Special Master determines that the intrinsic difficulties and ambiguities of data-coding death-eligible cases cannot be overcome, the Special Master shall consider the impact of anticipated coding errors on the AOC models;
(2) The Special Master shall review data-coding generally and make recommendations for improvements if appropriate;
(3) The Special Master shall attempt to determine, based on projections about the size of the database over time and other relevant considerations, how long it will take before frequency review results can attain a level of statistical reliability;
(4) The Special Master shall undertake a review of both the strengths and weaknesses of the index-of-outcomes test and make recommendations whether the statistical models can be modified and improved or whether the index-of-outcomes test should be eliminated;
(5) The Special Master shall consider methods by which to select a representative number of cases within the group of similar cases for consideration and comparison to the defendant's case in the salient-factors test and precedent-seeking review. The Special Master shall examine alternate case sorting approaches that account for mitigating factors. The Special Master shall assess whether some reduction in the number of case classifications is possible without compromising the principle that only similar cases be compared;
(6) The Special Master shall attempt to develop parsimonious statistical models for more reliable regression studies of race effect and shall consider whether the process of purging, i.e., the removal of the indirect effects of race from variables that appear to be unrelated to race, produces results that are useful;
(7) The Special Master shall consider Special Master Cohen's recommendation, submitted in State v. Loftin, supra [Loftin II], that the Court appoint a panel of Judges to perform periodic assessments of penalty-trial outcomes, along with the composition and mandate of such an independent judicial panel, as independent verification of the culpability ratings derived from the models;
(8) The Special Master shall develop a factual record and issue findings concerning the desirability of maintaining proportionality review as a separate proceeding or, alternatively, conducting proportionality review in connection with a capital defendant's direct appeal . . . . [Id. at 455-56.]
Pursuant to that order, Judge Baime interviewed all personnel of the AOC responsible for screening cases, and gathered all protocols pertaining to AOC procedures. See David S. Baime, Report to the New Jersey Supreme Court: Proportionality Review Project at 18 (Apr. 28, 1999) (Baime Report). He conducted an independent screening of 105 AOC files for death-eligibility and enlisted retired Appellate Division Judge Charles Villanueva to screen an additional 105 files. See ibid. He asked the Attorney General and the New Jersey County Prosecutors' Association to present evidence concerning the extent to which prosecutors decided not to seek the death penalty, and specifically sought information from prosecutors regarding cases that were not capitally prosecuted but that Judge Baime found to be "clearly death- eligible." See ibid.
With respect to data coding, Judge Baime directed two law clerks to simulate the process in accordance with AOC protocols for twenty-four case files each and to compare their results with the AOC's. See id. at 42. With the aid of John P. McCarthy, Jr., Esq., Director of the Office of Trial Court Services, and Joseph J. Barraco, Esq., Assistant Director of Criminal Practice, Judge Baime considered all 433 death-eligible homicides in an attempt to discern meaningful case characteristics and the effect of mitigating factors on deathworthiness. See id. at 52-53. Further, Judge Baime retained statisticians Dr. David Weisburd and Dr. Joseph Naus as consultants to study the performance of the index-of- outcomes test. See id. at 4.
The Special Master's report recommended the following:
"(1) retention of the clearly death-eligible universe of cases for proportionality review with the following improvements in the AOC's methodology;
(a) appointment of a retired Judge to serve as Standing Master to supervise AOC screening, data collecting and data-coding functions, and to preside over hearings pertaining to these subjects;
(b) adoption of a protocol or rule mandating that all hearings before the Standing Master be kept confidential, that all information divulged by counsel during such proceedings may not be used for any purpose other than proportionality review, and that all transcripts and records of such proceedings be sealed;
(c) adoption of a protocol or rule requiring that all criminal division managers forward to the AOC additional sources of information for data-coding and screening, including the judgment of conviction, presentence report, notice of aggravating factor(s), any order resulting from a motion to dismiss, decisions pertaining to evidentiary questions, the requisite plea form, indictment or accusation, all defendant statements provided in discovery, all witness statements provided in discovery, all investigative reports from any law enforcement agency, defense notice of mitigating factors, autopsy and medical examiner reports, psychiatric evaluations and psychological reports;
(d) adoption of a protocol on an experimental or pilot program basis requiring trial Judges to complete questionnaires in all potentially death-eligible homicide cases;
(e) modification of the AOC's evidence typology to require that evidence of capital murder elements must be "overwhelming" as a prerequisite for inclusion of a case in the death-eligible universe;
(2) adoption of a protocol requiring "double coding" and editing by individual members of the AOC staff, memorialization of all interpretive rules respecting key variables, and periodic updating of the data base to reflect changes in data-coding rules;
(3) adoption of a modified salient factors test that accounts for mitigation and contains fewer categories;
(4) abandonment of the index-of-outcomes test and logistic multiple- regression analysis for individual proportionality review because of the instability of current models and the relatively limited projected increase in New Jersey's data base;
(5) continued experimentation toward the creation of a reliable statistical model for the purpose of systemic proportionality review;
(6) adoption of a model charge reminding jurors of their duty to consider each case fairly and without regard to race, ...