Reference Range Analysis
Lessons from PSA

Paul E. C. Sibley, Ph.D.
International Marketing Manager, Tumor Markers

Editor's Note: This article derives from a presentation on age-related reference ranges given jointly with Catharine M. Sturgeon, Ph.D. (Royal Infirmary, Edinburgh) at the May 1999 London meeting of the PSA Working Party, a task force of the Association of Clinical Biochemists (ACB).¹ While Dr. Sturgeon's contribution surveyed the arguments for and against adopting age-adjusted decision limits for prostate-specific antigen (PSA), Dr. Sibley's focused on the emergence of more precise techniques for estimating PSA reference limits as a function of age.

It is a measure of the gulf between clinical research and laboratory practice that most respondents (nearly 85%) in a recent UK proficiency survey still quoted 4 µg/L as the upper reference limit for PSA in adult males.² This is recognizably an echo of the eighties—the legacy of a single, surprisingly consequential study which led to the entrenchment of 4.0 µg/L as an important decision limit in PSA testing.³ From a modern standpoint, the study had serious shortcomings; but it did not go unchallenged. Over the last decade, several research groups have revisited the issue, generally motivated by the hope that better delineation of PSA reference limits would result in the development of better decision limits.

The original study was based, of course, on an older (isotopic) technology, which predated an appreciation for the diverse molecular forms of PSA and the need for standardization.⁴ Moreover, a significant covariate, namely age, was largely ignored in both the design of the study and its analysis. The distribution of results was summarized in terms of a single upper reference limit, 4.0 µg/L, as if this were applicable to adult men of any age; and the reference group itself consisted principally of younger men, well over half of them under 40 years old.⁵ (In modern urological practice, where a decision limit of 4.0 µg/L would be relevant, PSA tests are applied mainly to samples from men 40 to 50 years of age or older.⁶)

Even in the 1980s, it was recognized that circulating PSA levels in men gradually increase with age. Some such pattern was to be expected, after all, due to increased prostate size (volume), though increased "leakage" of PSA into the circulation and other factors may also be at work.⁷

Limited pediatric investigations, by chronological age or pubertal stage, have shown that PSA levels are detectable in most boys by their late teens, though the distribution of values at that stage is overall lower than for men in their 30s or 40s.⁸ So far, no one has tried to extend the age-related analysis back to puberty, even though this might yield valuable insights into the proper functional form for representing the age-related increase in PSA. (Even with the help of a third generation assay, such a study would have to cope with results below the detection limit.⁹)

At the other end of the age spectrum, there is good evidence that the correlation between age and PSA gradually deteriorates, to the extent that we can not expect an age-related model to apply beyond the seventh decade.¹⁰ Even so, in men 40 to 70 years old, the increase is dramatic enough to require a more refined, age-related analysis and presentation of results, as well as larger, more carefully designed reference range studies.

A modern age-related analysis
As early as 1993, Oesterling published the results of a community-based reference range study of PSA, involving 461 men, 40 to 80 years old, with no evidence of prostate cancer by PSA, DRE or transrectal ultrasound.¹¹ The article included a "nomogram" depicting the continuous rise of PSA values as a function of age in this population. Figure 1 shows a similar nomogram, with a somewhat different spectrum of centile curves, constructed from the results of a cross-sectional study by Dr. Axel Semjonow (Münster, Germany).¹² This study, based on the IMMULITE® Third Generation PSA, was reasonably comparable to Oesterling's in size, age distribution, criteria of normality, and data processing. (Men with suspicious PSA or DRE results were subjected to ultrasound-guided sextant biopsy.) The continuous centile curves were generated parametrically, avoiding premature subgrouping by age, using a regression method like the one adopted in Oesterling's study but incorporating two lessons from subsequent work in this field: the analysis made allowance for an age-related increase in subject-to-subject variability, and did not assume that the distribution of results must be either gaussian or log-normal.¹³

IMMULITE® Third Generation PSA

Figure 1. Scatterplot of PSA vs. age, with representative centile curves superimposed—similar to the "nomogram" in Oesterling et al, JAMA 1993. PSA results were obtained on samples from the Semjonow study (see text). A family of regular contour lines was constructed by a modern parametric technique for determining reference limits as a continuous function of age. The results are highly skewed towards higher values, making it difficult to identify outliers in this representation.

According to IFCC and NCCLS guidelines, the analytical goal of a reference range study is inherently descriptive rather than normative.¹⁴ The aim is to characterize in terms of centiles (usually estimated from measurements obtained on a necessarily limited sample) the underlying distribution of an analyte concentration or similar quantity in a well-defined reference population.¹⁵ An estimated 95th centile, for example, is intended to exclude 5% of the underlying population and is judged to fit the data to the extent that approximately this percentage of the observations lies above the estimated value.¹⁶

The age-related analysis encapsulated in a nomogram like Figure 1 constitutes a natural extension of this concept. Each of the centile curves represents a genuine contribution towards mapping the distribution of values in the reference group, relative to which a physician can "locate" (make sense of) the PSA result for a new subject of known age.¹⁷ Goodness-of-fit now requires, in addition, that points excluded by the centile curve be fairly evenly distributed across the age span, rather than clustering at one end or the other.

Superimposed on a scatterplot of PSA versus age, centiles estimated without taking age into account would necessarily yield a family of horizontal lines. (In Figure 1, the conventional 4.0 µg/L limit coincides with one of the grid lines.)

Presentation matters
In the future, one hopes, laboratory report forms will incorporate graphs and pictorial elements; but for now they are generally text-based and extremely brief, listing two or three centiles at most, for a very small number of subgroups.

Starting from a nomogram (or an algebraic equivalent), a continuous centile curve can be adequately reduced to a list of values, each corresponding to an age bracket, simply by reading off from the curve the value at the midpoint of each age bracket.¹⁸

As for which centiles to quote, the IFCC and NCCLS guidelines remain neutral, treating this as a matter of convention. At the high end, the 95th or 97.5th centile is most commonly recorded.¹⁹

Where both unusually high and unusually low levels are of clinical interest, a central 95% interval (defined by the 2.5th and 97.5th centiles) is often quoted when the distribution of reference values must be characterized in minimal terms; but where a one-sided interpretation is appropriate, as for PSA, the lower 95% interval (from nondetectable to the 95th centile) is a more natural choice. For tumor markers, there is an additional argument for quoting the 95th centile —namely, fear that the criteria of normality applied may not have sufficed to exclude from the reference group all subjects with the disease in question.²⁰

Figure 1 demonstrates how important it is to know which centile is intended when an upper reference limit is quoted. The centile curves predict concentration levels (see Table 1) for the 95th, 97.5th and 99th centiles which are substantially different from one another when measured against the 4 µg/L span of the conventional reference interval. The curves do predict a PSA level of 4.0 µg/L as the upper limit for men of a certain age; but this depends on which centile is chosen to represent that limit: the 99th, 97.5th and 95th centile curves cross 4.0 µg/L at approximately 56-57, 60-61 and 65-66 years of age, respectively. (In surviving summaries of the study from the mid-1980s, 4.0 µg/L is identified as the 99th centile for the entire data set and also as the 97th centile for the 207 results from men at least 40 years of age—as if this coincidence were support for quoting 4.0 µg/L as the upper limit of normal, whereas it only shows that PSA levels may be age-related.²¹)

Furthermore, any centile purporting to represent the upper limit of the overall distribution would be highly dependent on the age mix of the reference group: the 99th centile for the Semjonow study is 6.0 µg/L as a whole, but 4.0 µg/L and 2.4 µg/L, respectively, when limited to subjects under 60 and 50 years of age.

Local methods
What's needed, clearly, is an analysis yielding centile estimates for men at any given age. Unfortunately, the IFCC guidelines make no provision for treating age as a continuous covariate. Instead, even in mature presentations of this approach, age is assimilated to sex and race, where one begins by partitioning the data and performing separate analyses—and then tries to determine whether the subgroups can be recombined after all.²²

In general, subgrouping with respect to age makes good sense only when there are physiological events (puberty or menopause, for example) determining the partition.²³ In other contexts, arbitrariness associated with the age brackets renders the analysis by subgroups unsatisfactory; such is the case for PSA. In practice, dividing the data into a small number of nonoverlapping age brackets has generally meant subgrouping by decades, beginning at age 40, reflecting a preference for memorable round numbers rather than any scientific basis.

This approach reduces the age-related analysis to a series of computationally simpler "local" analyses where existing guidelines can be applied and age no longer enters as a significant factor. Figure 2 illustrates the relative merits of four approaches to determining local reference limits for the 50- to 60-year data from Figure 1.

Figure 2. Four estimates of the central 95% interval for results in the 50- to 60-year age bracket. The simple parametric approach (mean ± 1.96 SD) applied directly, without some transformation to improve symmetry, yields falsely low estimates of the 2.5th and 97.5th centiles. In this case, when applied to log-transformed data, the parametric approach yields estimates in agreement with the distribution-free Harrell-Davis estimates.

For its simplicity, the IFCC and NCCLS guidelines recommend the following nonparametric technique for centile estimation, especially in laboratories where statistical expertise may be limited.²⁴ Given a set of N reference values, list them in ascending order; assign a rank: R=1,2,..,N; calculate a centile for each result as R/(N+1); then obtain by interpolation any centiles not corresponding to one of the reference values.²⁵ [It should be noted that built-in spreadsheet functions typically calculate centiles in a radically different way, as (R-1)/(N-1), making them entirely inappropriate for reference range analysis in small samples.²⁶]

An undesirable feature of this nonparametric approach is that estimates may depend critically on just a few of the observations—a matter of special concern when relatively extreme centiles are at issue, as they usually are, and the distribution is highly skewed, as for PSA. The parametric approach most widely used is based on estimating centiles as the mean plus or minus so many standard deviations.²⁷ This has the virtue of using the entire data set, but depends on the distribution being reasonably gaussian: its application, therefore, requires determining an adequate transformation to normality.²⁸

The Harrell-Davis approach represents a balanced alternative. This widely studied, distribution-free centile estimator met with a favorable reception in clinical chemistry as early as 1985, and is now considered the nonparametric method of choice in reference range analysis, having been advocated in the Harris and Boyd textbook and other key publications.²⁹ (In certain respects, it resembles Healy's well-known method for dealing with outliers in external proficiency surveys.³⁰)

The Harrell-Davis technique estimates any given centile as a weighted average of the sorted reference values—or (using another vocabulary) as a "linear combination of order statistics".³¹ Like the parametric approach, it exploits the entire data set, but without the need for transformations. On the other hand, the tools for implementing it are widely but not universally available; and outliers can distort the centile estimates, as in the parametric approach.³² Accordingly, DPC has made extensive use of Harrell-Davis estimators in recent years, but always in conjunction with suitable provisions for identifying outliers.

As shown in the background of Figure 3, the all too common practice of analyzing PSA reference range data on a decade-by-decade basis, starting (say) at age 40, begins to follow the age-related rise in circulating PSA. Equally apparent is the arbitrariness of both bin size and alignment. The approach yields a crude "stair case" approximation to the underlying centile curves, which can hardly be regarded by physician or patient as satisfactory: why should turning 50, say, have a precipitous impact on a man's PSA level or its classification? The "birthday effect" is pure artifact, tracing to an undesirably coarse analysis and/or summary of the reference data.

Figure 3. Three centiles (median, central 95% interval) estimated nonparametrically, decade-by-decade (white) and by discrete 5-year intervals (magenta). Using 10-year intervals, the 97.5th centile appears to double suddenly at age 50, and again at age 60. Using a 5-year window, the estimated 97.5th centile for age 60 is lower than for age 55—an artifact of the analysis rather than a genuine feature of the data. Disparities between the 5- and 10-year methods convey some sense of the uncertainties associated with a sequence of local estimates, chained together, especially at the more extreme centiles.

Attempts to refine the analysis, however, run up against an "uncertainty principle": smaller bins mean that the local calculations are each supported by fewer data points, resulting in rougher, less precise centile estimates. The white and magenta frames in Figure 3 illustrate the impact of different choices of bin size and alignment. Upper limits based on the narrower age brackets no longer increase monotonically; but the apparent dip is surely due to the greater sparsity of the data and the way stray points happen to be trapped on one side or the other of arbitrarily imposed age brackets.

A more promising nonparametric approach is illustrated by the dashes and the black curve in Figure 4. It involves generating a series of local centile estimates for overlapping age brackets, each large enough to contain a substantial number of data points, and then fitting a smooth curve (with not too much flexibility) to the estimates.³³ The local estimates can be expected to exhibit considerable variability, especially at outer centiles. Here, for the 95th centile, which is not especially extreme relative to the amount of data available, the nonparametric curve (black) is a good match for the parametric curve (red) reproduced from Figure 1. The two approaches are mutually supportive.

Figure 4. Dashes (black) represent 95th centiles, estimated nonparametrically for overlapping 5-year age brackets. The black curve, a smoothing of these estimates, is in good agreement with the 95th centile curve (red) generated by the parametric approach used for Figures 1 and 6. The curves generated by simple linear regression, with (green) or without (blue) prior transformation to symmetry, are both unduly flat—too high on the left, too low on the right. (See Figure 5.)

Global methods
Global, parametric approaches have much to recommend them—when they work. Figures 5 and 6 illustrate the basic idea.³⁴ The PSA values first require transformation, to make the distribution of data points more gaussian (or anyhow more symmetric) around a central trend line. For the Semjonow study, it suffices to raise PSA values to the 0.16 power—that is, to apply an optimal transform, capable of stretching and compressing the distribution of values somewhat more strongly than a quarter-root transform but less strongly than a logarithmic transform.³⁵

Figure 5. PSA vs. age, with PSA represented on a power scale (intermediate between linear and log) to optimize for vertical symmetry around the central trend line. Simple regression approaches incorrectly assume that the vertical spread is constant, i.e. age-independent. Parallel contours, based on adding or subtracting multiples of the standard deviation of the overall vertical scatter in this representation, exclude too many points at one end, too few at the other. The 95th centile contour (green) is the same as the like-colored contour in Figure 4.

Figure 6. Scaled as in Figure 5, but with a central 95% envelope doing better justice to the gradual, age-related increase in dispersion. There are no glaring outliers. Visual inspection confirms that the contours fit the data well: the number of points falling outside the envelope (above or below) is now approximately 5% of the total; and these are scattered uniformly across the age span, rather than bunched at one end. The 97.5th centile curve is the same, after back-transformation, as the corresponding centile curve in Figure 1.

This yields a better framework within which to inspect for outliers, because the transformation has succeeded in reducing the skewness evident in Figure 1. (On a linear scale, the results are very far from being symmetric, let alone gaussian.) Here there is less danger of wrongfully discarding a data point as an outlier based on visual inspection.³⁶

Figures 5 and 6 also reveal that the group-based spread of PSA values markedly increases with age—something not readily apparent in Figure 1 where PSA appears on a linear scale.³⁷ Figure 6, in contrast to Figure 5, shows the effect of accounting for this age-related increase in subject-to-subject variability. The end result is an envelope (targeting one or a pair of continuous centile estimates) which succeeds both in excluding an appropriate number of data points and in doing so more or less uniformly across the age span.³⁸ Back-transformation of the data points and selected centile curves now yields Figure 1.

Conclusions
Typical reference range analyses concentrate on just one distribution, that of some reference population; there is often no comparable information on the distribution(s) of subjects with the relevant disease(s). In this situation, one can adjust for specificity by selecting a reference limit (centile) which allows a given percentage of the reference population to be treated as unrepresentative outsiders. Determining a rational, disease-oriented decision level, on the other hand, requires comparable information on the sensitivity of the test at the concentration level selected, i.e. its ability to identify subjects with the relevant disease(s).³⁹

It may be that optimal decision levels for PSA do not, after all, coincide with any properly estimated, age-related centile curve. To take just one example: Catalona et al. concluded, in one context, that an age-independent decision level of 4.0 µg/L would have the desirable property of yielding a constant number of biopsies performed for each cancer detected—evidently a clinically more relevant consideration than maintenance of constant specificity across the age spectrum.⁴⁰

Nevertheless, the evolution of improved age-related reference range analyses for PSA represents genuine progress. Even in the prostate cancer field, these techniques can be, and have been, used to characterize the distribution of values in other significant populations, including men of different races (African-Americans, Japanese, etc.) and men with benign prostatic enlargement (BPE) or other urological symptoms.⁴¹

The same techniques are applicable to other assays and other covariates (such as gestational age or menstrual cycle position) in many fields of laboratory medicine. The importance of the study published by Oesterling derives in part from its being one of the first analyses, for a major analyte commonly measured by immunoassay, to demonstrate how to deal with continuous covariates. Subsequent PSA studies have also helped to raise the standards for reference range analysis.⁴² Review papers from Patrick Royston's circle in the UK, as well as the textbook by Harris and Boyd, provide useful perspectives on the current state of the art.⁴³

Modern statistical tools for discrimination and classification, such as logistic regression and neural networks, can be expected to yield more robust and meaningful results to the extent that the training sets are verifiably representative of the groups in question.⁴⁴ Moreover, reference range analysis is central to evidence-based medicine (EBM), with its emphasis on likelihood ratios, because a detailed characterization of the distribution of values in both subjects with the disease(s) in question and the appropriate reference population provides the basis for assessing the likelihood that a patient belongs to one group rather than the other, given a test result.⁴⁵

Notes

1. The presentations by Dr. P. E. C. Sibley and Dr. C. M. Sturgeon are to be combined for publication as part of a document summarizing the London meeting of the ACB’s PSA Working Party (Chairman: Prof. C. P. Price).

2. UK survey: [Mil99].

3. "Original" Hybritech Tandem-R study: [Hyb86], [Myr86]. Another study: [Cha87], [Roc87]. See also [Lin90], [Hol93].

4. Diverse molecular forms, standardization: [Sem96], [Sem98b], [Sta98], [Sta00].

5. Age distribution in Hybritech study: [Myr86]. Misconceptions: [Dea97], [Gus98].

6. Relevant age bracket: [Oes94], [ACS97], [Pol99]. Guidelines: [Mos98], [Car99], [God99].

7. Leakage model: [Bab92], [Oes95b], [Oes96], [Ste96].

8. Pediatric studies: [Vie94], [Rau96], [Juu97].

9. Censored samples: [Tsa79].

10. Loss of correlation: [Kir96], [Kir97].

11. Oesterling study, JAMA 1993: [Oes93]. Compare: [Bab92].

12. Semjonow study: [Sem98a]. Longitudinal vs. cross-sectional: [Pea94], [Roe00]

13. Age-related increase in variance: [And95], [Wri96], [Wri99]. Power transforms: [Hoa83], [Sol89], [H&B95].

14. IFCC/NCCLS approach: [Sol87], [Lin87], [Sol89], [Ken93], [Sol95], [NCC95], [Str96], [Sol99].

15. Centiles as fundamental: [Ree71], [Elv72], [Ros79], [H&B95]. See also: [Str86], [Par91], [Bis93].

16. Goodness of fit: [H&B95], [Hor98], [Wri99].

17. "Locating" patient results: [Elv72], [Fei74], [Ros79], [Alb81].

18. Reading off midpoints: [Oes93], [Oes95a], [Sib99d].

19. IFCC/NCCLS neutrality on which centiles to quote: [Sol87], [NCC95].

20. Lower 95% range in oncology: [Oes94], [Jac95], [Ste97], [Sib99d]. But consider: [Tsa79], [Goo88].

21. Rationale behind 4.0 ng/mL as upper reference limit: [Myr86].

22. IFCC perspective on age brackets: [Sol89], [Sol99].

23. When not to subgroup by age: [Har75], [Wri99].

24. Nonparametric approach has simplicity to recommend it: [Sol87], [NCC95].

25. IFCC/NCCLS rule, Rank/(N+1): [Ree71], [Shu85], [Sol87], [NCC95], [H&B95].

26. Other rules, including (Rank–1)/(N–1): [Cun78], [Dur98].

27. Local parametric approaches: [Mai71], [Hea95].

28. IFCC 2-stage transformation to normality: [Sol89], [Lin87], [H&B95], [Str96].

29. Harrell-Davis: [Har82], [Shu85], [H&B95], [Har98]. See also [Hor98], [Hor99].

30. Healy’s trimmed mean: [Hea79].

31. Trimean as a weighted average of order statistics: [Hoa83].

32. Harrell-Davis implemented in S-Plus: [Wil97]. See also: [Ven99].

33. Healy’s nonparametric approach: [Hea88], [Pan90], [Gol92], [Wri96]. See also [And95].

34. Global parametric approaches: [And95], [H&B95], [Wri96], [Roy98], [Wri99].

35. Power transforms: [Hoa83], [Str96].

36. Impact of scale on visual inspection: [Sol89], [Cle93], [Gre93], [Dur98].

37. Age-related increase in spread: [And95], [H&B95], [Kal99].

38. Goodness of fit: [Hor98], [Wri99].

39. Normal ranges and specificity levels: [Gal77], [Hea86], [Hea95], [And95], [Dal95], [Jac95].

40. Hopes for age-related reference limits: [Elg95], [Oes96], [Ste96]. Catalona’s objection: [Cat94]. Other representative critiques: [Par96a], [ACS97], [Dea97], [Nix97], [Bas98], [Pol99].

41. Age-related ranges: [Dal95], [Elg95], [Etz96], [Cra97], [Dea97], [Ric97], [Ata98], [Kal99]. African-American: [Dea96], [Mor96], [Smi96], [Wei98b], [Fow99]. Asian: [Chu93], [Oes95a], [Lin96], [Kao97], [Shi97], [Nak99]. Age-related ranges for special conditions: [Chu93], [Jac96], [Mei96], [Wei98b], [Res99], [Roe00]. See also: [Ran89].

42. Improved age-related reference range analyses for PSA: [And95], [H&B95].

43. State-of-the-art: [H&B95], [Wri96], [Roy98], [Wri99].

44. Neural networks and representativeness: [Kro99]. See also: [Gre93], [Smi93], [Rip94], [Wei98a].

45. EBM, likelihood ratios: [Alb82], [Lin88], [Boy97], [Moo97], [Goo99], [Rem99], [Sha99]. Some applications to PSA: [Cat94], [Jac96b], [Mei96].

References

[ACS97] Prostate cancer: detection and symptoms. American Cancer Society (ACS); Prostate Cancer Resource Center (www.cancer.org) 1997.

[Alb81] Albert A. Atypicality indices as reference values for laboratory data. Am J Clin Pathol 1981;76:421-5.

[Alb82] Albert A. On the use and computation of likelihood ratios in clinical chemistry. Clin Chem 1982;28:1113-9.

[And95] Anderson JR, Strickland D, Corbin D, et al. Age-specific reference ranges for serum prostate-specific antigen. Urology 1995;46:54-7.

[Ata98] Atalay AC, Karaman MI, Güney S, et al. Age-specific PSA reference ranges in a group of non-urologic patients. Int Urol Nephrol 1998;30:587-91.

[Bab92] Babaian RJ, Miyashita H, Evans RB, Ramirez EI. The distribution of prostate specific antigen in men without clinical or pathological evidence of prostate cancer: relationship to gland volume and age. J Urol 1992;147:837-40.

[Bas98] Bassler TJ, Orozco R, Bassler IC, et al. Most prostate cancers missed by raising the upper limit of normal prostate-specific antigen for men in their sixties are clinically significant. Urology 1998;52:1064-9.

[Bis93] Bishop JC, Dunstan FD, Nix BJ, Reynolds TM, Swift A. All MoMs are not equal: some statistical properties associated with reporting results in the form of multiples of the median. Am J Hum Genet 1993;52:425-30. Comment: ibid 1993;53:777-81.

[Boy97] Boyd JC. Mathematical tools for demonstrating the clinical usefulness of biochemical markers. Scand J Clin Lab Invest 1997;57(Suppl 227):46-63.

[Car99] Carter HB, Pearson JD. Prostate-specific antigen testing for early diagnosis of prostate cancer: formulation of guidelines. Urology 1999;54:780-6.

[Cat94] Catalona WJ, Hudson MA, Scardino PT, et al. Selection of optimal prostate specific antigen cutoffs for early detection of prostate cancer: receiver operating characteristic curves. J Urol 1994;152:2037-42.

[Cha87] Chan DW, Bruzek DJ, Oesterling JE, et al. Prostate-specific antigen as a marker for prostatic cancer: a monoclonal and a polyclonal immunoassay compared. Clin Chem 1987;33:1916-20.

[Chu93] Chute CG, Panser LA, Girman CJ, et al. The prevalence of prostatism: a population-based survey of urinary symptoms. J Urol 1993;150:85-9.

[Cle93] Cleveland WS. Visualizing data. Murray Hill (NJ): AT&T Bell Laboratories, 1993.

[Cra97] Crawford ED. Prostate Cancer Awareness Week: September 22 to 28, 1977. CA Cancer J Clin 1997;47:288-96.

[Cun78] Cunnane C. Unbiased plotting positions. J Hydrology 1978;37:205-22.

[Dal95] Dalkin BL, Ahmann FR, Kopp JB, et al. Derivation and application of upper limits for prostate specific antigen in men aged 50-74 years with no clinical evidence of prostatic carcinoma. Br J Urol 1995;76:346-50.

[Dea96] DeAntoni EP, Crawford ED, Oesterling JE, et al. Age- and race-specific reference ranges for prostate-specific antigen from a large community-based study. Urology 1996;48:234-9.

[Dea97] DeAntoni EP. Age-specific reference ranges for PSA in the detection of prostate cancer. Oncology (Huntingt) 1997;11:475-82, 485.

[Dur98] Durham AP. Reference range "verification". Roundtable presentation at Clinical Ligand Assay Society (CLAS) National Meeting, Rye Brook, NY, 1998 May.

[Elg95] El-Galley RES, Petros JA, Sanders WH, et al. Normal range prostate-specific antigen versus age-specific prostate-specific antigen in screening prostate adenocarcinoma. Urology 1995;46:200-4.

[Elv72] Elveback LR. How high is high? A proposed alternative to the normal range. Mayo Clin Proc 1972;47:93-7.

[Etz96] Etzioni R, Shen Y, Petteway JC, Brawer MK. Age-specific prostate-specific antigen: a reassessment. Prostate Suppl 1996;7:70-7.

[Fei74] Feinstein AR. The derangements of the range of normal. Clin Pharmacol Ther 1974;15:528-40.

[Fow99] Fowler JE, Bigler SA, Kilambi NK, Land SA. Relationships between prostate-specific antigen and prostate volume in black and white men with benign prostate biopsies. Urology 1999;53:1175-8.

[Gal77] Galen RS. The normal range; a concept in transition. Arch Pathol Lab Med 1977;101:561-5.

[God99] Godley PA. Prostate cancer screening: promise and peril. Cancer Detection Prev 1999;23:316-24.

[Gol92] Goldstein H, Pan H. Percentile smoothing using piecewise polynomials, with covariates. Biometrics 1992;48:1057-68.

[Goo88] Goodman SN. One-sided or two-sided p values? Control Clin Trials 1988;9:387-8.

[Goo99] Goodman SN. Toward evidence-based medical statistics. Ann Intern Med 1999;130:995-1004 and 1005-1013.

[Gre93] Greenland S. Summarization, smoothing and inference in epidemiologic analysis. Scand J Soc Med 1993;21:227-32.

[Gus98] Gustafsson O, Mansour E, Norming U, et al. Prostate-specific antigen (PSA), PSA density and age-adjusted PSA reference values in screening for prostate cancer. Scand J Urol Nephrol 1998;32:373-7.

[H&B95] Harris EK, Boyd JC. Statistical bases of reference values in laboratory medicine. New York: Marcel Dekker, 1995.

[Har75] Harris EK. Some theory of reference values. Stratified (categorized) normal ranges and a method for following an individual's clinical laboratory values. Clin Chem 1975;21:1457-64.

[Har82] Harrell FE, Davis CE. A new distribution-free quantile estimator. Biometrika 1982;69:635-40.

[Har98] Harris EK. Normal values of biological characteristics. In: Armitage P, Colton T, editors. Encyclopedia of biostatistics. New York: John Wiley, 1998:3069-75.

[Hea79] Healy MJR. Outliers in clinical chemistry quality-control schemes. Clin Chem 1979;25:675-7.

[Hea86] Healy MJR. Statistics of growth standards. In: Falkner F, Tanner JM, editors. Human growth; a comprehensive treatise. 2nd ed. New York: Plenum Press, 1986;3:47-58.

[Hea88] Healy MJR, Rasbash J, Yang M. Distribution-free estimation of age-related centiles. Ann Hum Biol 1988;15:17-22.

[Hea95] Healy MJR. Diagnostic and screening tests and reference values. Arch Dis Child 1995;72:358-61.

[Hoa83] Hoaglin DC, Mosteller F, Tukey JW, editors. Understanding robust and exploratory data analysis. New York: John Wiley, 1983.

[Hol93] Holmäng S, Lindstedt G, Mårin P, Hedelin H. Serum concentration of prostate-specific antigen in relation to prostate volume in 50 healthy middle-aged men. Scand J Urol Nephrol 1993;27:15-20.

[Hor98] Horn PS, et al. A robust approach to reference interval estimation and evaluation. Clin Chem 1998;44:622-31.

[Hor99] Horn PS, et al. Reference interval computation using robust vs parametric and nonparametric analyses. Clin Chem 1999;45:2284-5.

[Hyb86] Tandem-R PSA [package insert]. San Diego: Hybritech Inc, 1986(Feb):1-6. Document no. 701159-026C.

[Jac95] Jacobsen SJ, Oesterling JE. Age-specific reference ranges for serum prostate specific antigen levels. J Clin Ligand Assay 1995;18:93-97.

[Jac96a] Jacobsen SJ, Bergstralh EJ, Guess HA, et al. Predictive properties of serum prostate-specific antigen testing in a community-based setting. Arch Intern Med 1996;156:2462-8.

[Jac96b] Jacobsen SJ, Guess HA, Oesterling JE. Selection of optimal prostate specific antigen cutoffs for early detection of prostate cancer: receiver operating characteristic curves. J Urol 1996;155:1395-6.

[Juu97] Juul A, Müller J, Skakkebæk NE. Prostate specific antigen in boys with precocious puberty before and during gonadal suppression by GnRH agonist treatment. Eur J Endocrinol 1997;136;401-5.

[Kal99] Kalish LA, McKinlay JB. Serum prostate-specific antigen levels (PSA) in men without clinical evidence of prostate cancer: age-specific reference ranges for total PSA, free PSA, and percent free PSA. Urology 1999;54:1022-7.

[Kao97] Kao C-H. Age-related free PSA, total PSA and free PSA/total PSA ratios: establishment of reference ranges in Chinese males. Anticancer Res 1997;17:1361-6.

[Ken93] Kenny D, Solberg HE. RefVal for Windows. Clin Chim Acta 1993;222:19-21.

[Kir96] Kirollos MM. Prostate-specific antigen and age; is there a correlation? and why does it seem to vary? Eur Urol 1996;30:296-301.

[Kir97] Kirollos MM. Statistical review and analysis of the relationship between serum prostate specific antigen and age. J Urol 1997;158:143-5. Erratum: ibid 1997;158:1530.

[Kro99] Krongrad A, Lai S. Artificial neural networks in urology. Urology 1999;54:949-51.

[Lin87] Linnet K. Two-stage transformation systems for normalization of reference distributions evaluated. Clin Chem 1987;33:381-6.

[Lin88] Linnet K. A review on the methodology for assessing diagnostic tests. Clin Chem 1988;34:1379-86.

[Lin90] Lindstedt G, Jacobsson A, Lundberg P-A, et al. Determination of prostate-specific antigen in serum by immunoradiometric assay. Clin Chem 1990;36:53-8.

[Lin96] Lin WY, Gu CJ, Kao CH, et al. Serum prostate-specific antigen in healthy Chinese men: establishment of age-specific reference ranges. Neoplasma 1996;43:103-5.

[Mai71] Mainland D. Remarks on clinical norms. Clin Chem 1971;17:267-74.

[Mei96] Meigs JB, Barry MJ, Oesterling JE, Jacobsen SJ. Interpreting results of prostate-specific antigen testing for early detection of prostate cancer. J Gen Intern Med 1996;11:505-12. Comment: ibid 1997;12:200-1.

[Mil99] Milford-Ward AM, White P. UK NEQAS for PSA: reference range survey. Personal communication (to C.M. Sturgeon), 1999.

[Moo97] Moore RA. Evidence-based clinical biochemistry. Ann Clin Biochem 1997;34:3-7.

.[Mor96] Morgan TO, Jacobsen SJ, McCarthy WF, et al. Age-specific reference ranges for serum prostate-specific antigen in black men. New Engl J Med 1996;335:304-10.

[Mos98] Moss SM, Melia J. Screening for prostate cancer: the current position. Br Med Bull 1998;54:791-805.

[Myr86] Myrtle JF, Klimley PG, Ivor LP, Bruni JF. Clinical utility of prostate specific antigen (PSA) in the management of prostate cancer [technical report]. San Diego: Hybritech Inc, 1986:1-6.

[Nak99] Nakanishi H, Nakao M, Nomoto T, et al. The investigation of age-specific reference range as the cut-off values in the mass screening for prostatic cancer. Nippon Hinyokika Gakkai Zasshi 1999;90:853-8 [Japanese].

[NCC95] National Committee for Clinical Laboratory Standards. How to define and determine reference intervals in the clinical laboratory; approved guideline. Wayne, PA: NCCLS, 1995. NCCLS Document C28-A.

[Nix97] Nixon RG, Brawer MK. Enhancing the specificity of prostate-specific antigen (PSA); an overview of PSA density, velocity and age-specific reference ranges. Br J Urol 1997;79(Suppl 1):61-7.

[Oes93] Oesterling JE, Jacobsen SJ, Chute CG, et al. Serum prostate-specific antigen in a community-based population of healthy men; establishment of age-specific reference ranges. JAMA 1993;270:860-4.

[Oes94] Oesterling JE. Age-specific reference ranges for prostate-specific antigen. JAMA 1994;271:747.

[Oes95a] Oesterling JE, Kumamoto Y, Tsukamoto T, et al. Serum prostate-specific antigen in a community-based population of healthy Japanese men: lower values than for similarly aged white men. Br J Urol 1995;75:347-53.

[Oes95b] Oesterling JE. Prostate specific antigen: its role in the diagnosis and staging of prostate cancer. Cancer Suppl 1995;75:1795-1804.

[Oes96] Oesterling JE. Age-specific reference ranges for serum PSA. New Engl J Med 1996;335:345-6.

[Pan90] Pan HQ, Goldstein H, Yang Q. Non-parametric estimation of age-related centiles over wide age ranges. Ann Hum Biol 1990;17:475-81.

[Par91] Parvin CA, Gray DL, Kessler G. Influence of assay method differences on multiple of the median distributions: maternal serum alpha-fetoprotein as an example. Clin Chem 1991;37:637-42.

[Par96] Partin AW, Criley SR, Subong ENP, et al. Standard versus age-specific prostate specific antigen reference ranges among men with clinically localized prostate cancer: a pathological analysis. J Urol 1996;155:1336-9.

[Pea94] Pearson JD, Morrell CH, Landis PK, et al. Mixed-effects regression models for studying the natural history of prostate disease. Stat Med 1994;13:587-607.

[Pol99] Polascik TJ, Oesterling JE, Partin AW. Prostate specific antigen: a decade of discovery – what we have learned and where we are going. J Urol 1999;162:293-306.

[Pre96] Prestigiacomo AF, Stamey TA. Physiological variation of serum prostate specific antigen in the 4.0 to 10.0 ng/mL range in male volunteers. J Urol 1996;155:1977-80.

[Ran89] Ranke MB. Disease-specific growth charts – do we need them? Acta Paediatr Scand 1989;356(Suppl):17-25.

[Ran96] Randell EW, Diamandis EP, Ellis G. Serum prostate-specific antigen measured in children from birth to age 18 years. Clin Chem 1996;42:420-3.

[Ree71] Reed AH, Henry RJ, Mason WB. Influence of statistical method used on the resulting estimate of normal range. Clin Chem 1971;17:275-84.

[Rem99] Remaley AT, Sampson ML, DeLeo JM, et al. Prevalence-value-accuracy plots: a new method for comparing diagnostic tests based on misclassification costs. Clin Chem 1999;45:934-41.

[Res99] Resim S, Çek M, Gürbüz ZG, Fazlioglu A, et al. Serum PSA and age-specific reference ranges in patients with prostatism symptoms. Int Urol Nephrol 1999;31:221-8.

[Ric97] Richardson TD, Oesterling JE. Age-specific reference ranges for serum prostate-specific antigen. Urol Clin North Am 1997;24:339-51.

[Rip94] Ripley BD. Neural networks and related methods for classification. J R Statist Soc B 1994;56:409-56.

[Roc87] Rock RC, Chan DW, Bruzek DJ, et al. Evaluation of a monoclonal immunoradiometric assay for prostate-specific antigen. Clin Chem 1987;33:2257-61.

[Roe00] Roehrborn CG, McConnell J, Bonilla J, et al. Serum prostate specific antigen is a strong predictor of future prostate growth in men with benign prostatic hyperplasia. J Urol 2000;163:13-20.

[Ros79] Rossing RG, Hatcher WE. Percentiles as reference values for laboratory data. Am J Clin Pathol 1979;72:94-7.

[Roy98] Royston P. Reference intervals for normal clinical values. In: Armitage P, Colton T, editors. Encyclopedia of biostatistics. New York: John Wiley, 1998:3058-64.

[Sem96] Semjonow A, Brandt B, Oberpenning F, et al. Discordance of assay methods creates pitfalls for the interpretation of prostate-specific antigen values. Prostate Suppl 1996;7:3-16.

[Sem98a] Semjonow A, Weining C, Oberpenning F, et al. Application of assay-specific cut-off values can improve performance of PSA tests; results of the assay comparison study [abstract]. Tumour Biol 1998;19 Suppl 2:22.

[Sem98b] Semjonow A, et al. Pitfalls in the interpretation of prostate specific antigen levels. Proc UK NEQAS Meeting 1998;3:146-53.

[Sha99] Shapiro DE. The interpretation of diagnostic tests. Stat Methods Med Res 1999;8:113-34.

[Shi97] Shibata A, Whittemore AS, Imai K, et al. Serum levels of prostate-specific antigen among Japanese-American and native Japanese men. J Natl Cancer Inst 1997;89:1716-20.

[Shu85] Shultz EK, Willard KE, Rich SS, et al. Improved reference-interval estimation. Clin Chem 1985;31:1974-8.

[Sib99a] Sibley PEC, Sturgeon CM. Age-specific PSA reference ranges. Presentation to the Task Force of the Scientific Committee of the Association of Clinical Biochemists on the use of prostate-specific antigen in a screening program for prostate cancer; London; 1999 June.

[Sib99b] Sibley PEC. IMMULITE tumor marker assays: multicenter reference range data. Los Angeles: Diagnostic Products Corporation, 1999. Technical report ZB148-D. Available at DPC's Web site, www.dpcweb.com, under Technical Documents, Technical Reports.

[Sib99c] Sibley PEC. OM-MA (CA125) and ovarian cancer. News & Views (DPC) 1999 Summer;13(3):12-4. Technical report ZB195-A. Available at DPC's Web site, www.dpcweb.com, under Technical Documents, News & Views, Summer 1999.

[Sib99d] Sibley PEC. Tumor marker assays; the significance of normal range studies. News & Views (DPC) 1999 Fall;13(4):6-8. Available at DPC's Web site, www.dpcweb.com, under Technical Documents, News & Views, Fall 1999.

[Sib00] Sibley PEC. Reference range analysis; lessons from PSA. News & Views (DPC) 2000 Winter;14(1):9-12. Available at DPC's Web site, www.dpcweb.com, under Technical Documents, News & Views, Winter 2000.

[Smi93] Smith M. Neural networks for statistical modeling. New York: Van Nostrand Reinhold, 1993.

[Smi96] Smith DS, Bullock AD, Catalona WJ, Herschman JD. Racial differences in a prostate cancer screening study. J Urol 1996;156:1366-9.

[Sol83] Solberg HE. Inaccuracies in computer calculation of standard deviation. Anal Chem 1983;55:1611.

[Sol87] Solberg HE. International Federation of Clinical Chemistry (IFCC): Approved recommendation (1987) on the theory of reference values. Part 5. Statistical treatment of collected reference values. Determination of reference limits. Eur J Clin Chem Clin Biochem 1987;25:645-56. Also: Clin Chim Acta 1987;170:S13-S32.

[Sol89] Solberg HE, Graesbeck R. Reference values. Adv Clin Chem 1989;27:2-79.

[Sol95] Solberg HE. RefVal: a program implementing the recommendations of the International Federation of Clinical Chemistry on the statistical treatment of reference values. Comput Methods Programs Biomed 1995;48:247-56.

[Sol99] Solberg HE. Establishment and use of reference values. In: Burtis CA, Ashwood ER, editors. Tietz textbook of clinical chemistry. 3rd ed. Philadelphia: W. B. Saunders, 1999:336-56.

[Sta98] Stamey TA, Chen Z, Prestigiacomo AF. Reference material for PSA: the IFCC standardization study. Clin Biochem 1998;31:475-81.

[Sta00] Stamey TA, Yemoto CE. Examination of the 3 molecular forms of serum prostate specific antigen for distinguishing negative from positive biopsy: relationship to transition zone volume. J Urol 2000;163:119-26.

[Ste96] Stenman U-H, Leinonen J, Zhang W-M. Problems in the determination of prostate specific antigen. Eur J Clin Chem Clin Biochem 1996;34:735-40.

[Ste97] Stenman U-H. Prostate-specific antigen, clinical use and staging: an overview. Br J Urol 1997;79(Suppl 1):53-60. [Str86] Strike PW, Michaeloudis A, Green AJ. Standardizing clinical laboratory data for the development of transferable computer-based diagnostic programs. Clin Chem 1986;32:22-9.

[Str96] Strike PW. Measurement in laboratory medicine: a primer on control and interpretation. Oxford: Butterworth-Heinemann, 1996.

[Tsa79] Tsay J-Y, Chen I-W, Maxon HR, Heminger L. A statistical method for determining normal ranges from laboratory data including values below the minimum detectable value. Clin Chem 1979;25:2011-4.

[Ven99] Venables WN, Ripley BD. Modern applied statistics with S-PLUS. 3rd ed. New York: Springer-Verlag, 1999.

[Vie94] Vieira JGH, Nishida SK, Pereira AB, et al. Serum levels of prostate-specific antigen in normal boys throughout puberty. J Clin Endocrinol Metab 1994;78:1185-7.

[Wei98a] Wei JT, Zhang Z, Barnhill SD, et al. Understanding artificial neural networks and exploring their potential applications for the practicing urologist. Urology 1998;52:161-72.

[Wei98b] Weinrich MC, Jacobsen SJ, Weinrich SP, et al. Reference ranges for serum prostate-specific antigen in black and white men without cancer. Urology 1998;52:967-73.

[Wil97] Wilcox RR. Introduction to robust estimation and hypothesis testing. New York: Academic Press, 1997.

[Wri96] Wright EM, Royston P. Age-specific reference intervals (normal ranges). Stata Tech Bull 1996;34:24-34.

[Wri99] Wright EM, Royston P. Calculating reference intervals for laboratory measurements. Stat Methods Med Res 1999;8:93-112.