University of Rhode Island DigitalCommons@URI Open Access Master's Theses 1995 Evaluation of the Reliability of Serum Cholesterol Measurements Catherine Rhodes Allan University of Rhode Island Follow this and additional works at: https://digitalcommons.uri.edu/theses Recommended Citation Allan, Catherine Rhodes, "Evaluation of the Reliability of Serum Cholesterol Measurements" (1995). Open Access Master's Theses. Paper 1039. https://digitalcommons.uri.edu/theses/1039 This Thesis is brought to you for free and open access by DigitalCommons@URI. It has been accepted for inclusion in Open Access Master's Theses by an authorized administrator of DigitalCommons@URI. For more information, please contact digitalcommons@etal.uri.edu. EVALUATION OF THE RELIABlLITY OF SERUM CHOLESTEROL MEASUREMENTS BY CATHERINE RHODES ALLAN A THESlS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE lN STATISTICS UNIVERSITY OF RHODE ISLAND 1995 MASTER OF SCIENCE THESIS OF CATHERINE RHODES ALLAN APPROVED: Thesis Committee MajorProfessor /?,,. CA~IV'd IJM-~ ~5t~'~t::z-~/) ~ DEAN OF GRADUATE SCHOOL UNIVERSITY OF RHODE ISLAND 1995 Abstract Various statistical methods are utilized in the determination of reliable cholesterol measurements via the Reflotron desk-top analyzer. The analysis presented in this research concerns data from the Worksites, Occupational Nurses, and Cholesterol Change Program, a study consisting of volunteers at various worksites in Southern Massachusetts and Rhode Island. Accuracy and precision tests are utilized to determine reliability of cholesterol measurements. The initial lot testing of reagent strips gives estimates of bias resulting from Reflotron cholesterol measurement. Regression analysis techniques are employed to adjust for this bias. The accuracy testing failed to fall within the guidelines set by the National Cholesterol Education Program (bias<=5% ). A negative bias exists between the Reflotron and reference laboratory values when measuring total blood cholesterol. However, lots ofreagent strips were chosen to proceed to the next round of testing since resulting cholesterol measurements can be adjusted for bias. The second phase of testing involves analysis of experimental precision. The design of the experiment entails all apparent sources of variation inherent to the cholesterol measurement process by Reflotron determination. The precision testing was successful in meeting the NCEP guidelines (CV<=5%), and information concerning the various random components was collected. Estimates of variance components are utilized in the construction of tolerance intervals to be used for quality control procedures in the field experiment consisting of worksite-wide cholesterol screening. A standardized method of reporting accuracy termed the maximum percent error is introduced and employed to compare various trials of the experiment, along with the calculation of coefficients of variation. Approximate confidence intervals are constructed for each 11 variance component and for the overall variability in the experimental model. This gives a range of variability associated with the process, and an idea of which components contribute the greatest source of variation to the experimental model. The final calculations are for the adjustment of cholesterol measurements in the field experiment. The purpose is to utilize the quality control levels to determine the estimated true cholesterol values of participants in the study. Prediction intervals are then constructed to determine a range of values within which an individual's true cholesterol level falls . Thus, if misclassification of individual true cholesterol measurements occurs, the prediction interval gives information concerning the risk category to which an individual is likely to belong. Remedial measures are recommended for future testing. iii Acknowledgements There are many people who I would like to thank for their guidance and support in my graduate studies. As my major professor, I would like to thank Dr. R. Choudary Hanumara for his wisdom and patience in helping me successfully complete my degree. Others who I would like to thank include my remaining committee members, Drs. William Lawing, David Shao, and Cynthia Willey, and the persons from Pawtucket Heart Health Program instrumental in preparing this research, Dr. Tom Lasater for the use of the study data and Dr. Kate Lapane for her encouragement, enthusiasm, and sense of humor. Finally, I'd like to mention that none of this would have been possible without the continued support of my friends and family . lV Table of Contents Abstract Acknowledgements Table of Contents List of Tables List of Figures I. Introduction II . Accuracy Testing A. Percent Differences B . Simple Linear Regression HI . Precision Testing A. Design of Experiment B. Generation of Experimental Design C. Analysis of Experiment D . Computation of Tolerance Intervals IV. Maximum Percent Error V. Confidence Intervals on Variance Components A. Point Estimates of Individual Variance Components B . Construction of Confidence Intervals on Individual Variance Components C. Point Estimate of Total Variability D. Construction of Confidence Limits on Total Variability v 11 111 IV VI Vil 5 6 8 1 ] 11 14 17 20 24 29 30 31 34 36 VI. The Field Experiment 40 A The Two-Step Adjustment 42 B. Prediction Interval on Point Estimate 46 c. Recommendations for Next Time 49 VII. Summary 51 VIII . References 54 IX. Appendices 91 X. Bibliography I I 9 VI List of Tables Table 1 - Initial Lot Testing Results 56 Percent of Observations within NCEP Guidelines Table 2 - Initial Lot Testing Results 57 Simple Linear Regression Parameter Estimates Table 3 - Mean Cholesterol Value per Pool and Factor Levels & Values a. February, 1993 1st Lot 941 59 b. February, 1993 2nd Lot 941 60 c. February, 1993 1st Lot 942 61 d. February, 1993 2nd Lot 942 62 e. August, 1993 Lot 941 63 f. August, 1993 Lot 564 64 Table 4 - Estimation of Variance Components by Different Methods 65 Table 5 - 95% Tolerance Limits for Pool i of Sera 68 Table 6 - Maximum Percent Error & Coefficient of Variation 70 Table 7 - Analysis of Variance with Type III Sums of Squares 72 for Unbalanced Two-Fold Nested Mixed Model Table 8 - ANOV A with Type III Sums of Squares a. February, 1993 1st Lot 941 73 b. February, l 993 2nd Lot 941 74 c. February, 1993 1st Lot 942 75 d. February, 1993 2nd Lot 942 76 e. August, 1993 Lot 941 77 f. August, 1993 Lot 564 78 Table 9 - 95% Confidence Intervals on Variance Components a. February, 1993 1st Lot 941 79 b. February, 1993 2nd Lot 941 80 c. February, 1993 1st Lot 942 81 d. February, 1993 2nd Lot 942 82 e. August, 1993 Lot 941 83 f. August, 1993 Lot 564 84 Vll List of Figure.s Figure 1 - Initial Lot Testing, February, 1993 a. Bias of Lot 941 vs. Gold Standard b. Bias of Lot 942 vs. Gold Standard Figure 2 - Initial Lot Testing, August, 1993 a. Bias of Lot 941 vs. Gold Standard b. Bias of Lot 943 vs. Gold Standard Figure 3 - Initial Lot Testing, August, 1993 a. Bias of Lot 941 vs. Gold Standard b. Bias of Lot 564 vs. Gold Standard Vlll 85 86 87 88 89 90 I. INTRODUCTION Coronary heart disease (CHD) is the leading cause of death in the United States l _ Heightened awareness of the importance of cholesterol as a primary modifiable risk factor for heart disease has increased the number of individuals having their cholesterol levels measured. For each 1 % decreases in total blood cholesterol, the rate of coronary heart disease is reduced by 2%2. An estimated 36% of the population has hypercholesterolemia as defined by the National Cholesterol Education Program (NCEP). An estimated 41 % of American adults have cholesterol levels which justify further testing and possible treatment3 . When treated with diet modification and/or drug therapy, individuals with high blood cholesterol can decrease their risk of heart disease. The NCEP recommends that all persons over the age of twenty have their cholesterol levels measured at least once every five years, and those with high cholesterol (>240 mg/di) receive treatment to lower their cholesterol levels. In response, "simple" cholesterol testing procedures have become common with the development of portable desk-top analyzers which allow for inexpensive, fast, accurate measurements of serum cholesterol. Widespread public screening has become common place and allows for large numbers of individuals with high blood cholesterol levels to be identified and alerted. The Worksites, Occupational Nurses, and Cholesterol Change Program is a study affiliated with the Memorial Hospital in Pawtucket, Rhode Island designed to address many specific aims. One of the most important of these aims is to determine whether occupational nurses within a worksite (Internal) are more effective in lowering total blood cholesterol than technicians employed by an outside screening organization (External) to direct cholesterol screening, educate individuals about cholesterol, and conduct referral programs for individuals at risk. Another important goal is to determine if automated cholesterol assessment and feedback of suggested behavior changes (Rl) is as effective in reducing CHD modifiable risk factors as a face-to-face delivery of these recommendations in a brief I 0-12 minute counseling session (FF) . The following is an overview of the research design implemented in the Worksites, Occupational Nurses, and Cholesterol Change Program. Twenty-five worksites in Southern Massachusetts and Rhode Island with full-time occupational nurses were recruited and then assigned randomly to one of five experimental units, each condition consisting of five worksites (See Appendix A, Figure 1 ). Cholesterol measurements were taken and recorded for individuals within each worksite involved in the study to obtain baseline data. The cholesterol measurements were taken utilizing a Reflotron portable desk-top analyzer. A random sample of the employees within each worksite were tested at six-month and twelve-month follow-up . The 10 worksites which fall into the Internal condition employed the occupational nurse(s) within each worksite to conduct the cholesterol Screening, counseling, and Referral Events (SCOREs ). The 10 worksites within the External condition, however, had cholesterol SCOREs implemented by project staff, persons hired and trained for this research study. All persons involved in conducting SCOREs were trained in the same manner. Also, every employee participant received a nutrition kit, literature concerning various foods and their cholesterol contents. This should help individuals modify their eating habits to lower total blood cholesterol. One group of 5 worksites within the Internal condition (Internal-FF) and one group of 5 worksites within the External condition (External-FF) were given face-to-face cholesterol education and referral consultations with participants based upon individual risk factor analysis. These meetings take approximately 10-12 minutes. The risk factor analysis is based upon total blood cholesterol, blood pressure, self-reported eating patterns 2 (based on responses to a food frequency/dietary survey), and other cardiovascular disease (CVD) risk factor status (i .e. age, gender, smoking status, body mass index). In the remaining 5 Internal and 5 External condition worksites (Internal-RT & External-RI), each participant completed a questionnaire card which was then inserted into a RT2000 micro-computer optical scanner. The respondent then obtained a computer printout which contains at most 7 behavioral changes for lowering his/her risk of heart disease. These suggested changes are ordered from most to least important. The final conclusions drawn at the end of the study are based on the assumption that the cholesterol measurements taken on individuals are reliable. There are two components that determine the accuracy of measurements. These are precision and bias. Precision is defined as the closeness of repeated measurements on the same item. Bias is defined as the systematic error in repeated measurements, the difference between the true target value and the measurement value which can be corrected by calibration with standards. The goal of this work is to determine how accurate these cholesterol measurements are, and how they should be adjusted in the final stage so that they actually represent the true cholesterol values of individuals in the study. Much has been reported on the accuracy of Reflotron obtained cholesterol measurements in various experiments. Some experiments report reliable results were obtained with the Reflotron desk-top analyzer (within NCEP guidelines), while other experiments report the opposite. Hence, we need to know the process by which the cholesterol measurements are taken. The Reflotron is an electrochemical reflectance photometer that measures chromatic changes in blood which are produced by various chemical constituents. Reflectance is measured with a reference beam and an Ulbricht sphere located within the instrument. Two detectors in the optical module compare the emitted light from three 3 diodes with the reflected light from the reagent tab. The amount of reflected light is inversely proportional to the cholesterol concentration in the sample, and a signal processor transmits the associated voltage to a digital display4. The Reflotron system of cholesterol measurement works in the following manner. Approximately a 30-µL aliquot of human blood or sera is pipetted onto an application pad on a reagent strip. "The top layer of the glass-fiber pad retains erythrocytes when whole blood is used, allowing the plasma to diffuse to an adjacent reaction area. The reagent system is impregnated into the membranous layer on the strips ." When the reagent strip is inserted into the Reflotron machine, contact occurs between the reagent and sample layers which starts a chemical reaction. This reaction is given by the following : cholesterol esters + H20 cholesterol esterase cholesterol + RCOOH cholesterol + 0 2 cholesterol oxidase cholest-4-ene-3-one + H202 H202 + 3,3',5,5'-tetramethylbenzidine peroxidase oxidized benzidine (blue) + H20 It has been previously documented that variability exists between manufactured lots of reagent strips6. This is the reason for the initial reagent strip lot testing scheme. Since each lot of reagent strips is only ''good" for so long, each lot having a specific expiration date, testing is necessary to obtain a lot of reagent strips for worksite-wide cholesterol testing at baseline, 6-month follow-up, and 12-month follow-up . 4 II . ACCURACY TESTING In the Worksites, Occupational Nurses, and Cholesterol Change Program, the importance of reliable cholesterol measurements is imperative. When measuring an individual's cholesterol value by a Reflotron desk-top analyzer, many sources of variation are inherent to the process. For instance, no two lots ofreagent strips are identical. Viewed from a long-term perspective, these differences contribute to total random error6. The initial accuracy testing was performed in order to determine which lot of reagent strips give the best, most accurate results. This was done by determining what percentage of Reflotron-obtained values for each lot of reagent strips fell within specific guidelines set by the NCEP. The NCEP has set goals of reducing allowable bias to <=5%. However, the minimal guidelines utilized in this research are set at 14.2% and 8.9% of the target or true value. In this initial lot testing, approximately 20-30 volunteers are utilized. Each individual has his/her total blood cholesterol measured three times, twice by Reflotron and once by venous blood sample. The procedure for each individual volunteer is as follows . To obtain the two Reflotron-based cholesterol measurements, an individual has his/her finger lanced by a technician in order to produce a blood specimen. The technician captures the blood sample in a glass capillary tube and transfers it to a reagent strip from a specific lot of reagent strips. This reagent strip is then inserted into a Reflotron machine which gives a digital display of the individual's cholesterol value in 175 seconds. This value is then recorded by the technician. A second specimen of blood is taken, either from the same lance-induced wound or from a new one. It is then transferred in the same manner to a reagent strip from the second lot of reagent strips being tested and analyzed on a second Reflotron machine to obtain the second "finger-stick" based cholesterol value. 5 Note that already a second form of variability has been introduced into the system, machine-to-machine variability. To counter act this added variability, approximately the first half of the volunteers' first finger-stick values are obtained with reagent strip A in machine 1 and the second half of the volunteers' first finger-stick values are obtained with reagent strip A in machine 2. Likewise, the second finger-stick values are obtained for the first half of the volunteers with reagent strip B in machine 2 and for the second half of the volunteers with reagent strip B in machine 1. Finally, to obtain the third cholesterol value, a venous blood sample is drawn and analyzed at the Center for Disease Control standardized laboratory at Miriam Hospital. The results for an individual's cholesterol measurement via venous blood sample take approximately one to two weeks to obtain. The cholesterol values are taken as the true cholesterol value or "gold standard" for each person since the reliability of these estimates is great. The procedure for analysis is found in Appendix A, Figure 2. These gold standard measurements are made up of three components. Cholesterol values are divided into LDL cholesterol (bad), HDL cholesterol (good), and trigycerides. A. Percent Differences One can get an estimate of the accuracy of an individual lot of reagent strips by calculating the percent differences between the finger-stick and gold standard values. This provides a measure of the bias in Reflotron obtained results and gives a criteria in which to compare various lots of reagent strips. A detennination can be made as to what percent of cholesterol values fall within the minimal NCEP guidelines of 14.2% and 8.9% for each lot of reagent strips. This type of procedure is utilized first to get baseline data on the accuracy of a lot of reagent strips to detennine whether further reliability testing is warranted. Then, this same procedure is employed every time a new lot of reagent strips 6 is needed. Since the lots have only a 8-10 month shelf-life, new lots are necessary for baseline, 6-month follow-up, and 12-month follow-up cholesterol measurements at each worksite in the study. When protocol is followed and variability between experiments is kept minimal, reliable results can be obtained. For example, the first lot testing in February, 1993 consisted of 19 volunteers. The two initial lots of reagent strips compared were lot 941 and lot 942. The data for this initial lot testing is located in Appendix B. Scatter plots of the bias of Reflotron obtained cholesterol values versus the gold standard values can be found in Figures 1, 2, 3. For every experiment, the scatter plot implies a negative bias; the Reflotron results consistently underestimate the true cholesterol values. Let Y,1 (i= l ,2; j=l ,2, .. .,n) denote the ith fingerstick measurement (obtained with a reagent strip from a particular lot) on the jth volunteer, and let X 1 denote the gold standard value of the jth volunteer. In this example, let i= 1 if reagent strip from lot 941 and let i=2 if reagent strip from lot 942. To calculate the percent difference between the Reflotron obtained cholesterol values and the gold standard values, standardization is required as follows: Y -X V% = I) J *100 X 1 Results of accuracy for all initial lot testing are located in Table 1. 7 In this example, 100% of the Reflotron observations fall within± 14.2% of the gold standard values regardless of the lot of reagent strip utilized. However, lot 941 faired slightly better with 84.2% of its fingerstick values falling within ±8.g<>/o as opposed to lot 942 with 78 .9% of its fingerstick values falling within ±8.9%. Both of these lots of reagent strips are fairly accurate and warrant further testing. B. Simple Linear Regression Another method of determining the more accurate or "better" lot of reagent strips is to employ rudimentary regression techniques. This analysis is utilized later in this research when discussion turns to adjustment of fingerstick values in the worksite-wide cholesterol screening program. It is expected that the Reflotron values are exactly the same as the gold standard values if everything works perfectly. Thus, the relationship between the methods of cholesterol measurement should be: Yij = x j given no sources of variation. However, the measurements obtained from the fingerstick method are liable to many sources of variation, and the relationship between the Reflotron values and the gold standard values can be modeled as 7: It is expected that the intercept parameter Po is close to 0 and the slope parameter /3., is close to 1. The results ofregression analysis can be located in Table 2. 8 For example, in the initial lot testing of reagent strips, the .experiment included lots 941 and 942 . The results from the regression analysis are: Lot 941: Lot 942: Y = 5.62+0 .94X Y = -26.39+1.08X In the first lot testing, both lot 941 and lot 942 appear to be fairly accurate with parameters close to target values. Hence, both lots of reagent strips warrant further testing. Whichever fairs well in the next round of testing is to be used in the worksite-wide screening process. In August, 1993 a new lot of reagent strips is needed. The shelf-life oflot 941 (after all the testing, the chosen reagent strip lot used to collect baseline worksite-wide cholesterol values) is quickly approaching. The second lot testing experiment was run with reagent strips from lot 941 and lot 943 and consisted of 33 volunteers. The gold standard value of one individual is missing so calculations are based on a sample size of 32. Of those measured with reagent strips from lot 941 , 60.6% fell within ±8.9% and 75 .8% fell within± 14.2% of the true value. Lot 943 , unfortunately, failed miserably in comparison with only 27.3% within ±8.9% and 72 .7% within± 14.2%. The regression results restate this result . Lot 941 again has a slope close to 1 whereas lot 943 grossly underestimates the gold standard values. Thus, lot 943 is not acceptable, and no further testing of it is planned. Still searching for a new lot of reagent strips, the third initial lot testing is conducted with lot 941 and lot 564. The values obtained from lot 941 are again decreasing in accuracy. Now only 43 .3% of the fingerstick observations are within ±8.9% and 73 .3% within± 14.2%. Thankfully, lot 564 appears promising with 70.0% 9 within ±8.9% and 100% within± 14.2%. Again, the regression results support this finding, and Jot 564 moves on to the next round of testing (lot 564 is the lot chosen, after all reliability testing is performed, to collect 6- month follow-up worksite-wide cholesterol values) . Notice that the accuracy oflot 941 has decreased from the initial February, 1993 experiment to the August, 1993 experiment. This can possibly be attributed to the fact that the expiration date for lot 941 is approaching. Unfortunately, the rate at which accuracy has decreased is not known. The accuracy of the lot ofreagent strips may have delined steadily since the manufacturing date, or perhaps it fell sharply at some particular point in time. This aspect of reagent strip accuracy should be montiored further. The overall mean percent differences between the desktop results and the reference Jab results were not all within the established NCEP guidelines for accuracy (5 .0%). Only the February, 1993 lot testing of 941 and 942 meets the guidelines with -3 .6% and -5 .1 % respectively. Lot testing of 564 resulted in an overall mean percent difference of -6.5% which is outside the guidelines, yet it still appears to be a viable Jot of reagent strips. Finally, it is concluded that all lots of reagent strips result in negatively biased cholesterol values, estimates lower than the gold standard values. Thus, Reflotron obtained cholesterol measurements underestimate the true cholesterol value of an individual. How much each lot of reagent strips underestimates the true value varies. 10 III. PRECISION TESTING A Design of Experiment In the second stage of lot testing reagent strips, precision of the process is to be assessed. The goal of this testing is to determine how well this cholesterol screening identifies persons at risk of high cholesterol based on the measurement obtained from the Reflotron. The experiments are run in order to set tolerance limits for the purpose of quality control checks when performing mass cholesterol screening at worksites. There are several sources of variability associated with cholesterol measurement via the Reflotron desk-top analyzer. Sources of variability are any factors in the experiment that may cause repeated measurements on the same item to differ. In these experiments, frozen samples of sera are utilized here as opposed to an individual's whole blood. Each vial of frozen sera is actually a pool of approximately 30 different whole blood samples. Hence the exact cholesterol concentration is not known. However, the range in which each vial's cholesterol value should fall is known. The ranges are defined as high (>240 mg/dl), medium (200-239 mg/di), and low (<200 mg/di). The assigned technician extracts the sera sample from a specific tube with a glass capillary tube and transfers it onto a reagent strip from a specific vial of strips. The strip is then inserted into a designated Reflotron machine. Finally, the cholesterol measurement obtained is recorded. Let the pool the sera belongs to be factor P, the round of experimentation factor R, and the sample of sera from a specific tube factor S. Since each round is different for each pool of sera, factor R is nested within factor P. Also, factor S is nested within factor R and P since each sample of sera is unique for each round within each pool. Let the machine analyzing the sera sample be factor M and the technician conducting the experiment be factor T. Let the vial of reagent strips used to test the sera sample be factor 11 v. Lastly, let & denote the random error in the study. Tlus is the catch-all, everything else on explained by the model. Since there should be no significant interactions between factors, no crossed effects are in the model. The type of the effects are of importance in the model. They are either fixed or random. When deciding whether a set of effects is fixed or random, the question is that of inference and consequently how the levels are chosen8. If inferences are to be drawn from these data about only the levels of the factor in the experiment, then the effect in question is a.fixed effect. Otherwise, if inferences are to be drawn from these data about not only the levels of the factor represented in the experiment, but also those levels from an infinite set of factor levels, of which only a random sample occur in the data, then the effect is a random one. The factor P only has three possible levels. Sera samples fall into the following ranges: low (less than 200mg/dl), medium (200-239mg/dl), high (240 mg/di or above). Each of these values is considered in the experiment. P (pool) is a fixed effect. The round of experimentation within each pool has only two levels (1 ,2) in this testing procedure but the number oflevels could be infinite. We have no specific interest in the difference between the two rounds and any other rounds. Hence, R (round) is a random effect . Similarly, the samples of sera utilized within each round and pool are randomly taken from a population of samples. There is no interest in the difference between any of the specific sera samples and so S (sample) is a random effect. 12 M (machine) is a random effect . The machines utilized in the experiment are only a small, random subset of all possible Reflotron machines which could have been employed. The technicians conducting the experimental design are a random sample from a population of possible technicians. There is no specific interest in evaluating the difference between any one of the technicians in the experiment and any other of them. Thus, factor T (technician) is a random effect. V (vial of strips) is also a random effect. The vials of reagent strips within a specific lot used in the testing are a random subset of all possible vials of strips which could have been utilized. The model consists of a single fixed effect and five random effects. This is referred to as a mixed model. A single cholesterol measurement is taken for each treatment level combination, but each factor level does not contain the same number of observations. For example, in the August experiment of lot 94 I, machine #I and machine #2 were employed five different times while machine #3 was used six times. Thus, the model is said to be unbalance are the random error effects. Normality of random effects is assumed. Thus, we assume R j(i ) > sk(1J)> Mm • T n, VP , and eq(1jkmn ) are normally distributed with expectations zero and variances ~ , ifs , a!1 , a;. , a?,, , and 0-:, respectively. The variances associated with these random effects are also termed variance components and are analyzed in later evaluation in the calculation of tolerance limits, maximum percent errors, and confidence intervals for not only the variance components themselves but for the total variability of the model (observation). B. Generation of Experimental Design Using SAS proc plan, the experiment is designed as follows . Each time the experiment is to be run, the following SAS code is modified to produce the matrix of factor levels which will be available on the day of sera testing. 14 proc plan seed=3091966; factors tech=3 ordered round=2 ordered pool=3 cyclic sample=S ordered; output out=who tech cvals=('Lynn' 'Jen' 'Cat') random pool cvals=('High' 'Med' 'Low') random; factors tech=3 ordered round=2 ordered machine= 15 random; output out=which; factors tech=3 ordered round=2 ordered pool=3 cyclic strvial=S random; output out=what; data design; merge who which(keep=machine) where(keep=strvial); data final ; set design; by tech round pool notsorted; if first.tech then tube=O; if first. pool then tube+ 1; if first.tech then sequence=O; sequence+ 1 ; proc print; by tech notsorted; pageby tech; id sequence; var round tube pool sample machine strvial ; end; 15 Proc plan constructs designs and randomizes plans for nested and crossed experiments. Randomization is used both to neutralize the effect of any systematic biases that may be involved in the design as well as to provide a basis for the assumptions underlying the anaJysis. Proc plan generates designs by generating a selection of factor levels (for a given factor) for all combinations oflevels for factors which precede it 10. The proc plan statement is followed by the option seed which is a 5-,6-, 7-digit number used to start the pseudo-random number generator for random selection of factor levels. Selection of factor levels can be accomplished in three different manners for each factor in the design after the number of factor levels has been specified for every effect. 1. RANDOM selection -- levels are chosen in random order based on uniform pseudo-random order. 2. ORDERED selection -- levels are chosen in a standard order repeatedly. 3. CYCLIC selection -- levels are chosen by cyclically permuting the levels of the previous selection.10 After generating a design, it is stored in an output SAS data set using the output statement. Values are assigned to factor levels whether they be numerical (nvals) or character strings (cva/s) . Various generated plans are then merged together and a highly structured design is realized. The design matrix can then be reviewed and printed. Refer to Appendix C for the design matrices for aJl precision testing of reagent strips. The specific factor levels and values for each precision experiment designed are located in Table 3. Modifying the initiaJ seed vaJue results in different plans. A constant seed vaJue, however, generates the same plan repeatedly other things being equal. 16 The technjcians then run the experiment as designed following strict protocol. First the Reflotrons' optical eyes are checked to insure that the machines are reporting cholesterol values in the "correct" range. With a reagent strip from a lot of previously analyzed strips, a quality control (QC) check is performed. A sample of high sera and a sample oflow sera are measured separately. If both initial sera samples are within their respective 95% tolerance limits previously set, the machine is accepted and the optical eye cleaned. If either sample fails by falling outside of known 95% tolerance limits, another sample from the failed pool is measured. If the quality control check falls outside of the limits in the same direction, the malfunctioning Reflotron machine is replaced with a back-up. If the resulting value is within the limits, the experiment continues as mentioned above. Once all machines are calibrated, strip testing begins. Generally, for the design implemented, this testing process takes approximately 1-1. 5 hours to complete. Cholesterol values are digitally displayed by the Reflotron 175 seconds after insertion of the specified sera sample on the specified reagent strip and recorded by the selected technician. At the termination of the experiment, the optical eye in each machine is cleaned once again. After witnessing a precision experiment, I have a recommendation to make. The technicians transfer the sera sample to ·a reagent strip with a glass capillary tube. Each technician has his/her own vial of capillary tubes to use. However, since one technician emptied his/her vial of capillary tubes before the completion of the experiment, he/she "borrowed" some from another technician. Each of these vials of capillary tubes has an expiration date. This suggests that the vial of capillary tubes used could quite possibly be an additional source of variation and should be added to the experimental model. C. Analysis of Experiment 17 After inputting the results of the precision experiment and merging these results with the generated experimental design, the following SAS code is run to generate the appropriate ANOV A table (Table 7), calculate expected mean squares (Table 7), estimate the factor-level means of the fixed effect (Table 3) and estimate the variance components (Table 4). proc glm; class pool round sample machine tech strvial; model chol =pool round(pool) sample(round pool) machine tech strvial; random round(pool) sample(round pool) machine tech strvial I test; lsmeans pool; proc varcomp method=reml; class pool round sample machine tech strvial; model chol = pool round(pool) sample(round pool) machine tech strvial I fixed= 1; The variables are defined as follows: pool = low, medium, high round(pool) = 2 rounds within each pool sample(round pool)= 5 samples of blood sera in each tube for each round within each pool machine = Reflotrons used tech = 3 technicians conducting experiment strvial = 5 vials of reagent strips from lot being measured chol = cholesterol value obtained from the Reflotron, our dependent variable Variance components can be estimated in several ways. The four common methods of variance component estimation arel 1: 1. ANOV A Estimation (TYPE 1) 2. Minimum Variance Unbiased Estimation (MINQUE or MIVQUE(O) 3. Maximum Likelihood Estimation (ML) 18 4. Restricted Maximum Likelihood Estimation (REML) ANOV A estimation is the easiest to compute relative to the other methods mentioned, but there are inherent disadvantages. Not only are negative estimates possible, but the estimates lack distributional properties. And, there is no useful way to compare different applications of ANOV A methodology. The second method MJVQUE(O) gives the minimized generalized variance, and no distributional properties of the random effects are needed. However, estimators obtained by this method "are functions of a priori values used in place of the variance components in the estimation procedure itself' and this procedure "has what we deem to be a serious deficiency: the minimality property applies only at those a priori values" 11 . Maximum Likelihood estimation of variance components is based upon normality assumptions. The random effects have means and variances defined and are distributed normally. Calculating ML estimates involves iterative calculations which, before the existence of computers and computing packages, were difficult if not impossible to compute. The final method of variance component estimation is REML. Restricted Maximum Likelihood is ML on certain linear combinations of the data rather than on the data themselves. The idea behind it is to maximize that part of the likelihood which is invariant to the fixed effects in the model. One advantage to REML is that it "takes account of the implicit degrees of freedom associated with the fixed effects, whereas ML does not" 11 _ 19 In this study, REML is the method of choice when computing variance component estimates. Negative estimates are not a problem when utilizing REML, and it seems to give estimates in keeping with those obtained from the other methods available. Refer to Table 4 for variance component estimates computed by each of the afore mentioned methods for each of the precision experiments run. D. Computation of Tolerance Intervals The results of the precision testing are used to help maintain reliable cholesterol screening results in the field . When testing begins at worksites, certain quality control protocol must be followed. To determine if a Reflotron machine is suitable for usage in the field experiment or requires adjustment, a sera sample from each of the high and low pool will be tested. If the corresponding cholesterol values are within the allowable ranges, the machine is acceptable. These allowable ranges of variation are termed tolerance intervals. Tolerance limits are derived using REML variance estimates from the experiments designed to partition the variance into the following components: Round, Sample, Machine, Technician, Strip Vial (added after the February lot testing), Measurement Error. For each pool of sera, the (1- a)% tolerance limits are constructed for the February lot testing as 12: and for the August and subsequent lot testing as: P; ±Za12 ~o; +~+~ + cl,.+ clv + ci; 20 where P, is the overall mean cholesterol value for pool i, i=High, Medium, Low (Table 3). See Table 5 for 95% tolerance limits for each of the experiments run. To compute tolerance limits for lot 941 and lot 942 in the February, 1993 precision testing case, the tolerance limits were derived using pooled variance components for the two experiments per lot. Since the technician variation decreased dramatically from the first to the second run of the lot 941 and lot 942 experiments, the technician variance component, however, was estimated based on the results from the first run of the experiment to be conservative(*). For example, the computations for a 95% tolerance interval on lot 941 is actually based upon the following estimates of the variance components. Random Effect: Round Sample Machine Technician Error Total * Estimate: 4.88 0 1.89 9.50 21.42 37.69 The overall mean cholesterol level for each pool of sera, P, , were taken from the 1st lot 941 testing which resulted in the larger of the estimates. Since it is known that the reagent lots produce cholesterol measurements lower than the actual cholesterol measurement, the P; from the 1st testing oflot 941 are considered the most appropriate. Hence, the 95% tolerance limits for lot 941 for each pool are: High: Medium: Low: 239.75 ±1.96J37.69 ~(227,252) 214.86±1. 96J37.69 ~(202, 227) 157.28 ± 1.96J37.69 ~(145, 170) 21 These are the tolerance limits used for quality control checks in the field experiment when using reagent strips from lot 941 to conduct the worksite-wide cholesterol screening. Since the Reflotron displays cholesterol values as whole numbers, the lower tolerance limit has been rounded down to the next whole number, and the upper tolerance limit has been rounded up to the next whole number. In the February, 1993 precision testing, the technician component is significant in 3 of the 4 experiments run. When the random effect for vial of strips is added as a source of variation in the August, 1993 precision testing, this variance component is now significantly large, and the technician component is not. This suggests that perhaps strip vial and technician were confounded previously. The tolerance intervals computed from this experiment are wider than those from the February, 1993 experiment. Recall that the shelf-life oflot 941 is quickly approaching the expiration date. In the lot 941 testing, this random strip vial component is estimated to be more than three times greater than the measurement error. This gives strong indication of variability within a specific lot of reagent strips. However, this problem could be due to time since lot 941 is closing in on its expiration date. Unfortunately, since the component was added to the experimental model late, the information regarding strip vial precision over time is not available. This problem, though, is one of quality control for the producer of the reagent strips. It is an aspect of the experiment which must be monitored. Action should be taken if precision continues to decreases. Since this effect accounts for most of the variability in the model and is not a controllable factor , the overall experiment seems to run well. Lastly, the precision testing oflot 564 results in the random effect of vial of strips as the largest component of variability. This time, however, the magnitude is approximately half that of the measurement error. Since lot 564 is relatively new, this is further evidence that vials of reagent strips within the same lot decrease in precision over 22 time as the lot approaches its expiration date. Since lot 564 will be tested again when choosing a lot of reagent strips for 12-month follow-up, attention should be paid to the variance component estimates, in particular that of strip vial . All other variance estimates are negligible. Lot 564 is the chosen lot of reagent strips for the 6 month follow-up field experiment, and the tolerance intervals for each pool of sera are used when running quality control checks during the 6-month follow-up . 23 IV. MAXIMUM PERCENT ERROR One standardized procedure for reporting and monitoring the accuracy of cholesterol measurements is the maximum percent error 13' 14. The maximum percent error is an easily understandable measure of laboratory precision which incorporates all sources of variability in the model. An advantage of this procedure is that maximum percent errors from one experimental run to another can be compared, all things being equal. Since the primary concern in this part of the overall investigation is how well the cholesterol reading obtained by the Reflotron reflects the true cholesterol value, the variance of a particular cholesterol value is modeled. For a particular pool of sera: Yi = LLLYi(;hnnpq )jJKN J I: n where JKN denotes partially the replication scheme in which J denotes the number of rounds of testing, K denotes the number of samples of sera per round, and N denotes the number of technicians involved in the testing. For the February, 1993 experiments, for a particular pool i=l, J K N Yi (1kmnq ) = LL.~)~ +Rj(I ) +Sk(lj) + Mm +T,, +&l(;kmnq ) ) j =I k=I n= I Recall that after the initial precision testing in February, the experiment was modified to include the effect strip vial. Hence for the August, 1993 and subsequent experiments, for a particular pool i= 1, 24 J K N yl (jkmnpq ) = LLLU~ + R j (I) + s k(lj ) +Mm+ Tn + VP + 6i ukmnpq ) ) j =I k=I n=I The variance components of each random effect can be derived for a particular pool j_ Then the maximum percent error can be computed for the average cholesterol value for each pool of frozen sera by using the estimated variance components of the random effects in the model. Again, the variance component estimation method chosen is based on restricted maximum likelihood. If we assume that Yi is approximately normally distributed, a measure of accuracy termed the maximum percent errorl 3 is given by (for the February, 1993 lot testing): and (for the August, 1993 lot testing) : where c1,c2 ,c3 ,c4 ,c5 ,c6 are constants dependent on the number oftimes each particular factor level is employed in the experiment. They are derived seperately for each experiment since the factor levels of each effect in the model are subject to change. P; is the overall mean cholesterol value for pool i in the experiment (see Table 4). These Pi are also utilized in the adjustment of cholesterol values from worksites to be discussed in Chapter VI. This implies that the cholesterol measurement Yi will be within 100* P(a)% 25 of the true value of the sample mean with probability 1 - a 14. This maximum percent error can be used to describe the precision of readings already made, and comparisons of experiments is possible. From this we can also find the coefficient of variation of Y; as: CV(Y;) = P(a)/Za12 For example, in the August, 1993 precision experiment of Lot 941 , the three different pools of sera were measured in two rounds (J=2) with five samples per round (K=5) on a randomly assigned machine (M=l 5) by a particular technician (N=3) with a reagent strip from a randomly assigned vial of strips (P=5). The variance of each effect can be derived. J K N J J V(LLLR;(I) ) = V(:LI5R;(I))= 225* V(LR1(1) = 225* 2~ = 450~ j=I k=I n=I j=I ; =I JKN JK JK V(LLLSk(IJ)) = V(LL3Sk(IJ) ) = 9* V(LLSk(IJ) ) = 9 *(5* 2)~ = 90~ j=I k=I n=I j=I k=I j=I k=I J K N V(LLLMm)= V(a1M1 +a.µ 2 + ... +a,,,Mm)=a( a;.1 +a;a; ,+ ... +a!a-;1 =l ~ dM +1;a;,1 +. + 1 ~ a;1 =84a;1 J=I h i n=I where a1 +a2+ ... +am = 30 , a,.,= the number of times machine w is utilized in the experiment, and m= 1, 2, . . . , 15. J K N N N V(LLLTJ= V(L5* 2Tn) = lOO*V(LTJ = 100 * 3a; = 3ooa; J=I k=I n=I n=I n=I 26 J J: N / - 2_:;. b 2' b 2 -2_ 2 -2 2-2 +62 -2_36*5 ' -}80-2 V(LLLVr) = V(br1 + bF 2 + .. +bri r) - b1 '-'v + 2 o;:. + ... + rav - 6 av + 6 av +.. . av - ~ - av 1=t ,t=I n=l where bz = 6 =the number of times vial of reagent strips z is utilized in the experiment and p=l , 2, . . . , 5. J K N V(LL~:C·i;kmnpq) = 30a; j =I k=I n=I Thus, this implies that the variance of Y; is given by: which in turn gives the following equation for the maximum percent error: P ( ) - [ dR ds 7 d,.1 ~ o?, a; ]112 /p a -Z - + - +--+- +-+- . a/ Z 2 10 75 3 5 30 I After choosing the level of significance, the maximum percent error can be computed for a particular level of the pool effect . For example, the maximum percent error of the HJGH pool in the August, 1993 lot testing is: P(.05) =I. 96[ _2. + _Q_ + 7(2. 00268) + _2. + 40. 61229 + 13. 93 725]112/226. 04042 = 0.0257 oc. 2.57% 2 10 75 3 5 30 With 95% probability, the summary measurement for a pool will be within 2.57% of the true value. Note that the variance contribution due to vial of strips is large in comparison with all of the other random effects. And the coefficient of variation is: 27 CV= P( . 05)/za/~ = 0.0257/1 .96=0.0131 oc 1.31% Similarly, for all levels of the pool effect and each experiment run, the variance components of each random effect can be derived, the maximum percent error calculated, and the coefficient of variation computed. For the maximum percent errors and coefficients of variation for each experiment run, refer to Table 5. For each experiment run, the maximum percent error for each pool of sera is small (less than 4%). The results from the testing were well within the allowable ranges for coefficient of variation ( cv<5%) set by the NCEP. When comparing the August, 1993 experiments, it is noteworthy to mention that Jot 564 has maximum percent errors about half those of lot 941 . This is ultimately due to the magnitude of the strip vial effect. 28 V. CONFIDENCE INTERVALS ON VARIANCE COMPONENTS Thus far, point estimates of the variance components in the experimental model have been computed and analyzed. Point estimation entails selection of an estimate based upon a sample which "best" represents the parameter of interest. "Best" means that in some sense the estimate is close to the value of the unknown parameter 15. Sometimes this point estimate is close to the target value and sometimes it is not close depending on the sample utilized for estimation. This is because the sample values are values of random variables. Sometimes a "good" sample is taken, one that exemplifies the population of interest, and sometimes a "bad" sample is taken. Hence, sampling error is to be anticipated. Not only is a point estimate necessary, but information as to the accuracy of the estimate is needed. This is accomplished by utilizing confidence intervals. Burdick & Graybill consider interval estimation to be more informative than point estimation because in most studies, it is generally not enough to obtain a single value for the parameter under investigation. And, it is usually preferable to have some confidence that specified limits (upper and lower bounds) include the unknown value of the parameter. Also, confidence intervals can be used to test hypotheses. They argue that "confidence intervals are the most informative summary results of statistical inference and should be used whenever possible" 15 . · When dealing with the experimental model, not only does the interest lie in determining the magnitude of each variance component in the model, but also in the magnitude of the overall variance of the response variable. So, utilizing methods of constructing confidence intervals from Burdick & Graybill, confidence intervals were computed for each variance component in the model (a; , ifs , du , d;. , Of, , d!) and for the 29 total variance of the response variable ( 8 = var(Yvkmnpq )). The general procedure for computation is as follows . Each expected mean square is a linear function of the variance components. Setting confidence intervals on these variance components is accomplished by setting confidence intervals on linear combinations of the expected mean squares. Exact and approximate confidence intervals have been developed. When computing confidence intervals on each variance component, the representative model was the unbalanced two-fold nested random model. (See Table 7 for analysis of variance model). When the sums of squares are not independently distributed chi-squared which happens to be most of the time (all effects except the error) with this type of design, the calculations required to compute the confidence intervals become lengthy and involved. Unweighted sums of squares are utilized in constructing confidence intervals and, although the assumption that the mean squares are independent chi-square random variables is not generally met, this approach has worked well 15 . The resulting confidence intervals generally perform well and maintain the stated level of confidence. A. Point Estimates of Individual Variance Components A point estimate for each random effect is computed using the unweighted mean squares (MS) and expected mean squares (EMS) found in the analysis of variance table pertaining to this experimental design (Table 7). The resulting ANOVA tables for each precision experiment run were generated utilizing SAS proc glm (Table 8)11. These point estimates of each variance component are: Round · ~ = ({}, -(w 3u) * (). + (w 3u -1) * B >)w · R lu 2u 6 4u w 5u w 5u 30 Error: d! = 86 where equ is the expected mean square of the q th variance component ( q= 1, ... , Q) which is estimated by s:u, the mean square of the qth variance component. The formula for aj is found by algebraically manipulating the equations for the EMS and solving for the jth random effect. Any negative point estimate is defined to be zero. B. Construction of Confidence Intervals on Individual Variance Components Computation begins with the first random effect in the experimental model. Since the first random effect (round) has a second random effect (sample) nested within it, the formula for the computation of a confidence interval is somewhat more involved than the rest. Hernandez et al 16 recommended using unweighted mean squares when constructing this confidence interval. This approximate two-sided 1- 2a confidence interval on a; is given by: 31 where w C ---1.!L 2 -w 5u G1 = 1 - 1/f~. n, ,or:i for l = l,2,6 H 1 = 1/ F"i-a,n,."' - 1 for I = 1, 2, 6 and for I= 2,6 32 for l = 2,6 and S1~ ,S~u , S~ are the Type Ill unweighted mean sum of squares generated by SAS (Tables 7 and 8). For confidence intervals on the other random effects in the experimental model, the calculations are less cumbersome since no factor in the model has another factor nested within it. Again, these are only approximate confidence intervals. For random effect k, an approximate 1- 2a confidence interval on a; can be constructed as follows: w (k +3)u where 33 H 2s-+ G2S4 H c2. The lower bound on an upper I - a confidence interval on 8 is: [ L, oo ) where P Q P Q P - 1 P VL = L,c:c~s; + L. H;c;s: + L. L,Gqrcqcrs:s; + L,L,c;1cqc1s:s12 q=l r =P +l q=l r=P+l q=l t>q Since Q=P, this equation simplifies to: P P - 1 P VL = L,c:c:s; + L,L,c;1cqc1s:s12 q=l q=l t>q 36 where G = 1-11 Fa.n 00 for q = 1, ... ,P q '/ q • (n + n )2 G 2 n G 2 ) G·,=((1-1 1Fa.n+n oo )2 * q t _ __!l_!f_ _~) (P-1) q '/ · 9 ' ' n n n 11 . q t t q (t=q+l, ... ,P) The upper bound on a lower 1- a confidence interval on 8 is: [O,l! ] where, p Q p Q Q - 1 Q Vu = L.H:c:s; + L G;c;s: + L L.Hq,cqc,s:s; + L L.H:c,c,.s;s~ q=I r=P+I q=I r=P+I r= P+I u>r Since Q=P, this equation simplifies to: p Vu = "LH:c:s; q=I where H, =l/F..-a,n,,oo -1 for /=1,2, 3,4,5,6 37 for results of point estimation and confidence limits, refer to Table 9. The random effects for technician and measurement error prove to have the largest magnitude relative to the other variance components in half of the experiments, all performed in February. In turn, ifr has the widest confidence intervals. Also, the round of experimentation is a modest fraction of the variability in the model. In the August experiments, the lot testing of941 results in the newly added strip vial effect being 2.5 times the magnitude of the measurement error which greatly inflates the confidence range. Again, it is important to note that the technician component is not significant, suggesting possible confounding between strip vial and technician in the earlier experiments. The machine and strip vial effects have the widest intervals in the lot testing of 564. However, the lower bounds on the individual confidence intervals are almost and less than zero, respectively. ln summary, the experimental design utilized in this study can perfom1 well wi th reasonably minimal variability if protocol is followed . Since the strip vial effect is so great in August testing oflot 941 but not unreasonable large in lot 564, perhaps the effect is dependent on time. It seems reasonable to suggest that the older the lot of reagent strips, perhaps the greater the variability in vials of reagent strips within the lot. We expect confindence intervals on variance components to be skewed, since they are based on the F -distribution which is not symmetric. However, the individual point estimates appear to be much closer to the lower bounds of the 95% confidence intervals with some upper bounds being unreasonably large. This happens when the magnitude of the mean square of the variance component in question is large in comparison to the measurement error. Also, the fewer the degrees of freedom associated with the variance component in question, the larger the constants utilized in calculating these upper bounds. 38 Jn the February experiments, recall that the technician error is significant in 3 of the 4 experiments run . This suggests that protocol was not followed properly, and/or there is another source of variation not yet identified that attributes to this technician component, and hence resulting in the large upper bound for it. In the August experiments, the strip vial component proves to be the most variable. The technician variation in these experiments is negligible. Perhaps the technicians are more experienced with the cholesterol measurement process. Or, this new source of variation was confounding the technician variability. From these August experiments, it is clear that vial-to-vial variation within a lot of reagent strips is significant. However, when comparing the two experiments run, something else is evident. The new lot 564 has a strip vial estimate of reasonable size. Lot 941 does not. The resulting confidence interval is large. This may be partially attributed to the fact that lot 564 is close to its manufacturing date while lot 941 is approaching its expiration date. As the lot of reagent strips ages, the variability between vials of strips increases. Since no experiments were run between February and August, it is not known how quickly or at what rate the lots of reagent strips deteriorate. 39 VI. THE FIELD EXPERIMENT The final stage of this research centers around cholesterol screening at worksites in the study. Cholesterol measurement at a particular worksite takes a few days to complete. The types of worksites recruited for this study are primarily manufacturing and distribution companies with 250 to 800 employees that employ at least one full-time occupational nurse. All worksites chosen are within sixty miles of Memorial Hospital in Pawtucket, Rhode Island. So, now that a worksite is recruited, volunteers must be recruited to participate in the study. It has been found that a free cholesterol screening is very motivating. The volunteers participating in this experiment are required to meet certain criteria. They must be able to understand and sign an informed consent form. They must be at least 20 years old and not pregnant. They must not have suffered a stroke, heart attack, or surgery in the previous 2 months. They must be able to read and speak English. Lastly, they must plan to be employed at the worksite for the next year, making them eligible for follow-up study. Since the 6 month follow-up occurs at a different time of year than the baseline, the possible seasonal variation of cholesterol values was of concern. Total cholesterol levels have been documented as lower during the summer and higher during the winter, presumedly reflecting changes in dietary intake, physical activity, body weight, and other factors. This lead to the selection of September, October, November, and March, April, May as the two blocks of time that were ideal for baseline, 6-month and 12-month follow-ups. 40 Individuals meeting the afore mentioned criteria first fill out a participant registration form. Then the technician running the experiment takes a blood specimen from the individual and obtains a cholesterol value for him/her utilizing the fingerstick method. Depending on the experimental condition the worksite belongs to, appropriate action is taken regarding the individual's cholesterol level (Appendix A, Figure 1 ). The specific machine and the specific technician utilized are recorded. Since the participants are assigned randomly, it is possible to monitor the cholesterol values being produced by any specific technician or machine, identify measurement problems not found by the daily quality control procedures and take remedial action. The proposed daily quality control protocol to be employed in the field is the following. At the beginning and end of every day that a Reflotron machine is used the technician must : 1. Test the first sample of sera (HIGH or LOW) 2. If the first sample falls within the 95% tolerance range (for the specific lot of reagent strips being used), document the value and proceed with the scheduled screenings. 3. If the first sample is outside of the 95% tolerance range, test another sera sample (from the same pool) . 4. If the second sample is outside of the range IN THE SAME DIRECTION, reject the machine. Begin procedure again with another Reflotron desktop analyzer. The rejected machine is eliminated from further use until it has been repaired. Using this protocol, it is expected that a machine would be falsely rejected 1 out of 800 times. The overall Type I error rate was calculated as 0.025*0.025*2=0.00125 . This type of quality control check is suggested to be made periodically throughout the day while cholesterol testing is in progress (after every 20th measurement). This process of QC 41 measurement has been documented in some other cholesterol screening experiments 18, 19,20_ Hopefully, a malfunctioning machine is found before too many observations are taken. These Reflotron obtained cholesterol measurements must be adjusted since it has been shown previously that each specific lot of reagent strips underestimates the individual's true cholesterol value to some degree. Thus, for each day of testing at a worksite with a particular machine utilizing a specific lot of reagent strips, median high and low sera cholesterol values are computed for the calibration of individual cholesterol measurements. For the Baseline measurements, lot 941 was the lot of reagent strips utilized. It fared well in both the initial lot testing and the precision testing, and, hence, became the lot of choice. The data resulting from this worksite screening consists of the following variables of interest : 1. FORM NUMBER - an individual's _identifying number. 2. REFLOTRON MACHINE IDENTIFIER - number used to identify the machine used. 3. LOCATION CODE - identifies the worksite the individual belongs to . 4. DATE OF TEST 5. TECHNICIAN CODE - initials of technician who performed the test. 6. LOW SERA VALUE - quality control check value. 7. HIGH SERA VALUE - quality control check value. 8. COMMENTS - any comments the technician took note of A. The Two-Step Adjustment 42 The Reflotron obtained cholesterol value given to an individual participant is representative of his/her true cholesterol value. These Reflotron obtained cholesterol values must be adjusted or calibrated to reflect the actual or true cholesterol value. This is sometimes termed inverse estimation. The object is to transform the fingerstick value to a gold standard value. This results in values used to test hypotheses concerning total blood cholesterol on a worksite-wide basis. The two-step adjustment procedure discussed in this section is suggested by the research staff associated with this project at the Memorial Hospital . The first step in the adjustment of cholesterol measurements is to make a daily calibration as follows. For each individual cholesterol measurement taken, certain information is recorded about the machine and technician involved. For this particular machine and technjcian on a specific day are recorded quality control check values. The median values for the high and low sera QC checks (Pm,) are calculated. Now, these mediim values are linked to the overall mean cholesterol values for pool high and pool low (P,) in the precision lot testing for the particular lot of reagent strips used. Regressing the daily median high and low QC checks on the overall high and low mean cholesterol values gives the following equation: ·Z-a +aY - 0 I where Z is the observed cholesterol value measured on a particular day at a specified worksite, and Y is the observed cholesterol measurement adjusted for time. Let z0 be the observed of Z . A point estimate of the corresponding value of Y is given by 7: 43 Now, an adjustment must be made to transform a Reflotron obtained cholesterol value into a true cholesterol value. The relationship between fingerstick obtained cholesterol values measured by a specific lot of reagent strips and gold standard values was determined in Chapter II - Accuracy Testing as: where Y is the Reflotron obtained cholesterol measurement adjusted for time, and X is the true cholesterol value. Since y 0 is the observed value of Y , a point estimate of the corresponding value of X is given by: This approach is often called the classical estimator. x0 is a biased estimator of x0 which has an infinite mean square 7. The following example illustrates this two-step adjustment procedure. Suppose "Joe Smith" had his cholesterol level measured at his worksite on March 17. 1993 . The technician in charge used Reflotron machine "C" with a reagent strip from lot 941 . The resulting fingerstick value obtained is 230mg/dl. This is not Joe Smith's true cholesterol value. It is necessary to make a daily calibration of cholesterol values. To do this, one must first obtain the median quality control sera values per pool for the particular day of 44 testing on the specified Reflotron machine. The following are the quality control checks made on March 17, 1993 and the median values for the high and low sera quality control checks .. Low Sera 148 Hig_h Sera 240 174 234 Pm/ = 156.5 Pmh = 235 157 156 235 Notice that there is one more low sera check than high sera check. This is because the low sera check value of 174mg/dl fell outside of the 95% tolerance interval for the low pool. The technician was following protocol by checking another low sera sample to determine if the machine should be rejected. However, the next sera sample resulted in a cholesterol value within the specified range so testing continued with machine "C". Next, Z (Pmz and Pmh) are regressed on Y (P1 and Ph), the overall mean cholesterol values per pool from the precision testing (Table 3). The resulting equation is: pl = 157.28 Ph = 239.75 Z = 6.791257 +0.951861Y 45 and Joe Smith's Reflotron obtained cholesterol value z0 =230 after adjustment for day-to-day variation is: = zo - a o = 230-6.791257 = 234.50 Yo a1 0.951861 The second step of the adjustment procedure will transform Joe Smith's adjusted Reflotron obtained cholesterol value to a gold standard value. It is known from the initial lot testing of lot 941 that the Reflotron value (Y) is related to the true cholesterol value (X) by the following regression equation (Table 2). Y = 5.624063+0.936384X Taking y 0 =234 .50, Joe Smith's estimated true cholesterol value is: x =Yo::- /30 = 234.50-5.624063 = 244.43 0 /31 0.936384 Hence, Joe smith obtained a cholesterol value of230mg/dl from the worksite screening which places him in the moderate risk category. However, his estimated true cholesterol level is more than 14mg/dl higher. Joe Smith actually falls into the high cholesterol risk category. Since the study focus is not on the individual participant, this estimate is acceptable. Unfortunately, at the time of experiment, Joe Smith is given only his fingerstick cholesterol value which minimized his risk. B. Prediction Interval on Point Estimate 46 It has been discussed previously that point estimates may not contain enough information. The accuracy of the estimate is needed. This can be accomplished by computing prediction intervals for the point estimate. In this case, calibration intervals are appropriate. Montgomery and Peck 7 recommend testing the hypothesis Ho: A =0 If this hypothesis is not rejected, then no linear relationship exists between y and x rendering a point estimate or a confidence interval meaningless. The width of the prediction interval on x0 depends on how greatly Ho: {J1 = 0 is rejected. A short interval is obtainable if the hypothesis is rejected with a large F -statistic. And, a wide interval will result from a small F-statistic. However, this hypothesis was rejected for lot 941 (Table 2) . The 100(1 - a)% prediction interval for x0 is7,21: where d1 and d 2 are the roots of After some algebraic manipulation, the roots of the above equation are found to be22,23 : _I_[( - -) + t~12 .>J-2 (y2 I 2 Yo Y - A - c p I 47 This gives an individual observation fonnula, applicable to a single new cholesterol measurement. If we replace (1 + _!_) with ( _!_ + _!_) , where q is the number of observations n q n x is based on, the calculations are now based on true mean value calculations which 0 enables the generalization to the multiple predictor case. 21 To continue with the Joe Smith example, it is desirable to place a 95% prediction interval on x0 =244.43 , his estimated true cholesterol value corresponding to an adjusted Reflotron obtained cholesterol measurement of 234.50. This gives a range of values to be concerned with. For the data involved (Appendix B, Figure 1 ), S = = 39268.63, S y:.·= 35898.10, S xy = 36770.53 . Recall that p1 = 0.936384 and p0= 5.624063 . It follows that SSE = 1466. 77. The estimate of a 2 = 86.280319 is based on (n - 2) = 17 degrees of freedom. For a =.05, t a/i.17 = 2.093 . c2 = (2.093)2 (86.28) =.OllO (. 936384)2 (39268.63) and 1- c2=.9890. Using x0 = 244.43 , the upper and lower prediction limits for x when y=234.50 are as follows: (x x ) = 202. 58 +-1-[41.85 ± 2.093(9.29) (20)(. 9890) + (41.85)2 ] L> U .9890 0.9363841 19 39268.63 (XL,XU) = (223.0,266.8) 48 Joe Smith should be concerned with his cholesterol level. The prediction interval for Joe Smith's estimated true cholesterol value has a range spanning the moderate and high risk category. Since there is a very strong linear relationship between Reflotron obtained and gold standard values, the prediction interval is small and contains information a point estimate does not. C. Recommendations for Next Time The two-step adjustment procedure previously explained is the adjustment chosen to be utilized with the field experiment data. It is based on the quality control sera checks taken daily at a worksite. The protocol requires these sera checks to be run every 20 observations. However, this is not the case. No information is available as to the order of participant testing and the order in which the sera checks were made between participants. If these data were available, it would be easier to determine when a machine malfunctions. Depending on when the change took place, different adjustments could be made for cholesterol values of participants tested. But, how is it easier to determine when the process went astray? Prior information concerning participants' cholesterol levels must be known. Participants responded to the Cholesterol Change at Work survey which includes questions of past cholesterol testing . ·In particular, the following questions would give the prior information necessary to more readily detect the change point. How long has it been since you last had your cholesterol measured? 1. Never had it measured 2. Less than 6 months 3. Between 6 months and I year 4. More than 1 year 49 What was your blood cholesterol level the last time you had it measured? 1. (number) 2. I was not told 3. Don't remember 4. Never had it measured If an individual had his/her cholesterol measured within the last 6 months and recorded the actual number, then a comparison of the prior and current cholesterol values of this participant could be made to determine if the malfunction occurred before or after this person's cholesterol measurement in the field . Since the prior information includes a cholesterol value within the last 6 months, the current cholesterol value should be within a range of seasonal variability, other things being equal. 50 Vil. SUMMARY The results of the reliability testing are fair . The initial lot testing determined that the overall accuracy of the reagent strip lots is not within the NCEP guidelines (bias<=5%). The Reflotron obtained cholesterol measurements are negatively biased, consistently underestimating the true cholesterol values. However, both lots 941 and 942 in February, I 993 and lot 564 in August, I 993 warranted further testing. The precision testing resulted in the determination of the lot of reagent strips to be used in the field experiment. The coefficients of variation resulting from the precision experiment are well within the allowable range (<=5%). And, the maximum percent errors are small for each experiment conducted (P( .05)<=4%). In the August experiments, lot 564 has maximum percent errors approximately half those of lot 941 . Again, this may possibly be attributed to the expiration date of lot 94 I approaching. Lot 94 I was the reagent strip selected for the baseline field experiment and lot 564 selected for the 6-month follow-up . Confidence intervals on the individual variance components and the total variability in the experimental model give an estimated range of possible variability. The major component to the variability of the model appears to be the technician-to-technician variability in the February experiments and the vial-to-vial variation in reagent strips in the August experiments. The field experiment yields cholesterol measurements on participants at worksites. These values are underestimates of the true cholesterol levels of the participants. The two-step adjustment procedure utilizes the results of the accuracy and precision experiments to determine the estimated true cholesterol values. These predicted values are to be utilized in the analysis of the field experiment. The construction of prediction 51 intervals on individual predicted cholesterol estimates help minimize the misclassification of individuals into the cholesterol risk categories. The reasons for wide-spread cholesterol testing are well documented. And cholesterol testing·with the Reflotron machine has many advantages over other cholesterol measurement methods. The most important advantage is the portability of the Reflotron machine which allows the movement of the desk-top analyzer to where people are. This frees the process from usual laboratory space requirements. This characteristic increases participation rates and decreases expenses. Recently, medical waste disposal in the environment has been highly publicized and regulations regarding disposal have become increasingly restrictive. The Reflotron minimizes the amount of medical waste resulting from cholesterol screening since the process involves minimal supplies and no needles, unlike method involving veneous blood samples. This dry-chemistry method is also much less expensive to use than wet-chernistry methods. The materia~s needed for Reflotron use are mostly plastic and are mass produced. And, the Reflotron is simple to use. Thus, operators need minimal training to achieve reliable results. However, things can go wrong. Accuracy in the field is dependent on some important factors . Adequate cleaning of the Reflotron machine is critical to the accuracy of the test . Protocol must be followed to maintain reliable results. Deviations from protocol can reduce the accuracy, as can distractions in the testing area. Even subjecting the Reflotron machine and materials to excessive temperature changes can reduce the accuracy. 52 At half of the worksites, those belonging to the external condition (Appendix A, Figure I), project staff are employed. The Reflotron machines utilized by them are transported daily to and from the worksite in a project staff vehicle. This constant transportation is sure to affect the performance of the Reflotrons. Continuous data monitoring of the machines and a strict data reporting and analyzing protocol are necessary. However, there tends to be a built-in disincentive to report extreme values resulting from quality control sera checks. If a technician continues to employ a malfunctioning machine either unintentionally or intentionally, accuracy of the test would be affected. The ideal situation would be to have some sort of device on the Reflotrons involved in the study which automatically records the quality control values. 53 References 1. American Heart Association: 1989 Heart Facts. Dallas, Texas: American Heart Association 1989. 2. National Institute of Health: Treatment of Hypertriclyceridemia. NIH Consensus Development Conference Summary. Arterioscelerosis 1984; 4: 296-301. 3. Sempos, C, et al: The Prevalence of High Blood Cholesterol Levels Among Adults in the United States. Journal of the American Medical Association 1989; 262: 45-52. 4. Lefebvre, R. Performance Characteristics of the Boehringer Mannheim Diagnostics Reflotron. Public Health Reports 1987; 104: 266-270. 5. Sedor, F, et al : Reflotron Cholesterol Measurement Evaluated as a Screening Technique. Clinical Chemistry, 1988; 34: 2542-2545 . 6. Powers, D. Establishing and Maintaining Performance Claims. Arch Pathol Lab Med 1992; 116: 718-725. 7. Montgomery, D, et al. Introduction to Linear Regression Analysis. John Wiley & Sons, Inc: NY, !\TY. 1992. 8. Neter, J, et al. Applied Linear Statistical Models, Second Edition. Richard D. Irwin, Inc: Homewood, IL. 1985. 9. Montgomery, D . Design and Analysis of Ex.periments, Third Edition. John Wiley & Sons, Inc: NY, NY. 1991. 10. SAS Institute. SAS/STAT User's Guide, Version 6, Fourth Edition, SAS Institute, Inc: Cary, NJ. 1990. 11. Searle, S, et al. Variance Components. John Wiley & Sons, Inc: NY, NY. 1992. 12. DeVor, R, et al . Statistical Quality Design and Control. Macmillan Publishing Company: NY, NY. 1992. 54 13 . McShane, L, et al : Reporting the Accuracy of Biochemical Measurements for Epidemiologic and Nutrition Studies. American Journal of Clinical Nutrition 1991 ; 53 : 1354-1360. 14. McShane, L, et al : Application of Variance Components Methods.to Laboratory Quality Control for Biochemical Measurements. 1992 American Statistical Association Proceedings of the Biopharmaceutica/ Section . 250-256. 15. Burdick, R, et al. Confidence Intervals on Variance Components. Marcel Dekker, Inc: NY, NY. 1992. 16. Hernandez, R, et al : Confidence Intervals and Tests of Hypotheses on Variance Components in an Unbalanced Two-Fold Nested Design. Forthcoming in Biom. J. 1992. 17. Ting, N, et al. Confidence Intervals on Linear Combinations of Variance Components that are Unrestricted in Sign. Journal of Statistical and Computational Simulation 1990; 35: 135-143 . 18. Canadian Society of Clinical Chemists: Out-of-Laboratory Measurement of the Blood Cholesterol Level. Canadian Medical Association Journal 1990; 143 : 721-723. 19. Morgan, J. MRFIT, Quality Control of Technical Procedures and Data Acquisition. Controlled Clinical Trials: Design, Methods, and Analysis 1986; 7: 1 S-202S. 20. Nanji, A, et al : Quality of Laboratory Test Results Obtained by Non-Technical Personnel in a Decentralized Setting. AJCP, June 1988: 797-801. 21 . Draper, N, et al. Applied Regression Analysis, Second Edition. John Wiley & Sons, Inc: N Y, NY. 1981. 22. Ott, R. An introduction to Statistical Methods and Data Analysis, Fourth Edition. Duxbury Press: Belmont, CA. 1993 . 23 . Neter, J, et al. Applied Linear Regression Models. Richard D. Irwin, Inc: Homewood, IL. 1983 . 55 VI °' February, 1993 Lot 941 Lot 942 August, 1993 Lot 941 Lot 943 August, 1993 Lot 941 Lot 564 Table 1 Initial Lot Testing Results Percent of Observations Within the NCEP Guidelines % observations % observations Average Percent within within Change from Gold Standard Deviation ± 14.2% ± 8.9% J..% Bias_l 100.0 84.2 -3 .6 0.043 100.0 78.9 -5 .1 0.047 75.8 60.6 -8.0 0.057 72.7 27.3 -11.4 0.041 73.3 43 .3 -10.2 0.059 100.0 70.0 -6.5 0.043 VI -..J Lot 941 (2/93) Po P1 Lot 942 (2/93) Po Pi Table 2 Initial Lot Testing Results Simple Linear Regression Parameter Estimates Parameter Estimate Standard Error 5.624063 9.732 0.936384 0.047 -26.386923 9.777 1.085048 0.047 Prob> I Tl H . ro"O 0 . "'""' ,,,., 0.5709 0.0001 0.1925 0.0152 0.0001 0.0887 Vi 00 Lot 941 (8/93) Po p, Lot 943 (8/93) Po p, Lot 941 (8/93) Po p, Lot 564 (8/93) Po p, Table 2 (con't) Initial Lot Testing Results Simple Linear Regression Parameter Estimates Parameter Estimate Standard Error -29.047838 8.797 1.065342 0.041 -8.230145 6.990 0.927026 0.033 -68.184786 9.003 1.224978 0.042 -32.148122 10.064 1.088572 0.047 Prob>ITI ("o=O Ho: ,,,=o ,,,., 0.0025 0.0001 0.1234 0.2483 0.0001 0.0335 0.0001 0.0001 0.0001 0.0034 0.0001 0.0697 Vl '° Pool High Med Low Mean Cholesterol -Value= I>; 239.75 214.86 157.28 Standard Error 1.09 1.13 1.06 Table 3a P; and Factor Levels & Values February, 1993 Lot Testing 1st 941 Factor Levels Pool 3 High, Med, Low Round 2 1 2 Sample 6 123456 Values Machine 19 123 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 Technician 3 ] 2 3 Number of Observations in Data Set = 108 °' 0 Pool High Med Low Mean Cholesterol Standard - Error Value= I'; 236.87 0.77 214.74 0.84 152.01 0.87 Table Jb P; and Factor Levels & Values February, 1993 Lot Testing 2nd 941 Factor Levels Pool 3 High Med Low Round 2 34 Sample 6 123456 Values Machine 18 3 4 5 6 7 8 9 IO 11 I 2 I 4 15 16 17 18 19 20 21 Technician 3 l 2 3 Number of Observations in Data Set = 108 Mean °' Pool Cholesterol Standard Value=P; Error High 242.75 0.70 Med 220.15 0.77 Low 152.22 0.79 Table Jc P; and Factor Levels & Values February, 1993 Lot Testing 1st 942 Factor Levels Pool 3 High Med Low Round 2 I 2 Sample 6 123456 Values Machine 18 123456789101112131415161718 Technician 3 1 2 3 Number of Observations in Data Set = l 08 °' N Pool High Med Low Mean Cholesterol Standard - Error Value= P; 249.39 0.85 224.36 0.93 152.59 0.96 Table 3d P; and Factor Levels & Values February, 1993 Lot Testing 2nd 942 Factor Levels Pool 3 High Med Low Round 2 l 2 Sample 6 I 23456 Values Machine 18 123456789101112131415161718 Technician 3 1 2 3 Number of Observations in Data Set = 108 °' w Pool High Med Low Mean Cholesterol Standard - Error Value=?; 226.04 0.80 207.69 0.76 148.57 0 .82 Table 3e P; and Factor Levels & Values August, 1993 Lot 941Testing Factor Levels Pool 3 High Med Low Round 2 I 2 Sample 5 l 2 3 4 5 Values Machine 15 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Technician 3 1 2 3 Strip Vial 5 12345 Number of Observations in Data Set = 90 Mean Pool Cholesterol Standard ~ - Error Value=P; High 231.17 0.83 Med 212.85 0.79 Low 154.78 0.85 Table Jf P; and Factor Levels & Values August, 1993 Lot 564 Testing Factor Levels Pool 3 High Med Low Round 2 1 2 Sample 5 123 4 5 Values Machine 15 123456 7 8 9 10 11 12 13 14 15 Technician 3 I 2 3 Strip Vial 5 1 2 3 4 5 Number of Observations in Data Set = 90 °' Vl Variance Component Jst 941 Round Sample Machine Technician Error 2nd 941 Round Sample Machine Technician Error Table 4 Estimation of Variance Components by Different Methods February, 1993 Lot Testing Method of Estimation TYPE I MIVQUE(O) ML 7.9122 7.9500 2.9986 -2.7953 -3 .0684 0 1.0055 4.3376 3.5110 9.6260 9.2385 6.6468 28 .9153 26.2792 24.5482 3.4425 3.4569 1.2187 -0.7038 -0.6940 0 -0.5464 0.2971 0.2877 0.1926 0.2158 0.0884 19.7965 18.9611 18.4013 REML 6.3869 0 3.3500 9.5015 24.6336 3.3807 0 0.4213 0.2366 18.2133 °' °' Variance Component Jst 942 Round Sample Machine Technician Error 2nd942 Round Sample Machine Technician Error Table 4 (con't) Estimation of Variance Components by Different Methods February, 1993 Lot Testing Method of Estimation TYPE I MIVQUE(O) ML 0.6844 0.6962 0 2.7320 2.7401 2.4425 I .2903 1.9823 1.9747 6.5440 6.5630 4.3853 16.4820 I 5. 7967 I 5.9289 0.3794 0.4222 0 -0.5667 -0.5376 0 0.2294 2.7324 2.5123 3.0944 3.1633 2.0453 24.1847 21. 7055 21.0923 REML 0.7229 2.5252 2.0440 6.5599 I 5.9093 0.2041 0 2.6170 3.1721 21.3880 °' -....J Variance Component 941 Round Sample Machine Technician Strip Vial Error 564 Round Sample Machine Technician Strip Vial Error Table 4 (con't) Estimation of Variance Components by Different Methods August, 1993 Lot Testing Method of Estimation TYPE I MIVQUE(O) ML -0.1289 0. 1082 0 -0.3428 -0.5242 0 3.8312 4.7897 1.9901 -0.0706 0.0447 0 38.1914 41.2412 32.4448 15.0513 11.5927 13 .5874 -0.9731 -0.8002 0 3.7710 3.3566 0 4.7540 1.1428 2.6175 0.3138 0.3026 . 0.2235 4.0225 8.0873 7.3342 16.4201 16. 7556 17.4344 REML 0 0 2.0027 0 40.6123 13 .9373 0 0 2.7609 0.2673 9.1771 17.8136 °' 00 LOT 941 942 Table 5 95% Tolerance Limits for Pool i of Sera February, 1993 Lot Testing P; ± Z0025 ~CT~ +CT~ +CT~ +CT~ +CT; POOL LB High 227 Medium 202 Low 145 High 238 Medium 213 Low 141 LB = Lower Bound UB = Upper Bound UB 252 227 170 260 235 164 °' \0 LOT 941 564 Table 5 (con't) 95% Tolerance Limits for Pool i of Sera August, 1993 Lot Testing ; - 002s .an +as +a M +a r +a v +a,. p + z ~ 2 2 2 2 2 2 POOL LB High 211 Medium 192 Low 133 High 220 Medium 202 Low 144 LB = Lower Bound UB = Upper Bound UB 241 223 164 242 224 166 -...I 0 Table 6 Maximum Percent Error & Coefficient of Variation August, 1993 Lot Testing .>-; will be within 100 * P( a)% of the true value of the sample µ P, with 1 - a probability Ex_p_eriment Maximum Percent Error Coefficient of Variation a =.05 1st 941 High 2.21% 1.13% Medium 2.47% 1.26% Low 3.37% 1.72% 2nd 941 High 1.26% 0.64% Medium 1.39% 0.71% Low 1.96% 1.00% 1st 942 High 1.47% 0.75% Medium 1.63% 0.83% Low 2.36% 1.20% 2nd 942 High 1.09% 0.56% Medium 1.22% 0.62% Low 1.80% 0.92% -..J -Table 6 (con't) Maximum Percent Error & Coefficient of Variation August, 1993 Lot Testing Y; will be within IOO* P(a)% of the true value of the sample µ P, with 1 - a probability E~eriment Maximum Percent Error Coefficient of Variation 941 High Medium Low 564 High Medium Low a =.05 2.57% 2.79% 3.91% 1.41% 1.53% 2.11% 1.31% 1.43% 1.99% 0.72% 0.78% 1.08% -....J N Source of Variation Pool Round Sample Machine Technician Strip Vial Error Total Table 7 Analysis of Variance with Type III Unweighted Sums of Squares for Unbalanced Two-Fold Nested Mixed Model OF no = i-1 nt = iG-1) n1 = ij(k-1) n3 = m-1 n4 = n-1 n5 = p-1 "6 =by subtraction N-1 SS ssou SSlU SS2U SS3U SS4U SS5U SS6 SST MS S~u sl~I Siu Siu (' 2 ..i4u s52U s~ EMS (} 2 2 2 Q ou =aE +w,ua s +ww aR+ P (} 2 2 2 IU =a E + WJU (Y S + W4u CY R (} 2 2 w =aE+wsu CYs (} 2 2 3U = a E + w 6U a M (} 2 2 4V = a E +ww ar (} 2 2 su = a E + Wsua v (}6 =a~ -...J w Source of Variation Pool Round Sample Machine Technician Error Total Table Sa Analysis of Variance with Type III Unweighted Sums of Squares for Unbalanced Two-Fold Nested Mixed Model February, 1993 1st LOT 941 DF SS MS EMS 2 83204.12 41602.06 (} 2 2 2 Q OU = (J" E + 1.755 lu s + 10.53 lu R + p 3 233 .74 77.91 (}1u = a-~+ 2.323 lu~ + 13.939u~ 30 524.05 17.47 (}w = a-~ + 2.4173u~ 18 618 .94 34.39 (}w = a-~ + 3.6245u~ 2 738.84 369.42 (} 4u = (J"~ + 3 5J 730"~ 52 1503.60 28.92 ()5 = (J"~ 107 131216.77 -...J ~ Source of Variation Pool Round Sample Machine Technician Error Total Table Sb Analysis of Variance with Type III Unweighted Sums of Squares for Unbalanced Two-Fold Nested Mixed Model February, 1993 2nd LOT 941 DF SS MS EMS 2 100220.80 50110.40 Bou= a~ +2.2754a; + 13.652a~ +Qp 3 214.91 71.64 Bw = a~ + 2.5557a; + 15.334a~ 30 472.00 15.73 Bw =a~ + 2.5029a; 17 298.66 17.57 (} 2 2 3U =a E + 4.0784aM 2 53.46 26.73 (}4u = a~+ 36a~ 53 1049.22 19.80 (}s = a i 107 140643.52 ......i Vo Source of Variation Pool Round Sample Machine Technician Error Total Table Sc Analysis of Variance with Type III Unweighted Sums of Squares for Unbalanced Two-Fold Nested Mixed Model February, 1993 1st LOT 942 DF SS MS EMS 2 114323.47 571 6 1.74 B0u = CT~ +2 .2754CT~ + 13.652CT~ +Qp 3 92.98 30.99 (} IU = CT~ + 2.5557CT~ + 15.334CT7i 30 626.13 20.87 (}2U = CT~ + 2.5029CT~ 17 369.66 21.74 (}3U = CT~+ 4 .0784CT~ 2 504.13 252.06 (} 4u = CT~ + 36CT~ 53 873 .55 16.48 (}5 = CT~ 107 166119.85 ......i °' Source of Variation Pool Round Sample Machine Technician Error · Total Table 8d Analysis of Variance with Type III Unweighted Sums of Squares for Unbalanced Two-Fold Nested Mixed Model February, 1993 2nd LOT942 DF SS MS EMS 2 130461.63 65230.82 (} 2 5 2 2 Q ou = a E + 2.27 4as + l 3.652a n + P 3 59.87 19.96 2 2 2 Bw = a E + 2.5557a 8 +15.334a 11 30 510.80 17.03 Bw = a~ + 2.5029a~ 17 427.05 25 .12 () 3u = a~ + 4 .0784a~ 2 271.17 135 .58 (}4u = a~ + 36a~ 53 1281. 79 24.18 (} = a~ 5 ' 107 184596.67 Table Se Analysis of Variance with Type III Unweighted Sums of Squares for Unbalanced Two-Fold Nested Mixed Model August, 1993 LOT 941 Source of Variation DF SS MS EMS -...J -...J Pool 2 64700.33 323S0.16 (}OU = (Ji+ 2 .21160"~ + 1 J.0580"~ + Qp Round 3 11. 91 J.97 Ow = CJ~ + 2 .6237CJ~ + 13.119CJ~ Sample 24 292.06 12.17 ()2U = (Ji+ 2.34230"~ Machine 14 270.88 l9.3S (}3U = (Ji + 3.84640"~ Technician 2 2S .87 12.93 (} 4U = (Ji + 300"~ Strip Vial 4 1364.99 341.2S (}5U = ()~ + 8.54 JlCJ~ Error 40 602.05 IS.OS ()6 = () ~. Total 89 106640.10 -...I 00 Source of Variation Pool Round Sample Machine Technician Strip Vial Error Total Table Sf Analysis of Variance with Type III Unweighted Sums of Squares for Unbalanced Two-Fold Nested Mixed Model August, 1993 LOT 564 DF SS MS EMS 2 62788.17 31394.08 2 2 ~ 0 Bou = a E + 2.2116a s + 1 l.058a n + -P 3 64.42 21.47 B1u = a~ + 2 .6237a~ + 13.1l9a~ 24 492.95 20.54 Bw = a~ + 2.3423a~ 14 550.03 39.29 B3u = a~+ 3.8464a!, 2 51.67 25 .83 B4u = a~ + 30a~ 4 203.11 50.78 B5u =a~+ 8.541 ta; 40 656.81 16.42 ()6 =a~ 89 96485.60 ....J '° Table 9a 95% Confidence Intervals on Variance Components February, 1993 1st LOT 941 Variance Component Estimate Lower Bound Round 4.30 0.52 Sample -4.74 -11.60 Machine 1.51 -4.40 Technician 9.63 1.98 Error 28.92 20.37 Total Variation 39.62 28.57 Upper Bound 76.5 1.86 12.89 411.64 44.28 448.30 00 0 Table 9b 95% Confidence Intervals on Variance Components February, 1993 2nd LOT 941 Variance Component Estimate Lower Bound Round 3.65 0.34 Sample -1 .63 -6.35 Machine -0.55 -3 .73 Technician 0.19 -0.43 Error 19.8 13 .99 Total Variation 21.47 16.44 Upper Bound 63 .87 3.77 4.93 28 .76 30.18 88 .. 62 Table 9c 95% Confidence Intervals on Variance Components February, 1993 1st LOT 942 ,, Variance Component Estimate Lower Bound Upper Bound 00 -Round 0.65 -1.13 26.65 Sample 1.75 -2.82 8.53 Machine 1.29 -l.88 7.98 Technician 6.54 1.42 276.07 Error 16.48 I 1.64 25 . 12 Total Variation 26.72 19.59 297.68 00 N Table 9d 95% Confidence Intervals on Variance Components February, 1993 2nd LOT 942 Variance Component Estimate Lower Bound Round 0.20 -1.08 Sample -2.86 -8.48 Machine 0.23 . -3 .91 Technician 3.09 0.31 Error 24.18 17.08 Total Variation 24.85 19.30 Upper Bound 16.93 3.12 8.01 148.05 36.85 171.13 00 l..>J Table 9e 95% Confidence Intervals on Variance Components August, 1993 LOT 941 Variance Component Estimate Lower Bound Round -0.60 -1 .60 Sample -1 .23 -5 .79 Machine 1.12 -2.31 Technician -0.07 -0.53 Strip Vial 38.19 12.51 Error 15.05 10.14 Total Variation 52.46 26.33 Upper Bound 3.26 3.99 8.63 16.50 328.08 24.64 343 .05 00 ~ Table 9f 95% Confidence Intervals on Variance Components August, 1993 LOT 564 Variance Component Estimate Lower Bound Round 0.03 -1 .98 Sample 1.76 -3 .85 Machine 5.95 0.46 Technician 0.31 -0.42 Strip Vial 4.03 -0.03 Error 16.42 11 .07 Total Variation 28.49 21 .03 Upper Bound 21 .04 10.19 21.11 33.43 47.09 26.88 89.19 15 10 5 0 1~0 160 0 -5 • ftl 00 m -10 I Vt • -15 -20 -25 -30 Figure la: Bias vs. True Value February, 1993 Lot941 • • • 200 • • • • • • • True Value • 250 300 • • • • 20 15 I 10 5 t en 0 00 .9 T 120 140 °' m -5 • -10 • • -15 -+- • -20 -+--25 ' Figure 1 b: Bias vs. True Value February, 1993 Lot942 180 180 200 220 • • • • • • • • True Value • • • 240 260 280 300 • • 15 10 5 oi' I I I I I I I I I I I I I 150 _51 0 ,; -10 t 00 m -1 5 -....J • -20 • • -25 t •• -30 • -35 -40 Figure 2a: Bias vs. True Value August, 1993 Lot 941 I I I I I I I Ill I I I I I I I I I • • 200. 250 • • • • .. • • • • • • • • • • • True Value • I I I • 300 350 • • • 15 10 5 0 .51~0 150 en -10 • 00 ftl 00 m -15 • -20 • -25 • • -30 -35 • -40 Figure lb: Bias vs. True Value August, 1993 Lot 943 200 250 • • I • • • • • • • • • • • • •• • • • True Value 300 350 • • • • • 15 10 5 0 _51~0 120 140 en -10 I'll 00 iii '-0 -15 -20 I • -25 -30 + • -35 + • -40 Figure Ja: Bias vs. True Value August, 1993 Lot 941 160 180 200 220 • • • • I • • • • • • •• • • • • True Value • 240 260 280 300 • • • • • • • • • 15 10 5 0 100 120 140 en -5 I'll I • '° m 0 -10 -15 -20 l • • -25 -30 Figure Jb: Bias vs. True Value August, 1993 Lot 564 • 160 180 200 220 • • • • • • • • • • • • • • • • • • True Value • 240 • 260 • 280 300 • • • • Appendices Appendix A Figure 1 - Experimental Conditions 92 Figure 2 - Enzymatic Assay Method for Total Blood Cholesterol 93 Appendix B - Results of Initial Lot Testing Figure 1 - February, 1993 Lot 941 vs. Lot 942 Figure 2 - August, 1993 Lot 941 vs. Lot 943 Figure 3 - August, 1993 Lot 941 vs. Lot 564 Appendix C - Precision Experiment Generated By Proc Plan & Cholesterol Results Figure 1 - February, 1993 1st Lot 941 Figure 2 - February, 1993 2nd Lot 941 Figure 3 - February, 1993 1st Lot 942 Figure 4 - February, 1993 2nd Lot 941 Figure 5 - August, 1993 Lot 941 Figure 6 - August, 1993 Lot 564 91 94 95 96 97 101 105 109 113 116 OCCUPATIONAL NURSES (INTERNAL) PROJECT STAFF (EXTERNAL) Appendix A Figure 1 Experimental Conditions FACE TO FACE FEEDBACK (FF) 5 Worksites 5 Worksites USUAL CARE (Minimal education about cholesterol such as usual public screener) 5 Worksites 92 AUTOMATED FEEDBACK (R1) 5 Worksites 5 Worksites Appendix A Figure 2 Enzymatic Assay Method for Total Blood Cholesterol from : Allain CA, Poon LS . Enzymatic Determination of Total Serum Cholesterol. Clin Chem 1975; 20: 470-476. All samples analyzed for total cholesterol in the Lipid Metabolism Laboratory were assayed on a Gilford Impact 400 auto analyzer (Gilford Instruments, Oberlin, Ohio) using reagent supplied by Worthington Diagnostics (Cooper Biochemicals, Malvern, Pennsylvannia). Breifly, cholesteryl esters in serum are hydrolyzed by cholesteryl esterase to produce unesterified cholesterol and fatty acids. The unesterified cholesterol is oxidized to cholestem-3-one and hydrogen peroxide. A quinoneimine chromagen with an absorption maximum at 500 nm is produced when phenol is oxidatively coupled with 4-amino antipyrine in the presence of hydrogen peroxide and the enzyme peroxidase. The absorbance is proportional to the amount of total cholesterol in the test sample. Primary cholesterol standards were obtained from the Center for Disease Control - Lipid Standardization Program which were used to calibrate large secondary serum pool standards for daily use in calibration of the Gilford Impact 400. Low, medium, and high cholesterol pools, generated from salvaged serum, were analyzed in each assay as well as a commercial serum-based control and a commercial aqueous standard. 93 Appendix B Figure 1 Results of Initial Lot Testing February, 1993 Lot 941 & 942 Obs Lot 941 Lot 942 Miriam 1 216 204 228 2 231 222 227 3 206 193 209 4 214 211 224 5 263 237 258 6 188 158 176 7 206 194 210 8 146 137 149 9 100 100 111 10 148 146 154 11 161 153 166 12 207 193 206 13 246 281 274 14 219 247 243 15 265 297 281 16 151 157 160 17 178 180 188 18 224 230 235 19 142 135 150 94 Appendix B Figure 2 Results of Initial Lot Testing August, 1993 Lot 941 & 943 Obs Lot 941 Lot 943 Miriam 1 213 183 219 2 129 118 154 3 220 200 223 4 177 163 196 5 273 243 271 6 225 212 244 7 281 264 292 8 167 161 182 9 162 155 173 10 258 244 263 11 160 154 12 138 133 161 13 187 173 196 14 296 274 311 15 114 115 135 16 201 192 203 17 204 184 207 18 214 193 233 19 175 168 200 20 212 194 211 21 244 244 276 22 135 136 151 23 132 147 157 24 157 168 187 25 202 201 216 26 213 208 229 27 188 185 198 28 138 144 169 29 171 162 177 30 227 203 232 31 227 232 248 32 158 161 184 33 166 184 198 95 Appendix B Figure 3 Results oflnitial Lot Testing August, 1993 Lot 941 & 564 Obs Lot 941 Lot 564 Miriam 1 209 218 223 2 210 217 240 3 133 137 156 4 255 263 266 5 156 171 186 6 201 208 209 7 196 205 221 8 161 176 181 9 231 243 246 JO 220 224 233 11 217 211 229 12 220 219 238 13 166 186 203 14 195 197 213 15 312 308 300 16 159 174 188 17 194 197 225 18 171 176 204 19 228 237 237 20 155 172 182 21 173 176 195 22 182 180 202 23 116 144 151 24 121 131 151 25 144 161 184 26 167 168 189 27 209 211 223 28 233 231 243 29 195 193 216 30 227 225 240 31 160 180 181 96 Appendix C Figure 1 Precision Experiment Generated By Proc Plan & Cholesterol Results February,1993 1st Lot 941 obs tech seq round pool sample mach chol 1 I 1 1 high 1 16 249 2 1 2 high 2 13 243 3 3 high 3 9 245 4 1 4 1 high 4 18 248 5 1 5 1 high 5 17 243 6 1 6 1 high 6 10 244 7 I 7 low 1 15 165 8 I 8 I low 2 1 146 9 9 1 low 3 2 154 10 10 1 low 4 6 161 11 1 11 1 low 5 7 163 12 12 1 low 6 8 159 13 13 1 med I 4 222 14 1 14 med 2 5 230 15 15 ) med 3 1 ) 223 16 1 16 1 med 4 3 224 17 1 17 1 med 5 14 221 18 1 18 med 6 12 235 19 1 1 2 low 1 8 165 20 1 2 2 low 2 5 162 21 1 3 2 low 3 4 165 22 1 4 2 low 4 7 169 23 1 5 2 low 5 13 166 24 1 6 2 low 6 10 164 25 1 7 2 med 1 6 225 26 1 8 2 med 2 17 212 27 1 9 2 med 3 11 210 28 1 10 2 med 4 3 217 29 1 11 2 med 5 14 216 30 1 12 2 med 6 15 212 97 Appendix C Figure 1 (con't) ·' Precision Experiment Generated By Proc Plan & Cholesterol Results February,1993 1st Lot 941 obs tech seq round pool sample mach chol 31 1 13 2 high 1 9 242 32 1 14 2 high 2 19 234 33 15 2 high 3 18 231 34 1 16 2 high 4 1 229 35 1 17 2 high 5 12 237 36 1 18 2 high 6 2 239 37 2 1 med 1 11 212 38 2 2 1 med 2 19 220 39 2 3 med 3 6 214 40 2 4 1 med 4 12 217 41 2 5 1 med 5 14 214 42 2 6 med 6 17 215 43 2 7 high 1 15 238 44 2 8 high 2 16 243 45 2 9 high 3 2 237 46 2 10 high 4 10 243 47 2 11 high 5 5 243 48 2 12 high 6 18 238 49 2 13 1 low 1 3 154 50 2 14 1 low 2 4 166 51 2 15 1 low 3 7 157 52 2 16 1 low 4 9 156 53 2 17 1 low 5 13 154 54 2 18 1 low 6 8 156 55 2 1 2 high 1 16 238 56 2 2 2 high 2 2 243 57 2 3 2 high 3 3 237 58 2 4 2 high 4 19 237 59 2 5 2 high 5 1 234 60 2 6 2 high 6 11 237 98 Appendix C Figure 1 (con't) Precision Experiment Generated By Proc Plan & Cholesterol Results February, 1993 1st Lot 941 obs tech seq round pool sample ma ch chol 61 2 7 2 low 1 14 154 62 2 8 2 low 2 4 156 63 2 9 2 low 3 10 155 64 2 10 2 low 4 18 151 65 2 11 2 low 5 13 156 66 2 12 2 low 6 9 154 67 2 13 2 med 1 17 216 68 2 14 2 med 2 5 213 69 2 15 2 med 3 6 217 70 2 16 2 med 4 7 214 71 2 17 2 med 5 15 214 72 2 18 2 med 6 12 217 73 3 1 low 1 6 146 74 3 2 low 2 11 149 75 3 3 low 3 16 149 76 3 4 1 low 4 8 148 77 3 5 1 low 5 1 149 78 3 6 low 6 17 156 79 3 7 1 med 1 12 213 80 3 8 1 med 2 19 216 81 3 9 med 3 5 207 82 3 10 1 med 4 7 207 83 3 11 1 med 5 4 211 84 3 12 1 med 6 9 217 85 3 13 1 high 1 15 240 86 3 14 1 high 2 3 235 87 3 15 1 high 3 2 239 88 3 16 1 high 4 IO 236 89 3 17 1 high 5 18 231 90 3 18 1 high 6 13 239 99 Appendix C Figure 1 (con't) Precision Experiment Generated By Proc Plan & Cholesterol Results February, 1993 1st Lot 941 obs tech seq round pool sample ma ch chol 91 3 1 2 med 1 12 207 92 3 2 2 med 2 6 209 93 3 3 2 med 3 14 205 94 3 4 2 med 4 1 204 95 3 5 2 med 5 19 219 96 3 6 2 med 6 15 216 97 3 7 2 high 1 7 245 98 3 8 2 high 2 2 234 99 3 9 2 high 3 9 238 100 3 10 2 high 4 18 229 101 3 11 2 high 5 17 240 102 3 12 2 high 6 5 250 103 3 13 2 low 1 11 159 104 3 14 2 low 2 13 166 105 3 15 2 low 3 10 158 106 3 16 2 low 4 3 159 107 3 17 2 low 5 4 169 108 3 18 2 low 6 8 152 100 Appendix C Figure 2 Precision Experiment Generated By Proc Plan & Cholesterol Results February, 1993 2nd Lot 941 obs tech seq round tube pool sample mach chol 1 1 1 3 1 high 1 12 239 2 2 3 1 high 2 16 231 3 1 3 3 1 high 3 10 235 4 1 4 3 1 high 4 21 244 5 1 5 3 1 high 5 8 235 6 6 3 1 high 6 9 234 7 1 7 3 2 med 1 17 214 8 1 8 3 2 med 2 5 214 9 1 9 3 2 med 3 3 211 10 1 10 3 2 med 4 11 209 11 11 3 2 med 5 18 210 12 12 3 2 med 6 4 218 13 13 3 3 low 1 15 147 14 14 3 3 low 2 7 155 15 15 3 3 . low 3 14 151 16 16 3 3 low 4 20 155 17 1 17 3 3 low 5 19 151 18 1 18 3 3 low 6 6 152 19 1 19 4 4 med 1 14 216 20 20 4 4 med 2 20 217 21 1 21 4 4 med 3 16 215 22 1 22 4 4 med 4 10 214 23 1 23 4 4 med 5 17 216 24 24 4 4 med 6 18 214 25 1 25 4 5 low 1 7 149 26 1 26 4 5 low 2 19 148 27 1 27 4 5 low 3 6 148 28 1 28 4 5 low 4 5 152 29 1 29 4 5 low 5 12 165 30 30 4 5 low 6 3 153 101 Appendix C Figure 2 (con't) Precision Experiment Generated By Proc Plan & Cholesterol Results February, 1993 2nd Lot 941 obs tech seq round tube pool sample mach chol 31 1 31 4 6 high 1 21 247 32 1 32 4 6 high 2 4 243 33 1 33 4 6 high 3 8 241 34 1 34 4 6 high 4 9 244 35 35 4 6 high 5 11 241 36 1 36 4 6 high 6 15 247 37 2 1 3 1 med 1 7 213 38 2 2 3 1 med 2 4 217 39 2 3 3 1 med 3 6 206 40 2 4 3 1 med 4 17 212 41 2 5 3 1 med 5 9 218 42 2 6 3 1 med 6 18 209 43 2 7 3 2 low 1 20 157 44 2 8 3 2 low 2 5 156 45 2 9 3 2 low 3 8 148 46 2 10 3 2 low 4 3 148 47 2 11 3 2 low 5 21 152 48 2 12 3 2 low 6 12 155 49 2 13 3 3 high 1 14 235 50 2 14 3 3 high 2 15 236 51 2 15 3 3 high 3 16 230 52 2 16 3 3 high 4 11 234 53 2 17 3 3 high 5 19 233 54 2 18 3 3 high 6 10 235 55 2 19 4 4 low 1 15 153 56 2 20 4 4 low 2 3 160 57 2 21 4 4 low 3 8 144 58 2 22 4 4 low 4 7 146 59 2 23 4 4 low 5 14 153 60 2 24 4 4 low 6 21 146 102 Appendix C Figure 2 (con't) Precision Experiment Generated By Proc Plan & Cholesterol Results February, 1993 2nd Lot 941 obs tech seq round tube pool sample mach chol 61 2 25 4 5 high 1 16 246 62 2 26 4 5 high 2 4 235 63 2 27 4 5 high 3 5 232 64 2 28 4 5 high 4 17 237 65 2 29 4 5 high 5 20 233 66 2 30 4 5 high 6 18 235 67 2 31 4 6 med I 6 224 68 2 32 4 6 med 2 10 218 69 2 33 4 6 med 3 12 223 70 2 34 4 6 med 4 9 219 71 2 35 4 6 med 5 19 225 72 2 36 4 6 med 6 11 219 73 3 1 3 low 1 15 158 74 3 2 3 low 2 18 154 75 3 3 3 low 3 9 150 76 3 4 3 1 low 4 3 152 77 3 5 3 1 low 5 8 152 78 3 6 3 1 low 6 21 149 79 3 7 3 2 high 1 12 244 80 3 8 3 2 high 2 7 235 81 3 9 3 2 high 3 6 232 82 3 10 3 2 high 4 16 231 83 3 11 3 2 high 5 14 237 84 3 12 3 2 high 6 I I 234 85 3 13 3 3 med 1 5 220 86 3 14 3 3 med 2 19 212 87 3 15 3 3 med 3 17 212 88 3 16 3 3 med . 4 20 211 89 3 17 3 3 med 5 4 211 90 3 18 3 3 med 6 10 209 103 Appendix C Figure 2 (con't) Precision Experiment Generated By Proc Plan & Cholesterol Results February, 1993 2nd Lot 941 obs tech seq round tube pool sample ma ch chol 91 3 19 4 4 high 1 3 233 92 3 20 4 4 high 2 18 239 93 3 21 4 4 high 3 14 234 94 3 22 4 4 high 4 5 239 95 3 23 4 4 high 5 6 233 96 3 24 4 4 high 6 19 233 97 3 25 4 5 med 1 9 211 98 3 26 4 5 med 2 11 208 99 3 27 4 5 med 3 4 214 100 3 28 4 5 med 4 12 216 101 3 29 4 5 med 5 20 216 102 3 30 4 5 med 6 16 213 103 3 31 4 6 low 1 17 155 104 3 32 4 6 low 2 10 155 105 3 33 4 6 low 3 21 150 106 3 34 4 6 low 4 8 156 107 3 35 4 6 low 5 15 153 108 3 36 4 6 low 6 7 152 104 Appendix C Figure 3 Precision Experiment Generated By Proc Plan & Cholesterol Results February, 1993 1st Lot 942 obs tech seq round tube pool sample ma ch chol 1 1 1 1 1 high 1 12 243 2 2 1 1 high 2 16 242 3 1 3 1 1 high 3 10 245 4 1 4 1 high 4 2 237 5 1 5 1 1 high 5 8 241 6 1 6 1 1 high 6 9 249 7 1 7 1 2 med 1 17 219 8 1 8 1 2 med 2 5 213 9 1 9 1 2 med 3 3 219 10 1 10 1 2 med 4 11 241 11 1 11 1 2 med 5 18 218 12 1 12 1 2 med 6 4 228 13 13 l 3 low 1 15 151 14 1 14 1 3 low 2 7 145 15 15 1 3 low 3 14 152 16 16 1 3 low 4 1 153 17 1 17 3 low 5 13 149 18 1 18 1 3 low 6 6 151 19 1 19 2 4 med 1 14 221 20 1 20 2 4 med 2 1 222 21 21 2 4 med 3 16 217 22 1 22 2 4 med 4 10 219 23 1 23 2 4 med 5 17 219 24 1 24 2 4 med 6 18 218 25 1 25 2 5 low 1 7 152 26 1 26 2 5 low 2 13 148 27 1 27 2 5 low 3 6 149 28 1 28 2 5 low 4 5 150 29 1 29 2 5 low 5 12 153 30 1 30 2 5 low 6 3 150 105 Appendix C Figure 3 (con't) Precision Experiment Generated By Proc Plan & Cholesterol Results February, 1993 1st Lot 942 obs tech seq round tube pool sample ma ch chol 31 1 31 2 6 high 1 2 240 32 1 32 2 6 high 2 4 245 33 33 2 6 high 3 8 233 34 1 34 2 6 high 4 9 240 35 35 2 6 high 5 11 236 36 1 36 2 6 high 6 15 239 37 2 1 1 1 med 1 7 219 38 2 2 l 1 med 2 4 221 39 2 3 1 1 med 3 6 220 40 2 4 1 1 med 4 17 223 41 2 5 1 1 med 5 9 216 42 2 6 1 1 med 6 18 220 43 2 7 2 low 1 1 156 44 2 8 1 2 low 2 5 151 45 2 9 1 2 low 3 8 154 46 2 10 1 2 low 4 3 149 47 2 11 2 low 5 2 155 48 2 12 1 2 low 6 12 158 49 2 13 3 high 1 14 235 50 2 14 3 high 2 15 244 51 2 15 1 3 high 3 16 239 52 2 16 1 3 high 4 11 253 53 2 17 1 3 high 5 13 234 54 2 18 1 3 high 6 10 241 55 2 19 2 4 low 1 15 148 56 2 20 2 4 low 2 3 149 57 2 21 2 4 low 3 8 146 58 2 22 2 4 low 4 7 151 59 2 23 2 4 low 5 14 142 60 2 24 2 4 low 6 2 151 106 Appendix C Figure 3 (con't) Precision Experiment Generated By Proc Plan & Cholesterol Results February, 1993 1st Lot 942 obs tech seq round tube pool sample ma ch chol 61 2 25 2 5 high 1 16 241 62 2 26 2 5 high 2 4 238 63 2 27 2 5 high 3 5 242 64 2 28 2 5 high 4 17 239 65 2 29 2 5 high 5 1 237 66 2 30 2 5 high 6 18 240 67 2 31 2 6 med 1 6 215 68 2 32 2 6 med 2 10 217 69 2 33 2 6 med 3 12 213 70 2 34 2 6 med 4 9 216 71 2 35 2 6 med 5 13 214 72 2 36 2 6 med 6 11 221 73 3 1 1 1 low 1 15 156 74 3 2 1 1 low 2 18 150 75 3 3 1 1 low 3 9 156 76 3 4 1 1 low 4 3 153 77 3 5 1 1 low 5 8 157 78 3 6 1 1 low 6 2 154 79 3 7 1 2 high 1 12 245 80 3 8 1 2 high 2 7 243 81 3 9 1 2 high 3 6 249 82 3 10 1 2 high 4 16 243 83 3 11 2 high 5 14 238 84 3 12 1 2 high 6 11 254 85 3 13 1 3 med 1 5 224 86 3 14 1 3 med 2 13 215 87 3 15 1 3 med 3 17 221 88 3 16 1 3 med 4 1 231 89 3 17 1 3 med 5 4 227 90 3 18 1 3 med 6 10 220 107 Appendix C Figure 3 (con't) Precision Experiment Generated By Proc Plan & Cholesterol Results February, 1993 1st Lot 942 obs tech seq round tube pool sample mach chol 91 3 19 2 4 high 1 3 248 92 3 20 2 4 high 2 18 256 93 3 21 2 4 high 3 14 251 94 3 22 2 4 high 4 5 250 95 3 23 2 4 high 5 6 247 96 3 24 2 4 high 6 13 247 97 3 25 2 5 med 9 216 98 3 26 2 5 med 2 11 224 99 3 27 2 5 med 3 4 227 100 3 28 2 5 med 4 12 222 101 3 29 2 5 med 5 1 224 102 3 30 2 5 med 6 16 225 103 3 31 2 6 low 1 17 152 104 3 32 2 6 low 2 10 150 105 3 33 2 6 low 3 2 152 106 3 34 2 6 low 4 8 154 107 3 35 2 6 low 5 15 157 108 3 36 2 6 low 6 7 151 108 Appendix C Figure 4 Precision Experiment Generated By Proc Plan & Cholesterol Results February, 1993 2nd Lot 942 obs tech seq round tube pool sample mach chol 1 1 1 1 1 high 1 12 247 2 1 2 1 I high 2 16 252 3 3 1 high 3 10 241 4 I 4 1 high 4 2 247 5 1 5 1 1 high 5 8 244 6 1 6 1 1 high 6 9 245 7 1 7 2 med 1 17 221 8 1 8 1 2 med 2 5 238 9 1 9 1 2 med 3 3 216 10 1 10 1 2 med 4 11 221 11 11 2 med 5 18 231 12 12 1 2 med 6 4 223 13 I 13 1 3 low 1 15 151 14 14 3 low 2 7 148 15 15 1 3 low 3 14 154 16 16 1 3 low 4 1 154 17 17 1 3 low 5 13 152 18 18 1 3 low 6 6 154 19 1 19 2 4 med 1 14 222 20 1 20 2 4 med 2 1 221 21 1 21 2 4 med 3 16 214 22 1 22 2 4 med 4 10 227 23 1 23 2 4 med 5 17 220 24 24 2 4 med 6 18 225 25 1 25 2 5 low 1 7 147 26 1 26 2 5 low 2 13 155 27 1 27 2 5 low 3 6 148 28 28 2 5 low 4 5 151 29 1 29 2 5 low 5 12 159 30 1 30 2 5 low 6 3 152 109 Appendix C Figure 4 (con't) Precision Experiment Generated By Proc Plan & Cholesterol Results February, 1993 2nd Lot 942 obs tech seq round tube pool sample ma ch chol 31 1 31 2 6 high 1 2 244 32 1 32 2 6 high 2 4 255 33 33 2 6 high 3 8 255 34 1 34 2 6 high 4 9 250 35 1 35 2 6 high 5 11 241 36 I 36 2 6 high 6 15 260 37 2 1 I 1 med 1 7 214 38 2 2 1 1 med 2 4 228 39 2 3 1 1 med 3 6 219 40 2 4 1 I med 4 17 223 41 2 5 1 1 med 5 9 218 42 2 6 1 med 6 18 218 43 2 7 1 2 low 1 1 155 44 2 8 1 2 low 2 5 148 45 2 9 1 2 low 3 8 155 46 2 10 1 2 low 4 3 149 47 2 11 1 2 low 5 2 153 48 2 12 1 2 low 6 12 156 49 2 13 1 3 high 1 14 246 50 2 14 1 3 high 2 15 249 51 2 15 3 high 3 16 255 52 2 16 3 high 4 11 246 53 2 17 1 3 high 5 13 244 54 2 18 1 3 high 6 10 247 55 2 19 2 4 low 1 15 151 56 2 20 2 4 low 2 3 151 57 2 21 2 4 low 3 8 153 58 2 22 2 4 low 4 7 149 59 2 23 2 4 low 5 14 151 60 2 24 2 4 low 6 2 150 110 Appendix C Figure 4 (con't) Precision Experiment Generated By Proc Plan & Cholesterol Results February, 1993 2nd Lot 942 obs tech seq round tube pool sample ma ch chol 61 2 25 2 5 high 1 16 259 62 2 26 2 5 high 2 4 246 63 2 27 2 5 high 3 5 243 64 2 28 2 5 high 4 17 254 65 2 29 2 5 high 5 1 245 66 2 30 2 5 high 6 18 242 67 2 31 2 6 med 1 6 227 68 2 32 2 6 med 2 10 222 69 2 33 2 6 med 3 12 233 70 2 34 2 6 med 4 9 218 71 2 35 2 6 med 5 13 224 72 2 36 2 6 med 6 11 226 73 3 1 1 1 low 1 15 160 74 3 2 1 1 low 2 18 150 75 3 3 1 low 3 9 154 76 3 4 1 low 4 3 157 77 3 5 1 low 5 8 153 78 3 6 1 1 low 6 2 159 79 3 7 1 2 high 1 12 264 80 3 8 2 high 2 7 248 81 3 9 1 2 high 3 6 253 82 3 10 1 2 high 4 16 253 83 3 11 1 2 high 5 14 241 84 3 12 1 2 high 6 11 245 85 3 13 3 med 1 5 232 86 3 14 1 3 med 2 13 219 87 3 15 1 3 med 3 17 226 88 3 16 1 3 med 4 1 231 89 3 17 1 3 med 5 4 228 90 3 18 1 3 med 6 10 225 111 Appendix C Figure 4 (con't) Precision Experiment Generated By Proc Plan & Cholesterol Results February, 1993 2nd Lot 942 obs tech seq round tube pool sample ma ch chol 91 3 19 2 4 high 1 3 256 92 3 20 2 4 high 2 18 259 93 3 21 2 4 high 3 14 249 94 3 22 2 4 high 4 5 257 95 3 23 2 4 high 5 6 248 96 3 24 2 4 high 6 13 249 97 3 25 2 5 med 1 9 225 98 3 26 2 5 med 2 11 228 99 3 27 2 5 med 3 4 231 100 3 28 2 5 med 4 12 226 101 3 29 2 5 med 5 1 230 102 3 30 2 5 med 6 16 226 103 3 31 2 6 low 1 17 152 104 3 32 2 6 low 2 10 157 105 3 33 2 6 low 3 2 153 106 3 34 2 6 low 4 8 149 107 3 35 2 6 low 5 15 155 108 3 36 2 6 low 6 7 148 112 Appendix C Figure 5 Precision Experiment Generated By Proc Plan & Cholesterol Results August, 1993 Lot 941 obs tech seq round tube pool sample ma ch strvial chol 1 1 1 1 1 low 1 13 4 143 2 1 2 1 low 2 15 1 146 3 1 3 1 low 3 8 5 155 4 1 4 1 low 4 3 2 139 5 1 5 1 1 low 5 7 3 142 6 1 6 1 2 high 1 2 5 241 7 1 7 1 2 high 2 11 2 225 8 1 8 1 2 high 3 5 4 219 9 1 9 1 2 high 4 6 1 224 10 1 10 1 2 high 5 14 3 234 11 1 11 1 3 med 1 1 1 211 12 1 12 1 3 med 2 4 4 202 13 1 13 1 3 med 3 12 3 209 14 1 14 1 3 med 4 IO 5 211 15 1 15 1 3 med 5 9 2 198 16 1 16 2 4 high 1 6 5 234 17 1 17 2 4 high 2 13 1 231 18 1 18 2 4 high 3 i2 4 218 19 1 19 2 4 high 4 5 3 233 20 1 20 2 4 high 5 15 2 213 21 1 21 2 5 med 1 2 1 212 22 1 22 2 5 med 2 7 2 196 23 1 23 2 5 med 3 8 4 199 24 1 24 2 5 med 4 1 5 217 25 1 25 2 5 med 5 11 3 207 26 1 26 2 6 low 1 3 5 154 27 1 27 2 6 low 2 9 1 152 28 1 28 2 6 low 3 10 3 152 29 1 29 2 6 low 4 14 4 143 30 1 30 2 6 low 5 4 2 141 113 Appendix C Figure 5 (con't) Precision Experiment Generated By Proc Plan & Cholesterol Results August, 1993 Lot 941 obs tech seq round tube pool sample mach strvial chol 31 2 1 1 1 med 1 12 2 194 32 2 2 1 1 med 2 10 5 220 33 2 3 1 1 med 3 13 3 218 34 2 4 I 1 med 4 7 1 202 35 2 5 1 1 med 5 11 4 213 36 2 6 1 2 low 1 1 1 147 37 2 7 1 2 low 2 2 4 155 38 2 8 1 2 low 3 9 5 155 39 2 9 1 2 low 4 4 2 140 40 2 10 1 2 low 5 15 3 147 41 2 11 1 3 high 1 6 1 232 42 2 12 1 3 high 2 8 5 232 43 2 13 1 3 high 3 3 3 228 44 2 14 I 3 high 4 14 2 218 45 2 15 1 3 high 5 5 4 223 46 2 16 2 4 low 1 15 1 147 47 2 17 2 4 low 2 13 3 155 48 2 18 2 4 low 3 4 5 153 49 2 19 2 4 low 4 7 2 141 50 2 20 2 4 low 5 3 4 146 51 2 21 2 5 high 1 9 1 232 52 2 22 2 5 high 2 10 5 238 53 2 23 2 5 high 3 11 3 227 54 2 24 2 5 high 4 14 4 219 55 2 25 2 5 high 5 6 2 212 56 2 26 2 6 med 1 8 2 200 57 2 27 2 6 med 2 2 5 212 58 2 28 2 6 med 3 12 4 204 59 2 29 2 6 med 4 1 3 210 60 2 30 2 6 med 5 5 1 209 114 Appendix C Figure 5 (con't) Precision Experiment Generated By Proc Plan & Cholesterol Results August, 1993 Lot 941 obs tech seq round tube pool sample mach strvial chol 61 3 1 1 1 high 1 13 4 219 62 3 2 1 high 2 1 5 240 63 3 3 1 high 3 10 3 226 64 3 4 1 1 high 4 14 2 223 65 3 5 1 1 high 5 11 1 230 66 3 6 1 2 med 1 4 1 213 67 3 7 1 2 med 2 5 4 201 68 3 8 2 med 3 2 3 209 69 3 9 1 2 med 4 15 2 199 70 3 10 2 med 5 8 5 216 71 3 11 3 low 1 3 5 154 72 3 12 3 low 2 7 3 145 73 3 13 1 3 low 3 6 2 141 74 3 14 1 3 low 4 9 4 141 75 3 15 1 3 low 5 12 1 154 76 3 16 2 4 med 1 13 4 207 77 3 17 2 4 med 2 5 2 203 78 3 18 2 4 med 3 6 1 208 79 3 19 2 4 med 4 14 3 219 80 3 20 2 4 med 5 2 5 219 81 3 21 2 5 low 1 7 2 142 82 3 22 2 5 low 2 8 3 155 83 3 23 2 5 low 3 15 5 153 84 3 24 2 5 low 4 1 4 139 85 3 25 2 5 low 5 12 1 153 86 3 26 2 6 high 1 4 2 219 87 3 27 2 6 high 2 11 4 218 88 3 28 2 6 high 3 9 1 229 89 3 29 2 6 high 4 10 3 230 90 3 30 2 6 high 5 3 5 234 115 Appendix C Figure 6 Precision Experiment Generated By Proc Plan & Cholesterol Results August, 1993 Lot 564 obs tech seq round tube pool sample mach strvial chol 1 1 1 1 1 low 1 13 4 161 2 1 2 1 1 low 2 15 1 152 3 1 3 1 1 low 3 8 5 150 4 1 4 1 low 4 3 2 153 5 1 5 1 low 5 7 3 156 6 1 6 2 high 1 2 5 224 7 1 7 1 2 high 2 11 2 226 8 1 8 1 2 high 3 5 4 232 9 1 9 1 2 high 4 6 1 240 10 1 10 1 2 high 5 14 3 230 11 1 11 1 3 med 1 1 1 214 12 1 12 3 med 2 4 4 215 13 13 1 3 med 3 12 3 226 14 1 14 l 3 med 4 10 5 203 15 1 15 1 3 med 5 9 2 204 16 1 16 2 4 high 1 6 5 229 17 1 17 2 4 high 2 13 1 229 18 1 18 2 4 high 3 12 4 240 19 1 19 2 4 high 4 5 3 231 20 1 20 2 4 high 5 15 2 230 21 1 21 2 5 med 1 2 1 206 22 1 22 2 5 med 2 7 2 209 23 1 23 2 5 med 3 8 4 212 24 1 24 2 5 med 4 1 5 203 25 1 25 2 5 med 5 11 3 211 26 1 26 2 6 low 1 3 5 154 27 1 27 2 6 low 2 9 1 158 28 1 28 2 6 low 3 IO 3 152 29 1 29 2 6 low 4 14 4 157 30 1 30 2 6 low 5 4 2 156 116 Appendix C Figure 6 (con't) Precision Experiment Generated By Proc Plan & Cholesterol Results August, 1993 Lot 564 obs tech seq round tube pool sample ma ch strvial chol 31 2 1 1 1 med 1 12 2 217 32 2 2 1 1 med 2 10 5 211 33 2 3 1 1 med 3 13 3 211 34 2 4 1 1 med 4 7 1 211 35 2 5 1 1 med 5 11 4 213 36 2 6 1 2 low 1 1 1 153 37 2 7 1 2 low 2 2 4 155 38 2 8 1 2 low 3 9 5 154 39 2 9 1 2 low 4 4 2 155 40 2 10 1 2 low 5 15 3 156 41 2 11 1 3 high 1 6 1 230 42 2 12 1 3 high 2 8 5 230 43 2 13 1 3 high 3 3 3 237 44 2 14 1 3 high 4 14 2 227 45 2 15 1 3 high 5 5 4 230 46 2 16 2 4 low 1 15 1 156 47 2 17 2 4 low 2 13 3 155 48 2 18 2 4 low 3 4 5 152 49 2 19 2 4 low 4 7 2 152 50 2 20 2 4 low 5 3 4 157 51 2 21 2 5 high 1 9 1 234 52 2 22 2 5 high 2 10 5 223 53 2 23 2 5 high 3 11 3 230 54 2 24 2 5 high 4 14 4 228 55 2 25 2 5 high 5 6 2 238 56 2 26 2 6 med 1 8 2 223 57 2 27 2 6 med 2 2 5 206 58 2 28 2 6 med 3 12 4 212 59 2 29 2 6 med 4 1 3 221 60 2 30 2 6 med 5 5 1 206 117 Appendix C Figure 6 (con't) Precision Experiment Generated By Proc Plan & Cholesterol Results August, 1993 Lot 564 obs tech seq round tube pool sample mach strvial chol 61 3 1 1 1 high 1 13 4 232 62 3 2 I high 2 5 219 63 3 3 1 1 high 3 10 3 235 64 3 4 1 1 high 4 14 2 226 65 3 5 I 1 high 5 11 1 239 66 3 6 1 2 med 1 4 1 228 67 3 7 I 2 med 2 5 4 213 68 3 8 1 2 med 3 2 3 217 69 3 9 1 2 med 4 15 2 208 70 3 10 2 med 5 8 5 207 71 3 11 3 low 1 3 5 153 72 3 12 1 3 low 2 7 3 151 73 3 13 1 3 low 3 6 2 151 74 3 14 1 3 low 4 9 4 152 75 3 15 1 3 low 5 12 1 159 76 3 16 2 4 med 1 13 4 214 77 3 17 2 4 med 2 5 2 210 78 3 18 2 4 med 3 6 1 221 79 3 19 2 4 med 4 14 3 217 80 3 20 2 4 med 5 2 5 213 81 3 21 2 5 low 1 7 2 154 82 3 22 2 5 low 2 8 3 163 83 3 23 2 5 low 3 15 5 153 84 3 24 2 5 low 4 1 4 160 85 3 25 2 5 low 5 12 1 163 86 3 26 2 6 high 1 4 2 225 87 3 27 2 6 high 2 11 4 236 88 3 28 2 6 high 3 9 1 235 89 3 29 2 6 high 4 10 3 238 90 3 30 2 6 high 5 3 5 226 118 Bibliography Allain, C., and L. Poon. Enzymatic Determination of Total Serum Cholesterol. Clinical Chemistry, 1975; 20: 470-476. American Heart Association: 1989 Heart Facts. Dallas, Texas: American Heart Association, 1989. Bachorik, Paul S., Robert Rock, Teresa Cloey, et al . Cholesterol Screening: Comparative Evaluation of On-Site and Laboratory Based Measurements. Clinical Chemistry, 1990; 36: 255-260. Burdick, Richard K. and Franklin A. Graybill. Confidence Intervals on Variance Components. Marcel Dekker, Inc.: New York, NY. 1992. Canadian Society of Clinical Chemists: Out-of-Laboratory Measurement of the Blood Cholesterol Level. Canadian Medical Association Journal, 1990; 143: 721-723 . DeVor, Richard E ., Tsong-how Chang, and John W. Sutherland. Statistical Quality Design and Control. Macmillan Publishing Company: New York, NY. 1992. Draper, N ., and H. Smith. Applied Regression Analysis, Second Edition . John Wiley & Sons, Inc.: New York, NY. 1981 . Hernandez, R. , R. Burdick, and N. Birch. Confidence Intervals and Tests of Hypotheses on Variance Components in an Unbalanced Two-Fold Nested Design. Forthcoming in Biom. J. 1992. Lapane, Kate L. 1991 . A Statistical Study of Cardiovascular Disease Variables and Intervention Analysis. M. S. Thesis. University of Rhode Island, Kingston. Lefebvre, R. C. Performance Characteristics of the Boehringer Mannheim Diagnostics Reflotron. Public Health Reports, 1987; 104: 266-270. McShane, Lisa M., Larry C. Clark, Gerald F. Combs, Jr. , and Bruce W. Turnbull. Application of Variance Comp0nents Methods to Laboratory Quality Control for Biochemical Measurements. 1992 American Statistical Association Proceedings of the Biopharmaceutical Section. 250-256. McShane, Lisa M., Larry C. Clark, Gerald F. Combs, Jr., and Bruce W. Turnbull. Reporting the Accuracy of Biochemical Measurements for Epidemiologic and Nutrition Studies. American Journal ojClinical Nutrition, 1991; 53 : 1354-1360. Montgomery, Douglas C. Design and Analysis of Experiments, Third Edition. John Wiley & Sons, Inc.: New York, NY. 1991. 119 Montgomery, Douglas C., and Elizabeth A Peck. Introduction to Linear Regression Analysis. John Wiley & Sons, Inc.: New York, NY. 1992. Morgan, Julie. The Multiple Risk Factor Intervention Trial, Quality Control of Technical Procedures and Data Acquisition. Controlled Clinical Trials: Design, Methods, and Analysis, 1986; 7: 1 S-202S. National Cholesterol Education Program: A Report from the Laboratory Standardization Panel on Recommendations for Improving Your Cholesterol Measurement. United States Department of Health and Human Services, Public Health Service, National Institutes of Health, 1990; Pub. No. 90-2964. National Institute of Health: Treatment ofHypertriclyceridemia. NIH Consensus Development Conference Summary. Arteriosce/erosis, 1984; 4: 296-301. Neter, John, William Wasserman, and Michael H . K1.,1tner. Applied Linear Regression Models. Richard D. Irwin, Inc.: Homewood, IL. 1983 . Neter, John, William Wasserman, and Michael H. Kutner. Applied Linear Statistical Models, Second Edition. Richard D. Irwin, Inc.: Homewood, IL. 1985. Ott, R. Lyman. An Introduction to Statistical Methods and Data Analysis, Fourth Edition. Duxbury Press: Belmont, CA 1993 . Powers, Donald M. Establishing and Maintaining Performance Claims. Arch Pathol Lab Med., 1992; 116: 718-725 . SAS Institute. SAS Language Guide for Personal Computers, Version 6. 03. SAS Institute, Inc.: Cary, NJ. 1993 . SAS Institute. SAS Procedures Guide, Version 6, Third Edition, SAS Institute, Inc.: Cary, NJ. 1992. SAS Institute. SAS/STAT User's Guide, Version 6, Fourth Edition, SAS Institute, Inc.: Cary, NJ. 1990. Searle, Shayle R., George Casella, and Charles E. McCulloch. Variance Components. John Wiley & Sons, Inc.: New York, NY. 1992. Sedor, F., C. Holleman, S. Heyden, and K. Schneider: Reflotron Cholesterol Measurement Evaluated as a Screening Technique. Clinical Chemistry, 1988; 34: 2542-2545 . 120 Sempos, C., R. Fulwood, C. Haines, et al : The Prevalence of High Blood Cholesterol Levels Among Adults in the United States. Journal of the American Medical Association, 1989; 262: 45-52. Snedecor, George W., and William G. Cochran. Statistical Methods, Seventh Edition. The Iowa State University Press: Ames, IA. 1980. Ting, Naitee, Richard K. Burdick, Franklin A. Graybill, et al. Confidence Intervals on Linear Combinations of Variance Components that are Unrestricted in Sign. Journal of Statistical and Computational Simulation, 1990; 3 5: 135-143. 121

Original Article Investigation

Abstract

Extracted data

Original Article Investigation

Abstract

Extracted data

Related items

Related items