University of Tennessee, KnoxvilleTrace: Tennessee Research and CreativeExchangeDoctoral Dissertations Graduate School12-1998A Cost-Benefit Analysis of the Expanded Food andNutrition Education ProgramJanie L. BurneyUniversity of Tennessee, KnoxvilleThis Dissertation is brought to you for free and open access by the Graduate School at Trace: Tennessee Research and Creative Exchange. It has beenaccepted for inclusion in Doctoral Dissertations by an authorized administrator of Trace: Tennessee Research and Creative Exchange. For moreinformation, please contact trace@utk.edu.Recommended CitationBurney, Janie L., "A Cost-Benefit Analysis of the Expanded Food and Nutrition Education Program. " PhD diss., University ofTennessee, 1998.https://trace.tennessee.edu/utk_graddiss/3833To the Graduate Council:I am submitting herewith a dissertation written by Janie L. Burney entitled "A Cost-Benefit Analysis ofthe Expanded Food and Nutrition Education Program." I have examined the final electronic copy of thisdissertation for form and content and recommend that it be accepted in partial fulfillment of therequirements for the degree of Doctor of Philosophy, with a major in Human Ecology.Betsy Haughton, Major ProfessorWe have read this dissertation and recommend its acceptance:Paula Zemel, Carol Costello, Robert J. PursleyAccepted for the Council:Carolyn R. HodgesVice Provost and Dean of the Graduate School(Original signatures are on file with official student records.)To the Graduate Council: I am submitting herewith a dissertation written by Janie L. Burney entitled, "A CostBenefit Analysis of the Expanded Food and Nutrition Education Program." I have examined the final copy of this dissertation for form and content and recommend that it be accepted in partial fulfillment of the requirements for the degree of Doctor of Philosophy, with a major in Human Ecology. We have read this dissertation and recommend its acceptance: A Cost-Benefit Analysis of the Expanded Food and Nutrition Education Program A Dissertation Presented for the Doctor of Philosophy Degree The University of Tennessee, Knoxville Janie L. Burney December 1998 Copyright© Janie L. Burney, 1998 All rights reserved 11 DEDICATION This dissertation is dedicated to the program assistants and Family and Consumer Science agents in the Tennessee Agricultural Extension Service, who work to improve the health and well-being of families in Tennessee through the Expanded Food and Nutrition Education Program. For many of these families, they are a primary source of nutrition information and a major influence on food and nutrition behaviors. 111 ACKNOWLEDGMENTS I would like to acknowledge the following people who helped make this research project possible: Dr. Betsy Haughton, my major professor, who guided me through the long process and served as both teacher and adviser; Dr. Gail Disney and Dr. Pat Sobrero, who were a source of encouragement and support that helped me manage a full-time job and graduate school; Dr. Ann Reed for help with statistics; and Dr. Carol Costello, Dr. Jack Pursley, and Dr. Paula Zemel, for their insight and suggestions. Lastly, I am particularly grateful to Extension Family and.Consumer Science agents and program assistants involved in this project. IV ABSTRACT Data on the economic value of nutrition education programs, such as the Expanded Food and Nutrition Education Program (EFNEP), can help decision makers choose between alternative programs based on costs and benefits. A cost-benefit analysis of EFNEP was conducted to determine if savings in food expenditures exceeded implementation costs. Costs were collected over a 6-months using expenditure reports and other records. Benefits were determined using prospective data from 3 71 females enrolled in EFNEP who completed a 24-hour food recall, behavior survey, and recorded the amount of money spent on food monthly at program entry and exit. Two treatment groups received nutrition education and one group did not receive education. One treatment group estimated food expenditures from recall and the other collected register receipts or recorded expenditures. Control group subjects reported expenditures from recall. Net present value (NPV) was calculated using cost per participant subtracted from the change in food expenditures per participant over a 5-year period at a discount rate of 7%. NPV of EFNEP was $600.52, i.e., food expenditures were reduced by $600.52 over a 5-year period. At the same time individuals reduced food expenditures, they increased intakes of iron, vitamin C, vitamin B6, and fiber. They added less salt when cooking and more often read nutrition labels. They also reported less often running out of food at the end of the month. Findings from this research showed that EFNEP is cost-beneficial. The magnitude of the savings in food expenditures varies with how long participants retain behaviors they learned and by the rate at which future benefits are discounted. V PREFACE This dissertation was written in two parts. Part I consists of an introduction, an extensive review of the literature, and the research questions. Part 2 is the the study written in journal format. An extensive methodology is included in Appendix A to explain methods in more detail. To aid the reader, a glossary of terms is included in appendix G. VI Table of Contents Page Part 1: Introduction, Purpose, and Literature Review ....................... 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Purpose ....................................................... 4 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 The Expanded Food and Nutrition Education Program .............. 5 Choosing the Type of Economic Analysis ....................... 13 Conducting a Cost-Benefit Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 Summary ............................................... 70 Statement of the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 4 Part 2: A Cost-Benefit Analysis of the Expanded Food and Nutrition Education Program ........................................................... 86 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 7 Methods ...................................................... 90 Subjects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 Research Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 Sample Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 Statistical Analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 Comparing Costs and Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 Sample ................................................ 104 Program Costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 Program Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Net Present Value and Array of Effectiveness Measures ....... 119 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2 Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 Appendix A: Extensive Methodology ............................... 138 Appendix B: Participant Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 Bl. Family Record Form .................................. 165 B2. Dietary Recall Form .................................. 167 B3. EFNEP Survey Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 B4. Food Expenditure Record ......................... ·. . . . . 170 Appendix C: Instructions for Conducting a Cost Benefit Analysis ......... 172 Appendix D: Participant Consent Forms ............................. 182 vu D 1. Consent Form for Group A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 D2. Consent Form for Group B ............................. 185 D3. Consent Form for Group C .... ......................... 187 Appendix E: Grant and Contract Budget and Expenditure Report ......... 189 Appendix F: EFNEP Costs by Month ............................... 191 Appendix G: Glossary of Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 Vita 199 Vlll List of Tables Table Page 1. Expected outcomes as a result of participating in EFNEP . . . . . . . . . . . . . . . . . . 7 2. Participation rates by county for each group. . . . . . . . . . . . . . . . . . . . . . . . . . 105 3. Demographic characteristics of the sample: race/ethnicity and residence. . . . . 106 4. Demographic characteristics of the sample: household composition ......... 108 5. Participation in public assistance programs . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 6. Number oflessons received by subjects in group A and B. . . . . . . . . . . . . . . . 110 7. Type of instruction received by subjects in group A and B ................ 110 8. Direct costs for 12 months. . ..................................... 112 9. Average scores for food and nutrition related practices at program entry for the combined experimental group and control group. . . . . . . . . . . . . . . . . . . . . . . 114 10. Comparison of the difference between average number of meals and servings from food groups at program entry and program exit for combined experimental group and the control group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 11. Comparison of the difference between average nutrient intakes at program entry and program exit for the combined experimental group and the control group. 117 12. Average scores for food selection and preparation practices at program exit for the experimental groups and the control group. . . . . . . . . . . . . . . .. . . . . . . . . . . . 118 13. Net Present Value (NPV) from savings in food expenditures as a result of rt. . t' . EFNEP 120 pa 1c1pa 10n 1n . . ....................................... . 14. An array of food and nutrient outcomes for the combined experimental group that improved as a result of participation in EFNEP . ....................... 121 15. An array of food resource management, food selection, and food preparation outcomes for the combined experimental group that improved as a result of rt. . t' . EFNEP 121 pa 1c1pa 10n 1n ........................................ . IX 16. The nonequivalent control group design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 17. Activities related to the implementation ofEFNEP ..................... 147 18. Direct and indirect costs ofEFNEP during May, June, and July ........... 192 19. Direct and indirect costs ofEFNEP during August, September, and October ..................................................... 194 X Part 1 Introduction, Purpose, and Literature Review 1 INTRODUCTION The Problem: Does the Expanded Food and Nutrition Education Program help families use food resources wisely and improve nutritional intake? And, what does it cost to produce program outcomes? Research suggests that nutrition education programs, such as the Cooperative State Research, Education, and Extension Service's (CSREES) Expanded Food and Nutrition Education Program (EFNEP), improve the quality of diets of low-income individuals by improving their food buying and preparation practices (1 -2). Living below the poverty level makes it difficult to consume enough food with adequate nutrients. Diets must be planned wisely and foods must be selected and prepared carefully to make resources meet nutritional needs. When families cannot make their resources last long enough to meet their needs for food, they do without some foods and consume a disproportionate amount of others creating nutritional imbalances. Some families rely on temporary stop-gap measures, such as food banks and pantries in their communities (3 ). National food and health surveys have shown that certain populations, such as low-income women and children, consume inadequate diets ( 4). Over consumption of some nutrients, such as total fat, saturated fat, cholesterol, and sodium, also is a problem for low-income women and children (5). Under- and overnutrition, both fonns of malnutrition, are considered important issues because they impact health and well-being. Lack of food interferes with physiological and cognitive function ( 6) and creates social problems (7). Undemutrition also is associated with poor pregnancy outcome, impaired growth, anemia, and chronic infections (8-1 0). Over-consumption of some nutrients has been implicated in chronic disease ( 1 1 ). 2 The United States currently responds to malnutrition through public and private programs that distribute food or resources to buy food and through nutrition education programs. All of these programs have a cost and use resources that could be used for other purposes (12). Any time resources for programs are limited, decision makers must choose among alternatives to distribute those resources. Cost-benefit analysis can be incorporated into program evaluation to help decision makers allocate resources. Allocating resources to programs that are shown to be effective will help solve the problem of malnutrition. The results of a cost-benefit analysis typically are expressed in a benefit-cost ratio, where benefits (measured in dollars) are related to the cost of an intervention (12). Costbenefit analyses have been conducted for the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) (13 -15). Cost-benefit analyses for WIC have shown that $2.84 to $3 .13 was saved in Medicaid expenses for newborns during the first 60 days for every dollar spent to implement WIC (13-14). Evaluations conducted in EFNEP (1 -2) have examined program effectiveness, such as changes in knowledge, attitudes, and behaviors, but few have integrated measures of effectiveness with the costs of implementing the program. This would allow decision makers to choose between spending resources on EFNEP or using resources for other purposes. 3 PURPOSE The purpose of this research was to determine the following: 1) Does participation in EFNEP help households use their food resources wisely? 2) Does participation in EFNEP help households increase their nutrient intake? 3) How much does it cost to change food resource management practices ofEFNEP participants? According to Kennedy (16, p. 7), cost analysis in nutrition education "is an area that is ripe for very applied research. The science is there, but the application is lacking at the moment." This study was designed to apply cost-benefit analysis to EFNEP. 4 LITERATURE REVIEW The Expanded Food and Nutrition Education Program Overview The Expanded Food and Nutrition Education Program ( EFNEP) provides nutrition education to low-income families with children. EFNEP was funded in 1970 with Congressional appropriation under the Smith-Lever Act (Smith-Lever Act - U.S. C. 34 1-348, Public Law 9 1- 127). It is administered by the Cooperative State Research, Education, and Extension Service ( CSREES) of the United States Department of Agriculture (US D A) in cooperation with land grant universities and state and county governments that make up the Cooperative Extension System. Participants in EFNEP are taught individually or in groups by paraprofessional program aides who, when possible, are indigenous to the audience. Program aides have a high school diploma or equivalent and receive supervision and training from professionals employed by the land-grant universities. Professionals are college/university graduates with training in Family and Consumer Sciences. To be considered an EFNEP participant, individuals must enroll for a series of lessons on foods and nutrition. How many lessons and the determination that the participant has reached his or her educational objectives and is ready to graduate is decided by the paraprofessional responsible for teaching the participant and her supervisor. In 1995, 2,635 individuals in Tennessee completed the program because they met their educational objectives (17). The majority (84%) completed 7 to 18 lessons. Eighty percent (2,3 13) completed the program in less than six months ( 17). 5 "The objectives of EFNEP are to help low-income families and youth acquire the knowledge, skills, attitudes, and changed behavior necessary for nutritionally sound diets and to contribute to their personal development" (18, p. 5). Nutritionally sound diets are defined as diets that meet the minimum number of servings for each food group recommended in the Food Guide Pyramid (19). Families that participate in EFNEP are expected to improve food security and their health by: 1) improving their ability to select and prepare foods that meet their nutritional needs, 2) improving their resource management abilities, and 3) improving their food safety practices. The outcomes resulting from enrollment in EFNEP could be described using a hierarchal approach (Table 1) (20). Enrollment in EFNEP means low-income families would receive nutrition education, then they would improve their access to nutritious food, and then their health would improve. Intermediate outcomes would be improved food selection and preparation behaviors, improved nutrient intakes, improved food resource management behaviors, and improved food safety behaviors. Ultimately, participants would have improved access to nutritious foods at all times and improved health. Ultimate outcomes may include reduced risk for chronic diseases associated with diet, such as heart disease, cancer, stroke, diabetes. Improved diets may help prevent other health conditions such as obesity, hypertension, and osteoporosis that reduce quality of life and productivity. Each land-grant university receives federal funds to administer EFNEP following submission and approval of a management plan called a Plan of Work. This management plan is developed to assess program operations, accomplishments, and effectiveness. It is the responsibility of each state to meet program objectives in the most cost-efficient 6 Table 1. Expected outcomes as a result of participating in EFNEP Program Hypothesis Intermediate Ultimate Outcomes Outcomes If low-income families Improved food resource Improved access to receive nutrition education, management behaviors nutritious food then they will improve their food security and their Improved food selection Improved health health. and preparation behaviors Improved nutrient intakes Improved food safety practices manner (i.e., achieving program objectives with as many families as possible with the resources allocated). Program sites are selected based on the greatest audience potential using the number of paraprofessionals and professionals needed to achieve program goals with minimal administrative costs. The national EFNEP Reporting System (ERS) is used to collect and summarize data for county, state, and national program management (21). Program costs and evidence of effectiveness are reported to USDA using data from ERS and other supporting information, such as success stories and descriptions of exemplary programs in yearly progress reports. These reports provide accountability information for national program justification and funding. Accomplishments are reported also to state and county stakeholders to solicit funding and other contributions. 7 Several researchers have evaluated the effectiveness of EFNEP (1, 2, 22-24). Assessments have been conducted on dietary improvements, increases in nutrition knowledge, and improved food behaviors. Effectiveness of EFNEP Dietary improvements. Using a dietary score derived from the Basic Four Food Group 2-2-4-4 diet pattern for 24-hour recalls, Torisky et al. (24) showed that the number of homemakers who consumed optimal diet patterns of 2-2-4-4 increased from 17 of 224 (8%) homemakers at program entry to 83 (37%) at program-completion. Length of program enrollment varied from 6 to 18 months. Seventy-two (40%) of 180 homemakers had sustained dietary improvements by 6 to 36 months. The greatest improvements were seen in the milk, fruit and vegetable, and bread and cereal groups. Dietary improvements also were noted in an evaluation of24-hour recalls of355 EFNEP homemakers in California (23). Using a dietary score also derived from the Basic Four Food Group 2-2-4-4 diet pattern, researchers showed that mean total dietary scores for homemakers enrolled in EFNEP for six months increased significantly, while mean total dietary scores of a comparison group did not. Significant increases were noted for the milk, fruit and vegetable, and protein groups. Twenty-four hour recalls of 50 EFNEP homemakers in New York were analyzed at entry, graduation, and one year follow-up to assess caloric and nutrient content ( 1). Mean values for calories and other nutrients, except iron, exceeded or were within 80% of the age/gender-specific Recommende� Dietary Allowances (RDA) published in 1980 (25) 8 for each o f the three assessments . S ign ificant reduct ions in the amount and percentage o f ca lories from fat were reported from program entry to graduat ion. Mean intakes o f prote in, calc ium, and v itamin A were s ign ificantly lower one year a fter graduation compared to graduat ion . Data from 24-hour reca lls of 3 5 homemakers were analyzed in Wyo ming (2) and compared to the Bas ic Four Food Group pattern o f2 -2 -4-4 . Data were co llected at program entry and a fter 20 lessons. Ana lys is showed sig nificant increases in the number o f se rvings from the m ilk, b read and cereal, meat, and the fruit and vegeta ble group a fter the partic ipant had rece ived 20 lessons. All posttest mean intakes met the 2 -2 -4-4 pattern except for the fruit and vegeta ble grou p w hich was 3.3 se rvings. Amstut z and D ixon (26) showed that EFNEP part ic ipants improved the ir d iets w hile enro lled and ma inta ined some o f the im provements for at least 20 months a fter they graduated. Part ic ipants increased the ir intake o f milk , fruits, vegetables, breads, and cereals. Those who had excess ive intakes of fat and sugar be fore enro llment reported reduct ions in these com ponents by graduat ion and fo llow -up 20 months later. Cox et a l. (27) exam ined the d ieta ry intakes o f EFNEP part ic ipants be fore and a fter rece iv ing a 18 -lesson educat ional ser ies des igned to reduce t he ir r isk for cancer through d ieta ry and lifestyle changes. Those who rece ived the educat ional ser ies s igni ficantly decreased the ir intake of total fat (P=0.0003) , saturated fat ( P=0.000 1), se rvings from the meat group (P=0.05) , and se rvings from the fats/sweets group (p =.0005). The exper imental grou p also consu med s ign ificantly more v ita min E , d ietary fiber, and se rvings from the vegeta b le and fruit groups . 9 Nutrition knowled ge . Nut rition knowledge was assessed also in the Bri nk and S abal ( I) study. Pa rticipants had signi ficant inc reases at p rog ram entry, g raduation , and one yea r follow -up. Multiple -choice questions assessed knowledge of the need fo r b reads and ce reals and dairy p roducts, good sou rces · of i ron, and the frequency with w hich ca lcium -rich food sou rces should be included in the diet. Colo rado EFNEP homemake rs (22) also showed im provement in nut rition k nowledge following 18 months of participation. Almost all of the 1,960 homemake rs su rveyed showed inc reased knowledge about selection of nut ritious snacks compa re d to 38 % at en rollment . Knowledge of the desi rable numbe r of se rvings from the Basic Fou r Food Grou ps inc reased from 30 to 99 %. Homemake rs also showed imp rovements in knowledge a bout the importance of b reak fast . EFNEP homemake rs in Wyoming (2 ) sco red highe r on multip le -choice tests adm iniste red a fte r 20 lessons compa red to tests given at p rog ram entry. Homemake rs showe d imp rovement in knowledge conce rning food sho J?ping and food selection. Pretest sco res on classi fying foods into food g roups ranged from 8 to 9 1 % compare d to posttest sco res that ranged from 54 to 100%. Howeve r, less than half of the homemake rs could i denti fy co rrect se rving s izes fo r a ll of the food groups on the pos ttest. Homemake rs signi ficantly inc reased thei r ability to choose the best sou rces of vitamin C , calcium , and iron acco rding to posttest sco res . They also could identi fy p rope r cooking methods to retain nut rients in vegeta bles . Food behavio rs . Pa rticipants in the Co lo rado EFNEP study (22) also showed signi ficant imp rovements in food behavio rs . Less than 7% of homemake rs repo rted 10 always serving nutritious snacks at program enrollment compared to 3 7% at graduation. The number of homemakers who always used a food budget also improved from 1 5 to 55% from ent� to graduation. Homemakers improved food shopping skills (e.g. , menu planning and shopping lists) and food safety and sanitation skills. In the New York study (1), participants improved 10 out of 1 2 behaviors surveyed by the time they graduated from EFNEP. They improved their food preparation, food safety, and shopping skills. These changed behaviors were sustained at least until followup one year later. Murphy et al. (28) examined performance in nutrition-related practices (food storage and safety, kitchen sanitation, and food money management) as a furiction of how long the homemaker participated in EFNEP (six months, one year, eighteen months, and two years or longer). There was a trend toward improved performance among those who had participated six months. The greatest improvements were seen in the first six-months compared to other periods. There was some regression among homemakers who continued after two years. Similarly, Green et al. (29} found improvements in nutrition practices, knowledge, and attitudes, but further gains were minimal after the first year of enrollment. Limitations of evaluations. In general, results from these published studies showed improvements in dietary intake, nutrition knowledge, and food behaviors, but they have several limitations. Only the Del Tredici et al. (23), Amstutz and Dixon (26), Cox et al. (27), and Green et al. (29) studies included comparison groups and the New York study had a small sample size. Few evaluations used random selection of subjects. The ability 11 to gene ral ize resul ts from the New Yo rk s tudy to Tennessee is ques tiona ble because o f the hig h pe rcen tage o f African- Ame rican (62%) and Hispan ic-Ame rican ( 34 %) partic ipan ts , bu t not whi tes. All partic ipan ts surveyed lived in u rban a reas . In Tennessee in 199 5, 61 % o f pa rticipan ts we re white and 66% l ived in towns o fless than 50,000 people or on farms ( 17) . Nationw ide , 40% o f pa rticipan ts we re whi te , 36% we re African- Ame rican , and 18 % we re Hispan ic- Ame ricans (30). Fo rty- five pe rcen t l ived in cen tral c ities of ove r 50,000 and 55% l ived in towns o fless than 50,000 people o r on farms ( 30). Because o f these lim ita tions , it is impo rtan t to evalua te effec tiveness a t va rious p rog ram s ites con tinually. S ince its incep tion , nume rous unpu bl ished stud ies have been conducted on EFNEP, such as national s tudies in itia ted by the Ex tens ion Serv ice ( 31-32) , s tate coope rative ag reemen t projec ts funded by Ex tens ion ( 33- 34) , and s tate - o r indiv idual ini tia ted s tud ies , includ ing those conduc ted to ful fill re qu iremen ts fo r academ ic deg rees ( 35- 36). T hese s tudies have exam ined : pa rap rofess ionals ' c ha rac te ris tics , roles , and competenc ies ; p rog ram me thodology; p rog ram developmen t p rocess ; prog ram implemen tation ; p rog ram impac t evalua tion ; and mul ti-agency coope ration in p rog ram del ivery. EFNEP has been shown to be effec tive , bu t a t what cost? S hould resou rces be used fo r o the r pu rposes ? Some stud ies have exam ined the cos t-effec tiveness ofEFNEP in rela tion to p ilo t p rojec ts that implemen ted va rious teac hing tec hn iques (33-34 , 37). F �r example , Hannold et al . ( 3 8) conduc ted a cos t-e ffec tive analys is o f three types of nu trition del ivery sys tems that in tegra ted telev is ion , ma iled lessons , telephone ins truction , and tradi tional one -on -one ins truction by pa rap ro fessionals. Although all app roaches we re 1 2 co nsid er ed effecti ve, the traditio na l o ne-o n-o ne i nst ructio n pro ved the least cost -effecti ve whil e the dir ect mai l and t elevisio n with no p erso na l co ntacts by paraprofessio nals was the most cost- effecti ve. Thes e studi es exa mined new d eli very strat egi es · a nd not a n o ngoi ng program with typical op eratio ns. A r evi ew of lit eratur e r evea ls no pu b lished studi es that exa mine the costs of imp lem enti ng a n o ngoi ng EFNE P program i n r elatio n to b enefits expr ess ed i n mo netary t erms . Choosing the Type of Economic Analysis Eco nomic ana lysis r efers to the us e of cost t echni qu es i n program eva luatio n. It go es b eyo nd the usua l program evaluatio n that measur es impact o nly, b ecaus e it pro vides both impact and cost i nformatio n. D ecisio n-mak ers who a llocat e r esourc es need to k now both program impact and cost . Resourc es may not b e a llocat ed for a program if the costs ar � too high no matt er how effecti ve the program . Co nvers ely, a progra m that is i neffecti ve shou ld not b e fund ed sim ply b ecaus e it is inexp ensive. Accordi ng to Sp lett (3 9), programs with the highest pot entia l for achi evi ng outcom es shou ld r ec ei ve the r esourc es, particu lar ly when r esourc es ar e scarc e. Both m edica l a nd huma n s er vic e syst ems op erat e with scarc e r esourc es . Ther efor e, cost has b ecom e a n ess entia l part of program eva luatio n. Examples of Economic Analysis in Literature Eco nomic a na lysis in hea lth car e. Eco nomic ana lysis has b een us ed i n hea lth car e to d et er mi ne wh�t interventio ns ar e most effecti ve gi ven a fix ed amou nt o f r esourc es ( 40). 1 3 As medical technology continues to advance, efficacy is no longer the only consideration. Decision makers need to know which technologies provide the most benefit for the cost. In medicine, economic analysis lends itself best to population-based approaches with the objective of maximizing benefits for persons in a target population. In clinical services, physicians are less likely to use economic analysis because their objective is to the maximize the health of individual patients regardless of how it effects other patients (40). In health care economic appraisal, randomized controlled trials have been used as evidence of effectiveness ( 41 ). Evidence of effectiveness from retrospective clinical trials was combined with secondary sources of cost data, such as published literature, insurance claims databases, and expert opinion to develop summary measures of efficiency using a decision analysis model (42). More recently, there has been considerable interest in collecting prospective data as part of clinical trials so that both costs and benefits could be collected on the same patients. In one example of cost-effectiveness analysis, Greene et al. ( 43) concluded that community long-term care services were a potential cost-effective alternative to nursing home care. Previous studies of the effect of community long-term services suggested that they were less cost-effective than other alternatives. Greene et al. (43 ) developed a formal decision analysis model used to determine long-term care costs that took into consideration the probability that individuals would enter a nursing home or need particular community services. Individuals, rather than programs or services, were the unit of analysis. Costs were determined for nursing home care and for each major 14 category of co mmunity serv ice . Marginal econo mic analysis was used to esti mate e mpir ically how much money could be saved for every dollar spent on the co mmunity serv ices . Kwan -Gett et al. (44) studied the cost -e ffectiveness of prevaccination testing for hepatitis B vi rus in preadolescents and adolescents . A decision analysis model was developed to co mpare prevaccination testing for hepatitis B, no testing, and testing at the sa me ti me as the first vaccine dose . The main outcome measures were rate of co mplete vaccination, cost of testing and vaccination for each cohort, and cost per patient protected fro m hepatitis B vaccine . The authors concluded that no testing was the most cost e ffective strategy because, co mpared to vaccination without t esting, prevaccination with testing increased costs by $2.9 million for 100,000 patients and decreased the rate of co mplete vaccination by 22%. Cost -bene fit analysis was used to deter mine how favorably a protocol for studying patients with squamous intraepithelial lesions on Papanicolaou smears co mpared to other screening and treatment progra ms ( 45). Researchers calculated costs based on a model protocol and deter mined a cost per year of life saved fro m death due to invasive cervical carcinoma . They concluded that their model protocol co mpared favorably to other preven tative met hods and that the cost of pe rfor ming colposcopy on patients using a Papanicolaou smear with low grade squamous intraepithelial lesions was very low relat ive to its e ffectiveness. Neri et al. ( 46) used cost -bene fit analysis to deter mine the net cost -bene fit of two educational progra ms on asthma . One progra m consisted of sel f-reading a boo k of asthma 1 5 and one program was s ix lessons based on the same boo k.let plus educat ional v ideotapes. T he net cost -bene fit rat io o f the sel f-read ing program was I :2 . 5 (a sav ings of $2 .50 for ever y dollar spent) compared to 1 :2.6 ( $2 .60 saved for ever y dollar spent) on the lessons and v ideotapes . Econom ic analys is in nutr it ion . An important goal of the Amer ican D ietet ic Assoc iat ion (ADA) is to expand and secure stronger financ ial support for nutr it ion care serv ices. However , there are a l im ited num ber o f qual it y stud ies of econom ic anal ys is in nutr it ion care serv ices ( 4 7). G ilbr ide et al . ( 48) concluded from a survey of d ietar y managers and diet it ians that they d id not use econom ic analys is in t he ir workplace , alt hough they bel ieved that it would be requ ired to just ify serv ices in the future. In a cr it ical examinat ion o f econom ic analys is pu bl ished in nutr it ion care serv ices , Splett (4 7) concluded that , l ike research pu bl is hed in med ical l iterature , few had used t rue economic anal ys is. L imitat ions were noted due to poor research des ign (lack of random ass ignment or stat ist ical controls) , inade quate sample s izes , select ion b ias , the absence o f deta iled cost calculat ions , short t ime frames , and l im ited scopes ( fa ilure to determ ine all o f the d ire ct and ind irect impacts) (4 7). In one cost-e ffect ive anal ys is involv ing prenatal nutr it ion se rvices , Splett et al. ( 49) calculated costs of se rvices us ing standard cost-account ing pr inc iples and measured outcomes based on wr itten qual ity assurance standards. In a health department sett ing , nutr it ion ser vices were del ivered for $72 per cl ient. The cost in a hosp ital -based program was $ 12 1 per cl ient . Ninety-t hree percent of cl ients in t he ho .sp ital improved the ir d iets as a result o f the program compared to 8 7.5% at the health department. Sevent y-one percent 1 6 of clients from the hospital program achieved adequate weight gain compared to 31 % at the health department. When total cost was divided by the number of clients who improved their diet, the city health department was more cost-effective than the county hospital. However, when the total cost was divided by the number of clients who achieved recommended weight gain, the county hospital was more cost-effective than the city health department. Splett (50) also compared the cost-effectiveness of treatment alternatives to reducing cholesterol levels in adults at risk for coronary heart disease. Changes in blood cholesterol levels before and after intervention were used as measures of effectiveness to compare the impact of education only, education and drugs, and drugs only. Costeffectiveness ratios were determined for each intervention and were expressed as dollars per one percent of cholesterol change. Data showed that a drop in cholesterol of one percent cost $1 6.60 in the education only intervention, $54.1 1 in the education plus drugs intervention, and $50. 54 in the drugs only intervention. At least three cost-benefit analyses have been published of WIC (1 3-15). In each study, Medicaid and WIC data files were linked to birth certificates. Costs were calculated according to total value of food vouchers redeemed and administrative costs. Using the infant's Medicaid identification number, all Medicaid claims paid for any service were tracked for 30. days in the Schramm study (15) and 60 days in the Buescher (1 3 ) and Mathematica (14) studies. The difference between Medicaid costs for WIC participants and non-WIC participants were determined and compared to the WIC costs. In Missouri, Medicaid savings were similar to WIC costs. ($. 83 was saved in Medicaid costs for every 17 WIC dollar spent) (1 5). In North Carolina, South Carolina, Florida, Texas, and Missouri, $1 .77 to $3.1 3 was saved for every dollar spent (1 4). The Buescher study (1 3), also in North Carolina, reported a savings of $2. 91 in Medicaid costs for every dollar spent on WIC. Instead of expenses related to birth weight, Montgomery and Splett ( 51) examined the reduction in Medicaid expenditures for breast-fed infants enrolled in WIC. Costs to implement WIC included redeemed vouchers and administrative expenses. Benefits were determined from Medicaid expenditures for health care incurred in the first 1 80 days of life. Researchers concluded that breast-feeding infants enrolled in ·wic saved $478 in WIC costs and Medicaid expenditures for the first six months of life compared to formulafed infants. After the formula manufacturer's rebate, the difference was $1 12 per infant. Brauer et al. (52) demonstrated the economic benefit of nutrition counseling for non-hospitalized patients with Crohn's disease using cost-effectiveness analysis. Fifty-nine patients received nutrition education once a month for six months with a net savings of $1 64 per person. Savings resulted from fewer drugs, fewer missed work days, and decreased hospitalization. This study was one of few economic analyses in nutrition that used a control group. Types of Economic Analyses Four types of analytic techniques typically are used in economic analysis: costfeasibility, cost-benefit, cost-effectiveness, and cost-utility. Each technique is useful if used appropriately for specific applications (1 2). 18 Cost-feasibility analysis. Cost-feasibility is a limited form of economic analysis that considers only the cost of an intervention and not the outcome. It is used to determine if an alternative can be considered based on the costs and not whether the alternative should be chosen ( 12) . Cost-benefit analysis (CBA). Cost-benefit analysis also compares alternatives in terms of cost and outcomes, with both costs and outcomes are expressed in monetary terms (12). The Medicaid savings associated with prenatal WIC participation is an example of cost-benefit analysis. An estimated $ 1 .77 to $3 . 1 3 in Medicaid savings were reported for every dollar spent on a prenatal woman participating in WIC (14). Every alternative can be examined on its on merits, because outcomes are expressed in dollars ( 12). Alternatives also can be compared when their goals are not similar. A nutrition education program, for example, could be compared to a smoking cessation program. The disadvantage of CBA, however, is the difficulty determining pecuniary values for some outcomes { 12). Outcomes, such as improvements in self-esteem or changes in attitude, cannot be expressed in dollars easily. Cost-effectiveness analysis (CEA). In cost-effectiveness analysis, the cost of an intervention is expressed in dollars and the effect of the intervention is expressed as some outcome or set of outcomes { 12). For example, this outcome may be increasing test scores, reducing the risk of chronic disease, or reducing the risk of hunger. These outcomes are not expressed in monetary terms. 1 9 The strength of CEA is the ability to use effectiveness data normally gathered from an educational evaluation in combination with program costs (1 2). In many cases, it requires data that should be available from program evaluation. A major disadvantage of CEA is the inability to compare programs that do not have similar goals. In order to compare different programs and use CEA as it is intended, comparisons can be made only if the interventions have similar goals and the same measure of effectiveness is used to assess all the interventions (1 2). CEA, therefore, can not be used to determine the inherent worth of a program. It only can be used to evaluate alternatives being considered to accomplish the same goals ( 12). Cost-utility analysis. Cost-utility analysis evaluates alternatives in �erms of cost and the estimated values of their outcomes ( 1 2). Assessments of effectiveness and their values are estimated and, therefore, are subjective. The advantage of cost-utility analysis is the variety of quantitative and qualitative data that can be used to assess outcomes and estimate their value, compared to CEA that relies on specific types of quantitative data ( 12). Many outcomes can be included in the evaluation even when the data are less than perfect. The disadvantage is the subjectivity of the measures of effectiveness and their value that make cost-utility analysis difficult to replicate (1 2). Another evaluator may have different results despite using the same information and methodology because of the subjective nature of the assessments. Economic analyses help decision makers allocate resources by comparing alternative investments ( 12). Both costs and outcomes are part of comprehensive program evaluation and must be taken into consideration when choosing among 20 alternatives. A comprehensive program evaluation consists of three steps: 1 ) determining if a program was implemented as planned, 2) assessing actual program outcomes against intended outcomes, and 3) determining if funds were used efficiently to carry out an effective program (53). The outcomes to be measured for EFNEP in this research are related directly to the problem of food resource management and dietary adequacy, are intermediate outcomes, and fall into two domains: 1 ) food resource management and 2) food selection and preparation. Most evaluations of EFNEP have included process evaluation and assessment of outcomes, but few have determined if funds were used efficiently based on outcomes. To conduct a economic analysis of EFNEP, it is important to identify the problem, possible interventions (alternatives) that might address the problem, the audience for which the analysis is conducted (the perspective), and to select the appropriate analytic technique to use (1 2). Identification of the Problem In 1 989, approximately 13% of American families lived below the poverty threshold established by the United States Department of Health and Human Services (54). Over 21 .4% of families in Tennessee lived below the poverty level in 1 989 (55). Low-income, high housing costs, and other factors make it difficult for families to obtain sufficient amounts of nutritious foods (56). An estimated 20 to 3 0 million Americans are hungry or are at risk of hunger at any given time; about 1 2 million of these are children under 1 2 years of age (57-58). Hunger 21 has been defined from a physiological standpoint as the physical sensation resulting from an acute, current lack of food (8). According to Wehler et al. (59) hunger is viewed differently in industrialized countries. Instead of an acute problem it should be viewed as a chronic problem that can be assessed from the availability of food resources. Over time hunger may produce physiological, social, and economic problems. Lack of adequate, nutritious food affects physiological function preconceptionally and at every stage of the life cycle. It is associated with poor growth and development, and the development of chronic disease (60). Hunger impairs cognitive development and, therefore, the ability to learn (6). Billions of dollars are spent annually to combat hunger in the United States by government and private organizations. Hunger is a public issue and is part of public policy. Becau�e it is part of.public policy, many resources have been devoted to defining and measuring hunger (61). To shift the phenomenon of hunger from an individual perspective to a broader, societal perspective, hunger has been defined in terms of food security/food insecurity. Food security is defined as the access to enough food for a healthy life by all people at all times ( 62). At a minimum, this includes: I ) the ready availability of nutritionally adequate and safe foods and 2) the assured ability to acquire personally acceptable foods in a socially acceptable ways (62). Food insecurity occurs either when the availability of adequate, safe foods is limited or uncertain or when there is the inability to acquire personally acceptable foods in socially acceptable ways {62). When families cannot make their resources last long enough to meet their needs for adequate amounts of nutritious and acceptable food, they do without some foods and 22 consume a disproportionate amount of others, creating nutritional imbalances that affect health. Both inadequate nutrient intake and over- consumption of some dietary components have been associated with health problems. Heart disease, cancer, stroke, and diabetes, four of the 1 O leading causes of death in the United States, are associated with diets that are too high in calories, fat, saturated fat, cholesterol, and sodium (I I ). Too few fiber-containing foods also are associated with these conditions ( I I ). Inadequate intake of calcium may be associated with osteoporosis ( 11 ). In all, these diet related health conditions, cost society an estimated $250 billion in medical costs and lost productivity a year (63). Several health conditions related to diet occur more frequently among low-income individuals compared to individuals in higher socioeconomic strata. Low birth weight, �owth retardation, iron-deficiency anemia, and obesity are health conditions that are influenced by nutritional status. These are risk factors for diseases that can lead to debility and death. Low birth weight is associated with the increasing numbers of children afflicted with cerebral palsy and other birth defects (64) that have an estimated cost of $8 billion in the United States (65). Growth retardation is associated with decreased mental development (66). Iron-deficiency anemia in children, in addition to lowering their resistance to infection, can affect psychosocial and motor development adversely (67). Obesity is a risk factor for cardiovascular diseases, some cancers, stroke, and diabetes mellitus ( I I ). In addition to these health conditions, there are notable differences in health behaviors between low-income individuals and those with incomes above the poverty 23 level. For example, low-income women are less likely to breast-feed than other women (68) . The real problem is how do we reduce the risk of food insecurity for those 20 to 30 million Americans? The root cause of food insecurity is insufficient resources to buy food. In some instances lack of financial resources is a barrier to consuming an adequate diet no matter how well a family manages its resources. To have an impact on these families, the issue of poverty needs to be addressed at the socioeconomic and political level. For example, standards used to determine food stamp benefits should be addressed. USDA uses the Thrifty Food Plan, the lowest cost plan, to determine the type and quantity of food needed to meet minimum dietary standards for the Food Stamp Program. A study of the 1983 Thrifty Food Plan showed that the plan did not meet current nutritional recommendations for some nutrients and dietary components (69). It uses consumption and cost data from the 1977-78 Nationwide Food Consumption Survey, considered to be out-of-date, and dietary standards based on the 1980 RDA (25). In addition, research has shown that food stamp benefits may not cover the costs of the Thrifty Food Plan, especially in rural areas (70). The problem addressed in this research is part of the larger problem. How effective is the Expanded Food and Nutrition Education Program at improving the diets and food-related practices of low-income families faced with insufficient resources for food and what does it cost? This research will examine how effectively EFNEP addresses the problem in relation to: 1) food resource management and 2) food selection and preparation. 24 Food re source mana gement practice s. Several food re source management pract ice s have been exam ined among low- income famil ie s. The se include partic ipat ion in food a ssi stance program s, u se of tempora ry emergency food source s, planning and shopp ing skill s, and ba sic food preparat ion sk ills. For some famil ie s, food re source s could be stretched if they took full advantage o f program s ava ila ble to them . Nutrition educat ion program s, such a s E FNE P, re fer low income famil ie s to pu bl ic a ssistance program s. Program s, such a s the Food Stamp Program and commod ity food program s, are ava ila ble to many low- income famil ie s to help them stretch the ir re source s to meet nutrition need s. However , the se program s are o f no bene fit if tho se who are el ig ible do not partic ipate . The Food Stamp Program i s a mean s-te sted program by income that serve s lo wincome per son s regardle ss of fam ily statu s, age , or d isa b ility statu s. Approx imately 27 m ill ion familie s in the Un ited State s partic ipated in Janua ry 1994 (7 1) , which repre sented about 60% of fam il ie s who were el ig ible (72). Becau se such a large proportion �f el igible indiv idual s do not part ic ipate , sociolog ist s have attempted to ident ify factor s that are a ssoc iated w ith part ic ipation (73) . Cen su s da ta show the poverty rate is higher in rural than in ur ban area s (74). Average be fore-tax income o f single-parent fam il ie s wa s reporte d to be 13 % lower in rural area s compared to ur ban area s (75) . However , indiv idual s who partic ipate in the Food Stamp �rogram are more likely to l ive in ur ban rather than rural area s. A 1990-92 Con sumer Expend iture Su rvey showed that fewer single-parent fam il ie s (24 %) rece ived food stamp s in rural area s compared to 36% o f single-parent fam il ie s in ur ba n area s (75). 25 One reason why individuals living in urban areas are more likely to participate than those in rural areas could be because they are likely to live near or be associated with others who are eligible for public assistance (73). In addition, individuals living in urban areas may receive more accurate eligibility information than those in rural areas (73 ). Households that cannot make their resources last, either from lack of money or from poor management practices, tum to temporary stop-gap measures, such as food banks and pantries, or they do without the foods they need. Smith and Hoerr (3) examined the characteristics and food management behaviors of families using food banks to determine if they were different from low-income families that did not use food banks. Families who used food banks were more likely to shop for food more often and to shop at convenience stores than non-users. Smith and Hoerr (3 ) suggested that frequent food shopping, especially at convenience stores, may be related to fact that they had to either walk or use public transportation, which made carrying large quantities of groceries difficult. Food purchased with food stamps lasted about three weeks for food bank users compared to almost the whole month for non-users (3 ). Borrowing money to buy food and sharing or trading food with others was more common among food bank users than other families. Although food stamps are intended to supplement a family's food resources and not provide all of their food needs, many families use discretionary income for other necessities (3). The types of assistance a family receives influences its management practices. Emmons (76) also reported that low-income families participating in WIC spent over 90% 26 of their food s ta mps in the firs t two weeks of the month. However, they dis tri bu ted their WIC vouchers more evenly through the month. This could be due to the fact that, in so me s ta tes, indivi dual vouchers can be spen t only wi thin a cer tain time fra me, such as the firs t two weeks and the las t two weeks . Food bank users were more likely to report meal skipping due to lack of food than non-users ( 3). Children in one ou t of five households skipped meals because there was no food. Some mo thers repor ted tha t ra ther than making their chil dren skip a meal, they the mselves would skip a meal. This supports Campbell and Desjardins ' (77, p. 167) hypo thesis tha t 1 1 mo thers ' die ts de teriora te be fore their children 's . " In some cases, mo thers repor ted tha t the en tire family ate less , al though they might no t skip meals . Fa q1ilies surveyed by Smi th and Hoerr ( 3) were si milar in income levels, which sugges ted there were o ther fac tors tha t a ffec ted the disposi tion and manage ment of inco me. Families tha t used food banks were more likely to have more chil dren and older children. The resea �chers specula ted tha t households wi th younger children may have received WIC foods tha t con tri bu te d to their resources for food. Low -income fa milies may lack plann ing and shopping skills or fail to use them to s tre tc h their resources. Some surveys on meal planning and s hopping sk ills sugges t tha t so me consumers do no t use cos t-saving techniques, such as preparing shopping lis ts, using advertise men ts for specials , clipping coupons , develo ping a budge t, and c �mparing foods using uni t pricing (78). While other reports sugges t tha t low-income fa milies do use skills developed to he lp save money on foods (79-80). 27 Un it pr ic ing in for mat io n o n g rocery she lf la be ls is a method o f co mpar ison shopp ing that help s co nsu me rs make compa rison s amo ng d iffe re nt package size s and b rand s. Va riat io n in p rice due to size is removed in unit p ric ing so that compar ison s ca n be made o n the ba sis of p rice a lone. It is e st imated that 65% of shoppe rs u se un it pr ic ing (78). Becau se low- inco me shoppe rs mu st buy food s on a sma lle r budget tha n othe r shoppe rs, it wou ld see m that they wou ld be nefit mo re by using u nit pr ice in for mat ion. Ho weve r, Boya (78 ) concluded that ind iv idual s w ith low-educat ion a nd low- income we re le ss like ly to u se u nit p ric ing tha n othe r ind iv idual s. In a mo re rece nt focu s g roup report of food cho ice s of low inco me fa mil ie s, Brad ba rd et a l. (79) found tha t low- income shoppe rs ha d deve loped eco no mica l shopp ing method s. They made shopp ing list s, checked new spape r advertisement s and sto re c ircula rs to co mpa re p rices, and shopped at seve ra l sto re s to save money. Howeve r, the re we re d iffe re nce s amo ng low- income fam ilie s in how o fte n they shopped. It ha s bee n sugge sted that frequent shopp ing lead s to highe r expe nd itu re s due to impul se buying (79 ). Re sea rche rs have exa mined the cha racte rist ic s of ind iv idua ls a ssoc iated w ith shopp ing frequency. In one study, sing le mothe rs we re o b se rved to shop le ss freque ntly tha n mar ried mothe rs, which led re sea rche rs to co nc lude that this wa s part of the rea so n fo r the ir lowe r co nsu mpt io n o f fruit s, vegetable s, a nd milk p roduct s (8 1 ). B ra dba rd et a l. (79) found that H ispa nic and white no n-Hi spa nic , low- inco me shoppe rs we re l ikely to shop mo re freque nt ly tha n Africa n-Ame rica n shoppe rs who reported they d id majo r shopp ing once a mo nth, u sually a fte r rece iv ing the ir food stamp s. Ma ny o f the e mployed, 28 low-income, white females reported they did not plan meals more than a day or two ahead and shopped several times a week. In addition to frequency of shopping, low-income buyers who received food stamps have reported that their ability to purchase a low-cost diet is affected by time of the month they chose to shop. Some shoppers believe the foods are higher in price during the time immediately following distribution of their food stamps each month (79). These shoppers felt they were unable to take advantage of lower prices at other times of the month because the were unsure that they would have the food stamps. According to USDA's Economic Research Service (80), the poor face higher food prices than the national average. Many poor people live in central cities and rural areas where stores have higher operating costs because they tend to be smaller and fewer. These factors raise the food prices for poor households by one percent of the riational average. In response, low-income households use cost-saving techniques, such as buying lower quality foods, larger package sizes, and generic brands. With the exception of vegetable and fruit juices and eggs, the poorer consumers actually pay less per unit of nearly every major food group than consumers at other income levels. Use of "convenience" foods is cited as factor that affects the low-income families' ability to purchase a low-cost diet (79). In the Bradbard et al. (79) research, low-income shoppers who were employed purchased convenience foods more often than shoppers who were not employed, because they believed they were easier to prepare. These shoppers acknowledged that convenience foods were more expensive, but they believed they tasted good and were easy to prepare. 29 These reports of the food management practices suggest that not all low-income families have poor practices and that there are factors that affect their ability to maximize their resources. There are racial and ethnic differences not only in the way low-income families shop, but also in the kinds of assistance they receive, whether or not they are employed, family composition, and where they live. Food selection and preparation. Through the nutrition monitoring activities of the United States Department of Agriculture, the United States Department of Health and Human Services, and surveys, such as the Bogalusa Heart Study (82), data are available on the dietary and nutritional status of low-income families in the United States. A comparison of the 1 977-78 and 1 987-88 Nationwide Food Consumption Surveys (NFCS) conducted by USDA indicated that red meat consumption decreased, but the largest decrease (3 1 %) was among the highest income households (83). Meat consumption decreased by only 1 1 % in low-income households who consumed 5% more than the national average. Focus groups with low-income meal-preparers supported this finding (79). Respondents reported spending a large percentage of their food stamp allotment on meat. Surveys suggest that low-income families spend over one-third of their food expenditures on meat (79). All ethnic groups considered meat as an essential part of a meal . However, African-American respondents were more adamant than other groups. Many cited that meat was always served while growing up so they never had a meal without meat. The NFCS also showed that the highest income households consumed 26% more cheese, while the lowest income households decreased consumption in 1 987-88 compared 30 to 1977-78 (83) . Egg consum ption decreased by 29 % in the wealthies t househo lds, bu t increased by 8 % in poorest households . The higher income households increased consum ption of flour and cereals by 2 %, bu t the lowes t income households decreased consum ption by 16%. Per ca pi ta consum ption of break fast c ereals was lower for the poores t households com pared to others . Consum ption of fresh vege tables rose as income rose, but consum ption of fruits declined in all but the wealthies t households (83) . Resul ts from the 1989-9 1 CSF I I indica ted that fewer adults with incomes less than $ 10,000 a year {2 1.3%) ate the recommended five se rvings of frui t and/or vegeta bles per day com pared to o ther income grou ps {29 .9 % to 36.7%) (84) . Mean intakes of all frui ts and vegeta bles were lower for these adults (3 .6) com pared to those wi th higher incomes (4 . 1 to 4 .8) . A similar pa tte rn was seen in data fro m children . As income rose, frui t and vege table intake by children increased slightly (84). However , low-income children consumed comparable amoun ts of starchy vegeta bles, par ticularly white pota toes. French fries cons ti tuted about 23 % of all vege tables and 14 % of all frui ts and vegeta bles for all income levels . Nutrition monitoring through the Continuing Survey of Food In takes of Individuals ( CSFII) has provided in forma tion on dietary intakes oflow-income children . Data from the I-day food in take assessment in the 1 987-88 CSF I I sugges ted that, overall, mean calor ic consum ption for the low-income c hildren was similar to t ha t of children at higher income levels (85) . For low-income children, average intake of nutrien ts was a bove the RDA. However , for some racial /et hnic grou ps , low-income children consumed more fa t and saturated fa t than did children in households with higher incomes . In 3 1 contrast , Tippett et al . (86) concluded from 1985-86 CSFII data that intakes were below the RDA {25) for iron and zinc . Using 1986 CSFII data, Cook and Martin (87) also reported that significantly larger proportions of low-income children had intakes below 70% of the RDA for energy, folate, iron, magnesium, thiamin, vitamin A, vitamin B6, vitamin B 12, vitamin C, vitamin E , and zinc compared to children from higher income groups. According to Radimer et al. (88), additional qualitative measures of food insecurity are more useful than income alone. In an effort to obtain more information about the relationship between food security and nutrient intake, several questions on household food security were included in the 19 89-9 1 CFS I I . Respondents were asked if their households always had 1 ) enough of the kinds and amounts of food they wanted to eat , 2) enough food, but not always what they wanted to eat, or 3) sometimes or often not enough to eat (89). Nationally representative samples in the third National Health and Nutrition Examination Survey (NHANES III 1988-9 1 ) and CSFII 1989-9 1 revealed that about 9 to 13 % of individuals in low-income families experienced food insufficiency (90). Using the food security categories and three-day average intakes suggested that mean energy intake fell from the more food-secure categories to the less food-secure categories (85). Average level of fat and saturated fat increased from the food-secure categories to the less food-secure categories . Data from 3-day average food intake assessment for children suggested that the mean level of energy intake fo� children in the less food-secure categories was lower than the mean energy i_ntake for children in the food-secure categories (8 5). However , the average fat and saturated fat intake was higher 32 among children in the Jess food-secure categories compared to the food-secure category. !he researchers suggested that it is possible that for Jess food-secure households, higherfat foods are a less expensive source of calories than other foods and these households rely on them when resources are constrained . Data from the 1989-90 CSFII using a sample of single-parent female-headed households , the majority of whom were low-income, suggested that mothers and children in these households consumed diets very similar in total calories (8 1 ). However, the single mothers consumed fewer fruits, vegetables , and milk products than other households . Their children consumed fewer fruits than children in other households . Overall, the children of married mothers consumed a greater variety of foods. Data from the 1985 and 1986 CSFII suggested that low-income women, both · single-parent and married, consumed less than adequate amounts of food energy, vitamin B6, calcium, magnesium, iron, vitamin E, folacin, and zinc (86). Emmons (76) reported differences in food and nutrient intakes of low-income families depending on the week of the month. Families consumed fewer high-protein foods, fruits , and vegetables during the last two weeks of the month compared to the beginning of the month . Total calories were lower in the fourth week compared to the first week of the month . For most families , nutrient intakes did not decrease significantly over the course of the month . Intakes of some nutrients (e.g., protein, niacin , and riboflavin) were well above the RDA in both the first and fourth week of the month. Vitamin B 6, vitamin D, magnesium, zinc, calcium, iron, and vitamin E intakes were below the RDA during both weeks . 33 Results of these surveys of low -income families and individuals suggest that many do not consume enough of some nutrients while consuming too much of others . Some researchers have studied the food ha bits oflow -income to determine why (79 , 9 0-92) . Part of the reason low -income women and children consume poorer diets than other women and children may be due to lac k of basic coo king skills as suggested by some researchers. Focus groups with WIC pa rticipants revealed they spent little time coo king on a daily basis and relied on sandwiches and convenience foods (9 1). Women in EFNEP cited the lac k of s kills n �cessary to prepare healthy foods as a barrier to eating more healthfully (92). They said they wanted to learn how to coo k vegeta bles so they tasted good. Recipes for fast , easy, and tasty foods with ingredients on hand were highly desira ble. It has been recommended that nutrition education programs address the barriers to poor fruit and vegeta ble consumption and provide practical in formation on buying , storing , and preparing them (92). National Nutrition Monitor ing and Related Research Program surveys (9 0) suggest that African -American and low -income female meal planners may be less aware of the diet -health relationships than females who are white and from higher -income households. Other researchers have suggested that the reason low -income homema kers may have trou ble ma king healthy choices is that some have difficulty translating recommendations into speci fic foods (79) . Some use nutr itional in formation on food la bels , while others have trou ble understanding the la bels and how to use the in formation . Many also believe nutr itious foods cost more money than other foods (79). 34 Data from focus groups of low-income individuals responsible for preparing meals indi cate there are several factors that determine food choices (79) . Food price was considered the most important consideration (79). This was especially true for individuals with a large number of children . The most important factor to them was making sure no one was hungry. To buy sufficient quantities of food, they were willing to sacrifice taste, which is considered a very important factor 'Yhen individuals are choosing foods (79) . In addition to food price, the preferences of children are another factor that determines what foods low-income individuals prepare (79). Some low-income parents consider their children as the biggest influence on what they buy and cook. One parent in a focus group stated, "I make sure my kids eat right even if l don 't" (79, p . 7). A third factor that determines food choices is ethnic and cultural traditions (79) . For example, despite education about cholesterol and fat, low-income families continue to eat large amounts of high-fat meat . They have learned to prepare culturally familiar foods and spouses and children enjoy eating traditional meals . In many cases family members may resist attempts to change. Survey data and focus groups have revealed that many low-income families are aware of dietary recommendations, yet knowledge is not enough to change their practices. There are many factors that influence their ch�i ces, such as budget and time constraints, lack of food preparation skills , influence of family members, and ethnic traditions. These factors place them at risk for malnutrition . Nutrition programs are designed to improve the nutritional status of low-income families by making nutritious foods available and/or teaching them how to manage their 35 food resources, and select and prepare foods. Are some programs more beneficial than others? The premise of economic analysis is that decision makers can choose alternative ways to allocate resources (1 2). Alternatives that Address the Problem The Food, Nutrition, and Consumer Service of USDA administers 15 food assistance programs targeted primarily to low-income Americans. During the first six months of fiscal year 1 996, these programs cost taxpayers $1 9 .7 billion (93). Like EFNEP, the Food Stamp Program, the Emergency Food Assistance Program, and the Food Distribution Program for Soup Kitchens and Food Banks serve a broad segment of low-income families. WIC, Commodity Supplemental Food Program (CSFP}, Nutrition Program for the Elderly, Food Distribution Program on Indian Reservations, and Nutrition Assistance Programs in Puerto Rico and the Commonwealth of the N orthem Mariana Islands benefit a more narrow segment of the low-income population that meets specific eligibility criteria in addition to income. The National School Lunch Program, School Breakfast Program, Nutrition Education and Training Program, Summer Food Service Program, and Special Milk Program are targeted to children. The Food Stamp Program. The Food Stamp Program is the largest federal food assistance program designed to help meet the basic nutritional needs of all eligible lowincome individuals and families. It is considered a safety net to protect the nutritional health of Americans and families regardless of age or disability. In the first six months of fiscal year 1 996, the Food Stamp Program served almost 25.9 million Americans per 36 month at a cost of nearly $ 12.3 billion (93). Participants are provided monthly allotments of coupons or an account they can access with a Electronic Benefit Transfer (EBT) card at the point of sale. Funds for nutrition education are included as part of the food stamp budget . Each state receives funds as part of the state food stamp administrative budget with a required nonfederal match of 50%. States can participate in the Food Stamp Nutrition Education Program (FSNEP) by submitting a Nutrition Education Plan (NEP) to USDA 's Food and Consumer Service (FCS) for approval . The goal of FSNEP is to improve the dietary intake of families on food stamps through activities that increase self-sufficiency. According to federal regulations , nutrition education is provided to food stamp recipients only , unlike EFNEP, which provides nutrition education to recipients of any kind of public assistance. Participants in FSNEP can be of any age, but EFNEP participants are families with children or families with persons in their childbearing years . States with a FSNEP are encouraged to work with other agencies, but are not to duplicate EFNEP. The FSNEP often reaches individuals and families who are not in the EFNEP target audience, such as the elderly , or may exist in areas that do not have EFNEP. EFNEP is in all states but is not in every county . Unpublished reports on FSNEP projects suggest that there have been positive impacts in food resource management (94) and nutrition knowledge (95) . Oregon families participating in FSNEP were reported to run out of food at the end of the month less often, keep track of expenditures , and use resources more effectively than families that did not participate in FSNEP (94). 37 Does the Food Stamp Program improve dietary intake? For those who participate, the Food Stamp Program is believed to improve the availability of nutritious food because it increases total food expenditures (96-97). Studies have estimated that a dollar's worth of food stamps i�creases at-home food expenditures by about 26% (96). However, only 60% of eligible households participate (72). To some individuals, particularly the elderly, there is a stigma attached to food stamps because they are a welfare benefit. Also, some Americans do not understand the complicated application procedures and/or are unaware of their eligibility. Studies that have concluded that Food Stamp participation increases the availability of nutritious food have not shown that participation increases nutrient consumption (96-97). In a summary of 20 years of research on the effects of food stamps, Fraker (96) concluded that evidence that food stamps increase nutrient consumption is weak despite the ability to improve nutrient availability because nutrient intake is affected by dietary behavior, which is not directly measured. Morgan et al. (98) found no significant evidence that, except for calcium, food stamp recipients purchased more nutritious foods. However, a report from the Center on Hunger, Poverty, and Nutrition Policy at Tufts (87) stated that the Food Stamp Program did improve the nutritional status of low-income children significantly. Using data from the 1986 CSFII, researchers concluded that fewer poor children receiving food stamps consumed inadequate amounts of energy, calcium, folate, iron, magnesium, protein, riboflavin, vitamin B6, vitamin B 12, and zinc compared to poor children not receiving food stamps. Cook et al. (87) concluded that the Food Stamp Program was highly effective in 38 improv ing the die ts of low -inco me children . A more recen t report u sing the 1989 -9 1 CSF I I for children age 2- 17 also suggested tha t calories were higher for children in hou seholds receiving food sta mps than for other children , however, to tal fat, sa tura ted fat, chole sterol, die tary fiber, and sodiu m also were higher (99 ). In addi tion to calories, dietary fiber, calciu m, and iron in takes were higher for children liv ing in families receiv ing foo d sta mps than o ther children . The la ter s tudy co mpared children fro m households receiving food sta mps to o ther children while the Cook study co mpared poor children receiving food s ta mps to poor children no t receiving food s ta mps. Levedahl et al. (72) calcula ted a federal bene fit/cos t ra tio for the Food S ta mp Progra m based on the value of program bene fits to recipien ts divided by federal expendi tures . Using F Y 1 990 to 19 92 average opera ting cos t (i.e., ad minis tra tive costs) of7.6% and the assu mp tion that a dollar of food sta mps is equal to a dollar of inco me, the bene fit/cos t ra tio per dollar of federal expendi tures was 0.924 ( $ 1.00 minus 7.6% of $ 1.00). In reali ty, participan ts do not value a dollar 's worth of food s ta mps as a dollar of inco me (72). Fraker (96) also esti ma ted the value of a dollar in food s ta mps an d found tha t a dollar in food sta mps increases the average at-ho me foo d expendi tures by 26%. This i mplies tha t a dollar of food sta mps increases no nfood expendi tures by 74 %, because food s ta mps allow househo lds to su bstitute s ta mps for previous cash expendi tures on food. The bene fit/cos t ra tio is 0.684 ( $ 1.00 minus 7.6% of $ 1 .00 x 0.74), or less than tha t calcula ted by Levedahl et a l. (72). The E mer genc y Food Assis ta nce Progra m. The E mergency Food Assis tance Progra m ( T EF AP}, dis tri bu tes US D A-dona ted foods to low -inco me households through 39 local agencies, such as churches and food banks. Foods are distributed as surplus commodities from federal inventories and are purchased with additional funds. In the first six months of fiscal year 1996, TEFAP spent $ 1 5 .4 million on food assistance (93). The benefit/cost ratio per federal dollar for TEFAP (derived from market prices) was calculated at 0.95 when foods were donated as surplus commodities and 0.77 when foods were purchased (72). Federal costs were smaller when TEFAP used surplus donations instead of purchased commodities. In addition to the value b�sed on market prices, Levedahl et al. (72) used a recipient evaluation to determine the value of commodity foods. From this evaluation, the researchers calculated that recipients placed a value of $ 1 . 06 on a pound of TEF AP cheese. Since the retail price of the cheese was $2.60, this implied that the perceived value of a dollar of TEFAP cheese was $0.4 1 ($ 1 .06/$2.60). To determine the value based on recipient evaluation, each ratio was multiplied by 0.4 1 . Therefore, the benefit/cost ratio according to recipient value was 0.39 (0.95 x 0.4 1) if the foods were donated as surplus commodities and 0.29 (0 .77 x 0.4 1) for commodities �hat were purchased. Based on the cost/benefit ratios calculated for the Food Stamps and TEF AP, Levedahl et al. (72) concluded that food stamps are more efficient means of providing food than TEF AP. TEF AP benefits are limited compared to the wide variety of choices available to food stamp recipients. This lowers the recipient value of the foods. In addition, the Food Stamp Program uses the commercial sector to provide benefits, which is considered more efficient. On the other hand, TEF AP reaches eligible persons who do not take advantage of food stamp benefits. The two programs are seen as complementary. 40 Since TEF AP relies on local volunteer and other charitable organizations who know where the needy individuals are in their communities, TEF AP can refer these people to the F cod Stamp Pr�gram. The Food Distribution Program for Soup Kitchens and Food Banks. The Food Distribution Program for Soup Kitchens and F cod Banks was created through the Hunger Prevention Act of 1 988 primarily to serve the homeless. Churches operating community kitchens, meals-on-wheels programs, soup kitchens, food banks, temporary shelters, correctional institutions, group homes, and hospitals are among the nonprofit, charitable institutions that benefit from donations from USDA. Many serve meals to low-income people on a regular basis. The Commodity Distribution to Soup Kitchens and Food Banlcs and other food donation programs, such the Food Distribution Program on Indian Reservations, the Nutrition Program for the Elderly, the Commodity Distribution to Charitable Institutions and Summer Camps, and the Disaster Feeding Program along with the Emergency Food Assistance Program co�t taxpayers $32 million in the first six months of fiscal 1996 (93). The Special Supplemental Nutrition Program for Women. Infants. and Children {WIC). WIC provides a combination of supplementary food, nutrition education, and referrals for preventive health care for women and children considered to be at nutritional risk. Vouchers are issued for foods, such as milk and cheese, iron-fortified cereal, juice, eggs, peanut butter, and dry beans, that provide specific nutrients. WIC serves pregnant, lactating, and postpartum women and their children up to age five years. EFNEP serves 41 families with children of any age or families with individuals in their childbearing years . Approximately $1.8 billion was spent on WIC in the first six months of fiscal 1996 (93). Most studies on the effects of WIC examined birth weight, anemia , and anthropometric variables. Various researchers have shown that participation in WIC is associated with fewer low birth weight infants , very low birth weight infants , and premature births (100-101) . Owen (102) concluded that these benefits were likely the result of maternal nutritional supplementation, nutrition education, enhanced health care, and social services . Cost analyses have shown that improved outcomes decrease Medicaid costs (13-15) . Surveys of dietary intake suggest that pregnant women who participate in WIC have higher intakes of protein, iron, calcium, vitamin C, magnesium, phosphorus, thiamin, r iboflavin , niacin, vitamin B 6, vitamin B 12, and energy than pregnant women who do not participate (I 03). Women who participate in WIC, who receive advice on breast-feeding, report a higher likelihood of breast-feeding than other women who do not receive advice from a physician (104) . WIC participants who receive information about initiation of supplemental foods for infants are less likely to initiate feeding cereals before their infants are four months old than other women (104) . The National School Lunch Program and School Breakfast Program . The National School Lunch Program (NSLP) provides foods to school-age children to help ensure their diets are adequate and serves as an outlet for surplus agricultural commodities . In addition to NSLP, the school-based child nutrition programs include the School Breakfast Program (SBP) , special milk , and summer feeding programs. 42 Expenditures for these child nutrition programs were $4. 7 billion for the first half of fiscal 1 996 (93). The National School Lunch Program served an average of 24. 1 million meals per day during this time with $3 .3 billion in expenditures. An estimated 6. 1 million breakfasts were served every day (93). A USDA-sponsored assessment of the nutritional quality of school meals indicated that while school lunches provide one-third or more of the daily RDA for key nutrients, very few schools meet the dietary guidelines for total and saturated fat ( 105). Consequently, the Healthy Meals for Healthy Americans Act of 1 994 (Public Law 1 03-448) requires that meals served under the National School Lunch Program and School Breakfast Program meet the Dietary Guidelines for Americans. A recent study of food and nutrient intakes of children participating in NSLP showed that NSLP participants were more likely than nonparticipants to consume milk, meats, grain mixtures, and vegetables {I 06-107). As a result NSLP participants had higher intakes of fat, sodium, calcium, vitamin A, and magnesium than nonparticipants. They were more likely than nonparticipants to eat cakes, cookies, soft drinks, and fiuitades. Children participating in SBP consumed more milk and fruit juice than nonparticipants, resulting in higher intakes of calcium, riboflavin, phosphorus, and magnesium {106- 107) . They also were three times more likely than nonparticipants to consume meat, which explained higher intakes of fat and sodium. The Nutrition Education and Training Program (NET). NET is a federally funded nutrition education program for child nutrition programs. The goal of NET is to promote 43 healthy eating habits for children through educational experiences that help them make informed food choices. Funds are used in a variety of ways, including education for teachers and food service personnel, projects that involve parents and the community, and development of nutrition education materials. Because there is no single NET program model, effectiveness has not been established nationally. A study conducted by Abt. Associates, Inc. under contract with USDA (I 08) focused on programs in Nebraska, Georgia, California, and West Virginia between 1979 and 1 980, before NET was fully implemented nationwide. Data showed that students improved nutrition knowledge. Changes in attitudes, food preferences, plate waste, and other behaviors were positive for some grades and for some foods, but findings were not consistent across the states. There are programs, other than EFNEP, that are intended to improve the nutritional status of low-income families. To what extent they improve actual nutrient intakes has not been well-established for some programs. With the e.xception ofWIC, there have been no published evaluations that include economic analysis. The Perspective In addition to the type of problem and the alternative interventions, the type of technique used depends on the perspective for analysis (Le. , the audience whose resources are at stake) ( 109) . The audience for whom the analysis will be conducted will determine what costs and outcomes are most relevant. According to Rossi and Freeman ( 1 09), a single perspective should be used to define benefits and costs of an intervention. This would be one of three perspectives 44 typically used for analysis of social projects: I) individual, 2) program sponsors, and 3) society (1 09). For the individual perspective, the analysis is performed from the point of view of the program target (i.e., the person, group or organization that receives the intervention). In EFNEP, the individual perspective would be the EFNEP participant. If the economic analysis were done from the EFNEP participant' s point of view, the costbenefits would be high and the program would seem very effective because the participant contributes little to the costs other than time. The program sponsor's perspective takes the point of view of the funding source. The federal government and to a lesser extent state and local governments would be the program sponsors for EFNEP. This perspective is most appropriate when choices must be made involving distribution of funds. If EFNEP did improve food-related behaviors which resulted in improved nutrient intake, then expenditures for diet-related health problems might decrease. If participants improved their health, they might have fewer health problems and improve their earnings, which would increase tax revenues. The societal perspective includes most of the costs and benefits that are used for the individual and program sponsor perspectives, but they may be valued differently. For example, funds spent on �FNEP mean that those funds can not be used to build roads. This research was conducted from the societal perspective. The Office of Management and Budget (Ol\.1B) of the Federal Government provides guidance for economic analysis of government programs by periodically sending circulars to heads of executive departments and agencies. Circular A-94 distributed by the federal Office of Management and Budget (Ol\.1B) clearly states that economic analyses must be provided 45 and defined from the perspective of society rather than the Federal Government (I I 0). According to Warner and Luce ( I 11 ), the societal perspective should be assumed unless a more narrow approach is specified by the audience. If the more narrow program sponsor approach were taken, it would exclude some indirect costs ca�ed by individuals and organizations that give to The University of Tennessee. Legislators are concerned with costs and benefits accruing to constituents. Federal dollars for EFNEP are applied to direct costs. Administrators are concerned with all the costs, both direct and indirect, and benefits to society to justify future contributions from both public and private donors. Establishing the Analytic Framework The analytic framework for conducting economic analysis must be established before the analysis can begin. Based on the problem identified, the alternative interventions or programs, and the audience for whom the analysis is intended, costbenefit is the appropriate analytical technique. A benefit-cost ratio could be calculated using the benefit of saving dollars on food each month and the costs of implementing EFNEP. Some amount of dollars might be saved on food per person per month for every dollar spent to implement EFNEP. The following discussion provides the rationale for conducting a cost-benefit analysis. The problem. The problem can be stated as follows: How effective is EFNEP at improving food resource management practices and food selection and preparation practices of those families that participate and at what cost? These are the intermediate 46 outcomes expected as a result of participation in EFNEP that will determine nutritional and health status ultimately. Based on the problem, cost-benefit or cost-effectiveness analyses would be appropriate techniques. In cost-benefit analysis, both costs and benefits must be expressed in monetary terms and compared as a benefit-cost ratio. As a measure of food resource management, EFNEP participants provide an estimate of total money and other resources spent each month on food in dollars when they enroll and when they complete the program. Therefore, a benefit-cost ratio could be identified using the cost ofEFNEP in dollars and the dollars saved on the total monthly food bill . The outcomes, food resource management practices ( other than dollars saved) and food selection and preparation practices, are not measured in monetary units and could not be converted into dollars in a meaningful way. When the outcomes are not expressed in dollars, cost-effectiveness is the appropriate technique. The two other techniques, cost-feasibility and �Ost-utility would not be appropriate. Cost-feasibility would determine if EFNEP is feasible within available resource constraints. EFNEP has been in operation in Tennessee since 1 969. Therefore, cost-feasibility has been established. Cost-feasibility can not determine if EFNEP is beneficial or compare its effectiveness with other programs. Cost-utility analysis could be used to measure the effectiveness ofEFNEP. However, to use cost-utility analysis the measures of effectiveness would be expressed as estimated values determined by the probability of the outcomes and an assessment about their relative worth. For example in the case of food safety, some value would have to be 47 placed on improving food safety practices based on the probability that foodborne illness will be reduced and the worth of reducing foodborne illness. Since estimated value placed on improving food safety practices would be subjective, the cost-utility analysis would not lend itself to replication by alternative programs with similar goals and objectives. The alternatives. Based on the problem and the alternatives to EFNEP, costbenefit is the best technique for economic analysis. Both cost-benefit and costeffectiveness provide information that helps decision makers choose among alternative interventions that address the problem. Cost-effectiveness analysis is the only one that assumes the goals of the interventions are similar or identical and that a common measure of effectiveness can be used to assess them ( 12). According to Levin ( 12) and Splett (39), two or more alte�natives must be identified for comparison. Therefore, no claim that EFNEP is cost-effective . could be made until the cost-effectiveness ratio ofEFNEP could be compared to an alternative program with similar measures of effectiveness. A search of interventions with the goal of improving the nutritional intake of lowincome adult and youth resulted in identification of several federal food assistance programs. Evaluations have demonstrated that for several of these programs providing food or the resources to obtain food with some level of nutrition education has a positive effect on dietary intake. However, none measure their effectiveness by improvements in food resource management practices and food selection and preparation. The program with objectives and measures of effectiveness closest to those of EFNEP is the Food Stamp Nutrition Education Program (FSNEP). However, the measures of effectiveness may vary according to a state's nutrition plan, because there is 48 no national FSNEP model. FSNEP could be considered an alternative to EFNEP if similar objectives and measures of effectiveness are used. The overall goal of improving diets is the same for all states with FSNEP, but different states may focus on different objectives and, therefore, different measures of effectiveness. In addition, the FSNEP considered would need to be similar in other ways, such as the target audience. The target audience for FSNEP can vary from state to state, but according to federal regulations, the audience must be food stamp recipients. EFNEP would have a broader audience that includes families receiving any type of public assistance. Since cost-benefit analysis measures both costs and benefits in dollars, it is not necessary that alternative interventions have common objectives or measures of effectiveness. The outcome measured is in monetary terms and can be compared to programs that do not have the same goals and measures of effectiveness. The perspective. As discussed previously, the societal perspective was used for this research. The societal perspective on costs and benefits is the traditional perspective of cost-benefit and cost-effectiveness analysis (1 12). Society is not concerned about the feasibility ofEFNEP obtained from a cost-feasibility analysis. The outcomes from costutility analysis would be subjective and would make comparison of alternative interventions difficult for anyone not directly involved in valuing the benefits. Theoretical and Historical Background of Cost-Benefit Analysis Based on the problem identified, the alternatives, and the perspective, cost-benefit was selected as the type of economic analyses for this research. CBA is � technique 49 developed for the evaluation of public policy issues. It is designed especially for public projects for which outcomes are evaluated on the basis of public interest ( 1 12). In contrast to financial analysis, where all costs and benefits are measured in market prices (i .e. , cash and revenue flows), CBA measures the costs and benefits in terms of social utility gains ( 1 12). Welfare economics and public finance provide the framework to identify and assess costs and benefits from society's perspective. The use of welfare economics and public finance justify government involvement, which explains why CBA has been used historically in the design and formulation of policies in federal agencies (1 1 0). It was part of the Flood Control Act of 1 936, used extensively by the Department of Defense in the 1960s, and is used today as an essential evaluation tool by federal agencies under the guidance of the Office of Management and Budget. Resource allocation decisions are central to the theory of cost-benefit analysis. How much can you improve society's well-being for some individuals without making others worse off? In contrast to an accountant of a private firm who asks whether the owners of an enterprise will become less well off by a firm's participation in one activity instead of another, the economists using a cost-benefit appraisal considers whether society as a whole is better off participating in an activity or alternative activities ( 1 1 3). In this research, participation in EFNEP resulting in improved dietary intake could benefit society because society would spend less money on health care for chronic diseases related to diet. While it may seem that another benefit of teaching families to save money on their food purchases is to reduce the amount of food stamps they receive, which may 50 decrease federal spending, research has suggested that the current level of spending on food stamps is not sufficient to meet the minimal nutritional needs oflow-income families (70) . On the other hand, if families learn to budget their food dollars so they need less money to supplement their food stamps, they can purchase more of other goods, such as education or transportation to a job. Or, families may not require emergency food sources such as food banks as often. In economics there is the principle of social betterment in relation to resource allocation known as Pareto optimum, defined as "a state of economic affairs where no one can be made better off without simultaneously making at least one other person worse oft'' ( I 12, p. 1 I ). There is some point reached in resource allocation (the Pareto optimum) in which no further resources can be invested in a project without reducing resources to alternative projects. Therefore, rational decisions must be made to spend resources using an analysis of costs and benefits. Projects that are economically feasible must produce benefits that make everyone in society better off. However, in the real world, someone's welfare is improved at the expense of at least one person becoming worse off. Therefore, gains in welfare are not viewed as actual Pareto improvement, but as potential Pareto improvement ( 1 1 2- 1 1 3). In cost-benefit analysis, this is commonly referred to as KaldorHicks improvement ( I 1 2). Under the Kaldor-Hicks rule, allocation of resources can be justified as long as it raises net social benefits, and as long as those who benefit could compensate those who lose. According to this rule, actual compensation does not have to be carried out but can be a redistribution of gains (1 1 2). For example, in this research the 5 1 benefits of EFNEP would compensate taxpayers through reduced health care costs for tax dollars that would otherwise be spent on other projects. Conducting a Cost-Benefit Analysis Conducting a cost-benefit analysis is a systematic process. Once the objectives of the program to be evaluated are determined and type of economic framework is established , costs and outcomes must be identified and measured (3 9). Then the outcomes must be compared to costs . Finally, the results must be summarized , interpreted , and reported (3 9). Identifying and Measuring Costs Costs are inputs that are required to produce an intervention (I 09). Outcomes and benefits of a program need to be evaluated against the costs necessary to deliver the services or intervention (3 9). Identifying costs . The first steps, before costs can be identified, are to specify the intervention or program and determine the time horizon or a specific time period . Then, all costs should be identified from client recruitment through achievement of the final outcome (39) . Once costs are identified , they must be measured in a systematic manner, summarized , and reported (39). What is it that must be costed? For example, this may include the cost of prevaccination testing for hepatitis B surface antibodies ( 44) , the cost of providing health checks io a group at risk for coronary heart disease ( 114), or the costs of nutrition 52 serv ice s for pregnant women in a c ity health depa rtment ( 4 9). The target of the imp lementat ion or program help s de fine what is to be co sted The target of nu tr it ion se rvice s in the c ity health depa rtment is pregn ant women . The target of an economic analysis ofEFNE P would be ind iv idual s who enro ll and co mplete the program. There fore, the implementat ion to be co sted would be nutr it ion educat ion prov ided by E FNE P. Once the co st s to be mea sured are ident ified, a t ime hor izon mu st be e sta b lished. The t ime hor izon repre sent s the t ime fra me for the inte rvent ion for track ing outcome s and input co st s. The t ime hor izon mu st be real ist ic and take into con siderat ion the l ikel ihood that the outcome s to be mea sured w ill occur w it hin the spec ified t ime frame (3 9). For exa mple, expect ing a sign ificant we ight lo ss after one or two contact s is not real ist ic. Once the inte rvent ion is spec ified , the t ime frame mu st be determined. T ime hori zon s can be sho rt or long depend ing on the type of inte rvent ion and the outcome s de sired. The major ity ofEFNEP part ic ipant s spend from three to six months in the program and graduate when it is determ ined they have met educat ional o bject ive s. There fore, the co st s and outco me s w ill be mea sured for six months to be sure mea su rement s are co llected from the major ity of part ic ipant s who enroll at the beg inn ing of the study. After e sta bl ishing the t ime hor izon, act iv it ie s are de fined . Any act iv it ie s needed to implement a program mu st be de fined be fore they can be co sted ( 1 15) . Act iv it ie s in a nut rit ion educat ion progra m might include conduct ing a nutr it ion a sse ssment, prov id ing ind iv idual ized coun sel ing , implement ing a follow-up evaluat ion, etc . In E FNE P, act iv it ie s 53 may include recruiting homemakers, completing a records, and teaching a homemaker about nutrition. Measuring costs. Once the activities are defined, costs can be determined. Determining the cost of an intervention involves calculating both direct and indirect costs of all activities ( 115). Direct costs are the resources spent to provide the service or intervention, such as personnel, equipment, office supplies, educational materials, and travel. The greatest expense in most programs is personnel, which includes salaries and fringe benefits of all full-time and part-time employees. It includes expenses paid for consultants and the value of volunteers. Resources that support the service or intervention, such as administrative overhead, office space, maintenance, and bookkeeping, are considered indirect costs. Levin (12} recommends an "ingredients method" to identify and measure costs. Using this method, all ingredients are identified and costs are determined to estimate total costs of an intervention. To facilitate the identification and specification of ingredients, they are divided into four main categories (12). These typically include personnel, facilities, equipment and materials, and other program inputs. Other program inputs are expenses that do not fit into the other categories, such as theft or liability insurance beyond what is typically required by the sponsoring agency. Ingredients must be specified in enough detail to determine how they can be measured and valued ( 12). To determine personnel costs, all personnel should be listed according to the responsibilities ( e. g, administration, coordination, teaching), qualifications (i.e. , training and experience}, and time commitment {12). The amount of time committed is the 54 percentage of a full-time position. In the University of Tennessee Agricultural Extension Service, personnel time is described in full-time equivalent (FTEs) . One FTE is equal to 40 hours per week. Facilities include classrooms space, offices, storage space, or other physical space requirements for the intervention. Facilities are included whether or not they are paid by the project (12). According to the Code of Federal Regulations (116), property and some equipment fulfill cost-sharing or a matching requirement for federal programs, such as EFNEP. Equipment and materials are furnishings, instructional equipment, computers, books and other printed materials, paper, and other supplies . If any other ingredients are needed that do not fall in other categories, they are listed as other inputs, which may include extra liability or theft insurance, and other expenses. Levin ( 12) lists three overriding considerations when identifying and specifying ingredients . First, sufficient detail must be provided in order to place a value on each item. Second, when listing ingredients, more consideration must be given to those that contribute the most overall to the total cost of the intervention. In most cases, salaries and fringe benefits represent the largest portion of costs of an intervention . The greater the contribution, the more precisely the costs must be measured. A 10% error in personnel expenses would make a larger difference in the total costs of an intervention than a 10% error in office supplies. Third, there needs to be some consistency when placing items into categories. When determining costs for different programs, how the ingredients are placed in categories should be the same. If theft insurance is categorized under equipment expenses, it also should be done for each alternative costed. 55 Economic analysis is the process of quantifying costs (115). It involves a series of steps that must be carried out systematically for each alternative included in the comparison . Once activities are listed and principle cost components are identified, data must be collected for components . To estimate the quantity of major cost components for each activity, data may be collected using work schedules and existing reports, such as accounting records, time studies, productivity studies, and other methods. Some expenses can be determined directly from accounting records while others must be estimated using a variety of methods. Data needed for determining personnel costs can be collected in a variety of ways (115). Personnel may be asked to keep daily logs or time-and-activity reports to estimate the time spent on activities. Time contribution also may be estimated using work sampling, which involves making several observations about time spent on specific activities and then averaging the times. According to Splett and Caldwell {115), times studies are considered the most valid for determining personnel costs. For a specified time period, employees are asked to keep daily records of the time they spend in activities. After the time period, data are tallied and organized by employee classification and activity studied. It is important that forms used for time studies and daily activity logs be detailed enough to differentiate among the activities to be costed . After it has been determined how much time personnel have spent on the specified activities, the personnel time is converted to costs by multiplying the amount of time (in hours) and the hourly pay of the employee. Usually, the mid-range of a salary scale for each employee classification is used to avoid very high salaries for long-term employees and very low-salaries for those 56 employed a short time. Once the salaries are determined, the cost of fringe benefits are added. Fringe benefits include health and life insurance, employer contributions to social security, pension contributions by employers, and other benefits. Many employers use a standard percentage of the salary, such as 28%, to determine fringe benefits. This percentage is multiplied by the salary to get total personnel costs. After total personnel costs are determined, they are annualized to determine the total personnel cost per year. Travel costs are the actual dollars paid to employees to conduct the intervention or program. These costs are typically the number of miles traveled multiplied by the allowed travel costs per mile. If an employee travels 1 00 miles and is reimbursed $.24 a mile, then travel costs for that employee are $24.00. The time associated with travel, however, is considered under employee salaries { 1 1 5). Continuing education costs are the actual costs associated with an educational opportunity provided through the agency. Only those costs related to the program are considered. Continuing education costs may include honorarium for guest speakers, registration fees, or other costs. The cost of facilities can be determined in two ways ( 12). One way is to calculate the cost to rent or lease the space. The second way is based on ownership of the property. When the space used for a program is part of a property that is owned, the replacement value of the property and the life of the property are determined to calculate depreciation for every year of use ( 1 2). An opportunity cost of having resources invested in the undepreciated part of the property also is determined from an appropriate interest 57 rate ( 12). The annual interest forgone (the opportunity cost) and the annual cost of depreciation are added to determine the annual value ( 12). Once the total cost of a facility is calculated, the percentage of space used by the program of interest should be determined from the total cost since most facilities benefit more than one program. Usually, this determination is based on the amount of space, number of employees, or actual usage by the program ( 1 1 5). The cost of the facilities can be calculated by determining the total square feet of space used by all programs multiplied by the cost per square foot per year multiplied by the percentage of time used by the service to derive a cost to the program per year ( 1 1 5) . In addition to the cost of the facilities, other indirect costs such as maintenance, heat and lights, administration, accounting services, and personnel services must be considered in the cost of delivering a program. Many agencies have a fiscal officer who can explain how indirect costs are considered in the overall agency budget. How much of these costs can be attributed to the program being studied then would have to be determined and expressed as a percentage of time used multiplied by the total agency costs. The cost of equipment can be determined by the cost of leasing or renting. When this information is unavailable or the equipment is owned, the replacement cost of a piece of equipment is used. The replacement cost is annualized, like facilities, taking into consideration depreciation and opportunity costs. When equipment is used by more than one program, the percentage of time used for the programs of interest is determined and costs are calculated as a percentage of the total annual costs. 58 Costs for supplies often account for less than 5% of the total costs of operating an educational program. Therefore, listing each item and determining market prices is not recommended ( 12). The cost of supplies can be estimated by adding total expenditures. The cost of educational materials are determined in a similar manner. Like office supplies, only those materials used by the program during the period of time studied are included ( 1 1 5). For example, if 5,000 publications are printed and 500 are used during the period of study, then only 500 are considered a cost to the program. In some cases, many of the direct costs can be taken directly from budget ledgers kept by the fiscal officer of an agency. All expenses charged to the EFNEP restricted budget account at The University of Tennessee are itemized monthly on a Grant and Contract Budget and Expenditure Report. Expenses listed in the report include salaries, longevity pay, retirement contributions, social security contributions, insurance, travel, printing, telephone, office supplies, and minor equipment. Once costs are measured, they are summarized by relating them to outcomes. This can be done in different ways ( 1 1 5) . Full cost refers to the total cost of a program over a time period (e.g, the cost of implementing EFNEP for one year). Average cost is the cost per unit of outcome ( e.g., the cost of implementing EFNEP per dollar saved on food expenditures). Two types of cost summaries relate the extra cost to produce an outcome. Incremental cost is the cost of adding a service or program to an existing program and marginal costs are the cost of doing slightly more or less within a service or program. 59 Identifying and Measuring Benefits Assessing the outcomes of a program or intervention is considered to be a prerequisite for cost-benefit analysis. According to Rossi and Freeman ( 109), how a benefit is specified and measured is central to economic analysis. Identifying benefits. Outcomes are measurable changes achieved by a participant between entry and exit from a program (20). What is expected to occur as a result of participation in EFNEP? The expected outcomes based on the problem identified are that families will : 1 ) improve food resource management practices and 2) improve food selection and preparation practices. Measuring benefits. Once the expected outcomes are specified, some type of measures must be identified. Improved food resource management practices can be measured by the amount of money saved on food as a result of participating in EFNEP. Saving money on food does not necessarily mean food selection or preparation practices improve. Other measures also must be used to establish whether or not families improved their food selection and preparation practices. It is possible from the EFNEP Reporting System to determine if dollars are saved on food as a result of participation. At the time of program enrollment and graduation, the EFNEP participant is asked to estimate how much the household spends on food in a month. This amount includes all cash, food stamps, and vouchers. Similar questions were part of the 1 995-96 CSFII {1 1 7) . Solely using a self-report of how much money was spent on food in the past month to measure benefits of EFNEP can be problematic for several reasons. The first problem 60 is the reliability of the data. It depends on the respondents' ability to recall, which can be poor ( 1 1 8). Second, because a family spends less on food does not necessarily mean that dietary intake is adequate. It can mean the family is purchasing less food or less expensive food. Using other methods of recording purchases, such as register receipts, may improve the reliability of the data. Mean monthly expenditures reported in the 1 995 EFNEP Reporting System ( I 7) had large standard deviations indicating possible inconsistencies in data collection. Keeping register tapes might improve the accuracy of recording food expenditures. Register tapes have been used to assess food and nutrient intake ( 1 1 9- 120). They can help respondents recall food items they have eaten and are useful tools for nutrition education (1 20). In the case of recalling dollars spent on food, the register tapes may help prompt respondents to recall their expenditures. Using register tapes is considered a relatively low cost method of data collection. Gerace (1 20) asked patients to collect register tapes when initial clinic appointments were made over the telephone and provided no training. In contrast, DeWalt et al . ( 1 1 9) gave mothers a collection packet with instructions on how to collect register tapes and keep logs of foods eaten away from home, foods from home production, foods from household stores, foods from gifts, and foods purchased from stores without itemized receipts. Because the researchers were using the receipts to analyze dietary intake, there was no mention of nonfood items on the receipts because these items could be eliminated without 6 1 affecting data analysis. In a study using receipts to calculate food expenditures, it would be necessary to subtract the nonfood items from the total grocery bill. Collecting register tapes requires more effort on the part of respondents to remember to save the tapes. In addition, some factors may confound the analysis of tapes, such as expenditures for food eaten away from home, food produced by the household (e .g., gardens), food from household stores (e.g, freezers), food gifts, and food from commodity programs (119). Results of consumer expenditure surveys show that in 1993, Americans spent $282.9 billion on away-from-home meals, an increase of 5.5% from 1992 to 1993 (121). As income increased , the percent of total food expenditures away from home increased . Therefore, eating away from home would still need to be considered when estimating total food costs, but to a lesser extent with low income families. According to the EFNEP Summary of Adu lt Participant Profiles in Tennessee (17), of the participants who reported household income (64%), the majority (70%) had incomes 75% or less of the poverty level. Very few EFNEP participants have gardens or household stores of food, and commodity foods are distributed infrequently due to recent federal budget cuts . Therefore, it is expected that these factors would have minimal influence on total food expenditures of the EFNEP population. If nutrient intake improved at the same time the family spent less money on food, with other factors remaining constant , this could be viewed as a benefit from EFNEP . Therefore, the benefit could be stated as : the difference between dollars EFNEP families spent on food before participating in EFNEP and after participating in EFNEP. 62 In addition to nutrient intake, other benefits, such as improved food selection and preparation, could be measured in nonmonetary terms using the national EFNEP Reporting System . . Data are collected from participants using a Dietary Recall Form (Appendix B l ), EFNEP Survey (Appendix B2), and Family Record (Appendix B3). These surveys are implemented upon entry into EFNEP before education occurs and upon graduation from EFNEP, when it is determined by the paraprofessional and supervising Home Economist that the homemaker has met educational objectives. Outcome measures for dietary intake are determined from a 24-hour recall taken on the Dietary Recall Form. Measures include mean number and percent of homemakers who ate a specific number of servings from each food group using the recommended number .of servings from the Food Guide Pyramid ( 19). Energy and the following nutrients also are measured: mean percentage of calories from carbohydrate, fat, and protein; and mean nutrient intake and percent of RDA for protein, iron, calcium, vitamin A, vitamin C, and vitamin B6• Appropriate RDA are used for age and gender of each participant and when the participant is pregnant or lactating. The 24-hour recall has been used to evaluate dietary intake in EFNEP for many years. The recall typically is taken by the paraprofessional in a personal interview. In some cases, such as during group instruction, the participant may be asked to complete the recall with help from the ·paraprofessional . This method is fairly easy to administer, does not take a lot of time away from teaching, and requires little burden on the part of the respondents. The 24-hour recall method has been demonstrated to be comparable to other 63 methods that take more resources to administer when used to analyze dietary intake for groups (1 22). Behavior scores based on how often a homemaker followed recommended practices would be measures of food resource management practices and food selection and preparation practices. Information about food-related practices comes from the EFNEP Survey (Appendix B3). The majority of measures from the survey are ordinal with the following five values on a scale: Do Not Do, Seldom, Sometimes, Most of the Time, and Almost Always. Three questions require an interval number response, one question a categorical (yes or no) response, and two questions require one response to four items on a 5-point Likert-type scale. Both content and face validity have been established for questions on the survey instrument ( 1 23). Content validity refers to whether the questions represent the concepts taught in EFNEP ( 1 24). To establish content validity the researchers identified all major food-related practices from the EFNEP curriculum, "Eating Right is Basic" ( 1 25) and various state-produced lessons. In Tennessee, most lessons are taught from "Eating Right Is Basic." Practices identified were reviewed by randomly selected EFNEP paraprofessionals and supervising agents, then practices were confirmed and ranked in order of importance. Those with low ranking were dropped and the remaining practices were prioritized by randomly selected state EFNEP Coordinators to derive a list of items with a Likert-type scale ( 1 23)._ Face validity is an assessment made by the researchers of whether the questions actually measure what they are intended to measure ( 1 24). Face validity was established 64 for this survey through subjective judgement by experts in nutritional assessment, instrument development, educational research, and EFNEP (123). Responsiveness and cultural sensitivity of the instrument also were established. Responsiveness refers to ability of the questions to detect change over time or to detect minimally important differences between subjects (126). Responsiveness was established using a "different-groups" method that compared responses to the survey by a group of new EFNEP participants to responses by a graduated group of participants ( 1 23). The survey was administered to both groups by paraprofessionals and the completed instruments were scored. A student's t-test was used to determine which items distinguished between the two groups. If the group of graduated clients scored higher than the group of new clients, the questions were considered responsive. Cultural sensitivity refers to the use of simple, concrete words and phrases that the EFNEP clientele can understand. Bowens et al . ( 123) used simple, direct sentences to improve the readability for low-literacy audiences based on meanings of words and phrases to the clients. Items that caused confusion were eliminated. Both responsiveness and cultural sensitivity also were established through additional field tests with 14 7 new EFNEP participants. Mean pre- and posttest scores from the field tests were significantly different, indicating the survey questions were responsive to detecting change ( 1 23). The EFNEP Survey is designed to measure overall change in food-related practices; however, it is possible that the survey could be divided into separate subscales representing different concepts. Bowens et al. ( 123) determined that the questions could 65 be classified into four subscales: A) planning, selecting, and buying food, B) food handling, C) limiting fat in foods, and D) reading food labels. Data from the field test were used to perform a factor analysis, which determined if the different subscales were addressed by the subgroups of questions. Low factor loadings and low item total correlations were used to eliminate items to refine the scales. Reliable questions consistently convey the same meaning to all people surveyed with the instrument ( 127}. Cronbach's alpha coefficients ( 128) were used to determine internal consistency reliability for the subscales (1 29). Questions related to subscales A, B, C, and D had pretest alpha coefficients of 0.74, 0.63, 0.69, and 0.90, respectively. If 0. 70 is considered the minimally acceptable level, then subscales B and C, and possibly A, had questions considered reliable. When posttest alpha coefficients (A=0.62, B=0.40� C=0.68, D=0.90) were compared to pretest coefficients, subscales C and D remained about the same, indicating they replicated well. The authors concluded that more research was needed to determine what items could more accurately measure subscales A and B . How well the benefits of participation in EFNEP are measured and the ability to generalize the benefits to the EFNEP population depend on how well the evaluation is designed. The elements of design include who is measured and what measures are made at what times ( 130). Measuring the benefits of EFNEP involves collecting data about a federally funded program that serves a special population of low-income people. Because laws that prohibit discrimination rule out evaluation designs that withhold services, it is not possible to form a true control group using random assignments to treatment groups. Therefore, a 66 quasi-experimental design called the Nonequivalent Control Group is recommended so that individuals who participate in EFNEP ( the experimental group) are matched as closely as possible to individuals who do not participate in EFNEP (the control group) ( 129- 1 30). Without the control group, it is difficult to attribute the benefits measured to participation in EFNEP. The control group also will help account for factors considered a threat to the reliability of the measurements, such as history, maturation, testing, and instrumentation ( 130). Comparing Costs anti Benefits After the costs and benefits of a program are measured, they are compared in an analysis (39). In some cases, costs and benefits must be discounted before they are compared ( 1 12). Discounting is "a mathematical procedure used to convert future costs and future outcomes to present value" (39, p. 55). When costs and benefits are determined over a period of years or when costs and benefits between programs are compared for different years, the dollar value of costs and benefits for the study period must be considered. Costs and benefits often are presented in a benefit-cost ratio with both sides of the ratio expressed in dollars. The benefit-cost ratio is considered by many to be an appropriate way to express the worth of a program relative to competing alternatives (1 1 1 ). However, the measure of a program's worth in cost-benefit analysis also can be expressed as terms of net benefit. According to Warner and Luce ( 1 1 1 ), a ratio may sometimes be misleading, while net benefit always identifies how a program compares to 67 alternatives when programs are ranked. When analyzing costs and benefits of program, benefits can be considered as benefits or as negative costs. If a program reduces future medical costs, the dollars saved can be subtracted from the costs or added as a benefit. This affects the benefit-cost ratio, but it makes no difference when calculating net benefit . Because it is difficult to reduce many different outcomes to a single benefit in dollars in CBA, or to a single measure of effectiveness in CEA, an array of multiple effectiveness measures sometimes is used to express the worth of program (39, 1 1 1 ). Several outcomes can be expressed in an· array associated with a total cost of a program Summarizing, Interpreting, and Reporting Findings Readers of CBA typically focus on the summaries of the analyses, such a abstracts and news briefs. Therefore it is important that the analyst present findings clearly and fully, no matter how costs and benefits are presented { 1 1 1 ). This includes giving careful at tention to any specific problems and making any assumptions explicit (39, 1 1 1 ). In the real world of public projects, there are uncertainties and limitations to cost and benefit data because it is difficult to control for all variables. Therefore, some assumptions must be made. For example, in health care practices, some differences always exist among different set tings, such as the skill level of staff. It is important to state explicitly assumptions made in the investigation and then test the conclusions if different assumptions were used (47, 1 1 1 ). Sensitivity analysis is a technique designed to test the assumptions used to determine costs and outcomes. 68 The more uncertain the costs or outcomes, the more important it is to conduct sensiti�ty analysis. For example, if the costs to implement EFNEP were estimated using salaries of Extension agents with short tenure, it is possible that personnel costs would be underestimated if several Extension agents actually had many· years of service and higher salaries. Sensitivity analysis would test how using a lower or higher salary would affect costs when actual salaries were not used in the analysis. The basic principles of CBA include discounting, explicitly stating assumptions made, and using sensitivity analysis to test assumptions. Only three of 77 economic analyses in medical articles published in 1 978 to 1 980, or 1 985 to 1 987 and reviewed by Udvarhelyi et al. ( 1 3 1 ) adhered to recommended principles of economic analysis. It is recommended that researchers conducting economic analyses use these principles and that reports be interpreted cautiously when the principles are not used ( 1 3 1 ). Selecting a Project or Program Once the findings of an economic analysis are presented, alternative programs can be compared and one or more programs selected for implementation. CBA is intended to be a practical tool to provide decision makers with the information they need to allocate resources. It is not intended to be the sole criterion for policy decisions. According to Splett ( 45), ethical issues arise when programs are judged solely on the basis of economic analysis. For example, although many of the poorest families live in rural areas, reaching these families can be more expensive than reaching poor families in 69 urban areas. When the results are reported for an economic analysis, ethical considerations should be part of the discussion and implications ( 1 1 1) . There also has been some concern about conflicts of interest. Many economic analyses in health are supported by grants from the National Institutes of Health. However, there is some concern among physicians that some analyses supported by private industry, such as drug companies, may be biased and will be used to justify prices. The policy of the New England Journal of Medicine is that economic analyses are reviewed carefully before they are considered for publication to determine any biases or conflicts of interest ( 1 32). Summary It can be concluded from food and nutrition surveys and other sources of data on the dietary intakes and food-related practices of low-income families that these families consume less than adequate amounts of some dietary components and too much of others. Lack of sufficient resources to obtain enough nutritious food is a major reason lowincome families consume less adequate diets than other families. However, there may be other factors, such as poor resource management practices and poor food selection and preparation. Because low-income �amilies consume inadequate diets, their risk for chronic disease increases. This increases health care costs in a system already consuming more and more of our resources. There are several assistance programs that have been shown to improve the diets of low-income populations. However, programs and services are no longer funded on the 70 basis of efficacy alone. It has become imperative that programs maximize efficacy while minimizing costs. For this reason, economic analysis has become a tool for establishing the value of programs, including those delivering nutrition services, in relation to other programs also competing for fixed resources. 7 1 STATEMENT OF THE PROBLEM Data collected from national surveys and surveillance systems suggest that lowincome households and individuals consume diets that place them at risk for health problems that result in premature mortality and morbidity (90). Low-income populations have the highest prevalence of anemia, low-birth weight deliveries, growth retardation, overweight, high serum total cholesterol, hypertension, and osteoporosis (90). The second nutrition monitoring report indicated that low-income adolescents and adults were less likely to consume adequate intakes of vitamin A, vitamin C, vitamin B6, folate, calcium, iron, and zinc compared to higher-income groups (90). Malnutrition, both under- and over-, in low-income populations has been attributed to lack of resources to acquire adequate food and lack of knowledge and skills needed to acquire adequate diets with limited incomes (3). For some families, limited incomes make it difficult to acquire adequate food no matter how well they manage food resources (56) . For other families the knowledge and skills needed to plan, shop, and prepare foods may help them to manage their resources (3). Managing resources wisely might help them meet their nut�tional needs so they are not forced to do without some foods, while consuming disproportionate amounts of others. Several federal food assistance programs currently serve low-income populations with the goal of improving their ability to acquire food and increase nutrient intakes. Many of these programs are implemented at a large cost to taxpayers. For example, the Food Stamp Program cost American taxpayers over $26.6 billion in FY 1995 (71) . Yet evidence that food stamps increase nutrient intake is not conclusive (96). Welfare reform 72 and budget accountability require that federally funded agencies develop and implement an accountability system based on performance measurement, which is the basis for the Government Performance and Results Act of 1 993 { 1 33). As resources become more scarce, programs that are the most effective at the least cost are likely to be funded. Evaluating programs on the basis of effectiveness and cost can be accomplished through analytic techniques such as cost-benefit analysis. Based on the problem of consuming an adequate diet with limited resources, the research questions in this study will be: 1 . Does participation in EFNEP improve the ability of participants t o manage resources wisely? 1 . 1 . How much money do EFNEP families report spending on food each month before and after three to six months of instruction? 1 .2 What is the magnitude of the difference between changes in food resource management practices before and after EFNEP as a result of three to six months of instruction? 2. Does participation in EFNEP improve the nutritional status of families? 2. 1 What is the magnitude of the difference between mean food and nutrient intakes before and after EFNEP as a result of three to six months of instruction? 2.2 What is the magnitude of the difference between changes in food selection and food preparation practices before and after EFNEP as a result of three to six months of instruction? 3 . How much does i t cost t o change food resource management practices ofEFNEP participants? 3 . 1 How much money, if any, do participants save on food after three to six months of instruction for every dollar spent to implement EFNEP. 73 REFERENCES 1 . Brink MS, Sobal J. Retention of nutrition knowledge and practices among adult EFNEP participants. J Nutr Educ. 1 994;26:74-78. 2. Romero VA, Medeiros DM, Melcher L. Use and effectiveness of Wyoming EFNEP's lesson series. J Nutr Educ. 1 988;20: 1 5- 19 . 3 . Smith PK, Hoerr SL. A comparison of current food bank users, non-users and past users in a population of low income single mothers. J Nutr Educ. 1 992; 24(suppl) : 59S-66S. 4 . Tippett KS, Mickle SJ, Roidt L. Food and nutrient intakes of low-income women and children in metro/nonmetro areas, 1 985/86. Family Econ Rev. 1 990;3 : 1 2- 1 5 . 5. Kennedy E, Goldberg J. Review of What American Children Are Eating. Washington, DC: US Department of Agriculture, Center for Nutrition Policy and Promotion; 1 995 . 6. Center for Hunger, Poverty and Nutrition Policy. The Link Between Nutrition and Cognitive Development in Children. Medford, MA: Tufts University; 1 993 . 7. Food security and methods of assessing hunger in the United States. Testimony by S. Margen. Washington, DC: US House of Representatives, Select Committee on Hunger; 1 989, Page 9. Serial No. 1 0 1 -2. 8. Allen LH. Functional indicators and outcomes of undernutrition. J Nutr. l 990; 1 20:924-932. 9. Food and Nutrition Board, Institute of Medicine. Nutrition During Pregnancy. Washington, DC: National Academy Press; 1991 . 1 0. Peterson KE, Chen LC. Defining undernutrition for public health purposes in the United States. J Nutr. 1 990; 120:933-942. 1 1 . The Surgeon General's Report on Nutrition and Health. Washington, DC : US Dept of Health and Human Services� 1988. DHHS (PHS) publication 88-502 10. 1 2 . Levin HM. Cost-Effectiveness: A Primer. Newbury Park, CA: Sage Publications, Inc.; 1 983 . 1 3 . Buescher PA, Larson LC, Nelson :MD, Lenithan, AJ. Prenatal WIC participation can reduce low birth weight and newborn medical costs: a cost-benefit analysis of WIC participation in North Carolina. J Am Diet Assoc. 1 993 ;93 : 1 63 - 166. 74 1 4. The Savings in Medicaid Costs for Newborns and Their Mothers from Prenatal Participation in the WIC Program. Washington, DC: Prepared by Mathematica Policy Research Inc for Food and Nutrition Service, Office of Analysis and Evaluation, US Dept of Agriculture; 1 991 . 1 5. Schramm WF. WIC prenatal participation and its relationship to newborn Medicaid costs in Missouri: a cost/benefit analysis. Am J Public Health. l 985;75:851 -857. 1 6. Kennedy E. Contemporary budget and policy realities: the state of nutrition education in USDA and the importance of evaluation. In: Doner L, ed. Charting the Course for Evaluation: How Do We Measure the Success of Nutrition Education and Promotion in Food Assistance Programs? Washington, DC: Food and Consumer Service, Office of Analysis and Evaluation, US Dept of Agriculture. Washington DC; 1 997. 17. Summary of Adult Participant Profiles, EFNEP Reporting System. Knoxville, TN: The University of Tennessee Agricultural Extension Service; 1995. 18. Expanded Food and Nutrition Education Program: Guide to Program Management and Supervision. Washington, DC: Cooperative Extension System, US Dept of Agriculture; 1983 . 1 9. The Food Guide Pyramid Washington, DC: Human Nutrition Information Service, US Dept of Agriculture; 1 992. Home and Garden Bulletin No. 252. 20. Kettner PM, Moroney RM, Martin LL. Designing and Managing Programs: An Effectiveness-Based Approach. Newbury Park, CA: Sage Publications, Inc. ; 1990. 21 . EFNEP Evaluation/Reporting System User's Guide, Version 3. 0. Laurel, MD: Cooperative Extension System, US Dept of Agriculture; 1 993 . 22. Anderson JEL. EFNEP evaluation in Colorado using a simplified progression form. J Nutr Educ. l 988;20:227-231 . 23 . Del Tredici AM, Omelich CL, Laughlin SG. Evaluation study of the California Expanded Food and Nutrition Education Program: 24-hour food recall data. J Am Diet Assoc. 1988;88: 185-1 90. 24. Torisky OM, Hertzler AA, Johnson JM, Keller JF, Hodges PAM, Miffiin BS. Virginia EFNEP homemakers' dietary improvement and· relation to selected family factors. J Nutr Educ. 1 989;2 l :249-258 . 75 25. Food and Nutrition Board. Recommended Dietary Allowances. 9 th ed. Washington, DC: National Academy Press; 1 9 8 0. 26 . Amstutz :MK, Dixon DL. Dietary changes resulting from the Expanded Food and Nutrition Education Program. J Nutr Educ. 1 9 8 6 ; 1 8 :55-6 0. 27. Cox RH, Gonzales-Viligar MCRV, Novascone MA, Silva-Barbeau I. Impact of a cancer intervention on diet-related cardiovascular disease risks of white and African-American EFNEP clients. J Nutr Educ. 1 9 9 7;28 :209 -2 1 8 . 28 . Murphy MJ, Wright HS, Heasley DK, Hamilton LW. Impact of EFNEP on some nutrition-related practices. J Am Diet Assoc. 1 9 8 0;76 : 570-574. 29. Green LW, Wang VL, Ephross PH. A 3-year longitudinal study of the impact of nutrition aides on the knowledge, attitudes, and practices of rural poor homemakers. Am J Public Health. l 9 74;64:722-724. 3 0. Summary of Adult Participant Profiles, EFNEP Reporting System. Washington, DC: Cooperative State Research, Education, and Extension Service; US Dept of Agriculture; 1 9 94. 3 1 . An Experimental Evaluation of Nutrition Education Methods. Menlo Park, CA: Prepared by SRI International for the Food and Nutrition Service, US Dept of Agriculture; 1 9 8 1 . Contract Number : 53-3 1 9 8 - 1 - 1 05 . 32. French AM:, Roesel CJ, Shieh SJ. An Assessment of the Objectives, Implementation and Effectiveness of the Adult Expanded Food and Nutrition Education Program. Rosslyn, VA: Prepared by Inter-American Research Associates, Inc for the Cooperative Extension Service, US Dept of Agriculture; 1 9 8 1 . Contract Number: 53 l -3K06 -0- l 03 . 33 . Enyart Y. Evaluation of Limited One-to-One Nutrition Lessons Plus Mailed Lessons and Follow-up Telephone Contacts. Columbia, MO: Prepared by The University of Missouri for the Cooperative Extension Service, US Dept of Agriculture; 1 9 84. Cooperative Agreement Number : 12-05-300-550. 34. Jackson RW. Cost Efficiency and Effectiveness of Two Nutrition Education Methods/or Food Stamp Recipients. Burlington, VT: Prepared by The University of Vermont for the Cooperative Extension Service, US Dept of Agriculture; 1 9 8 6 . Cooperative Agreement Number: 12-05-300-550. 76 35 . Ezell P . Comparisons Among Dietary Parameters for Wayne County, Tennessee, Homemakers, Preschool Children, and Families Enrolled in the Expanded Food and Nutrition Education Program. Knoxville, TN: The University of Tennessee; 198 1 . Master's Thesis. 36. Kucharski B. Characteristics of High and Low Performing Extension Food and Nutrition Education Program Assistants in Tennessee. Knoxville, TN: The Universjty of Tennessee; 1 976. Master's Thesis. 37 . Neilan AM. Evaluating the Effectiveness of Small Group Methods in Nutrition Education for Young Mothers Receiving Public Food Assistance. Blacksburg, VA: Virginia Polytechnic Institute and State University; 1 985 . 3 8 . Hannold RH, Kristiansson KK, Coffey FA, Gunn B, Yancy E. Cost Effectiveness of Three Nutrition Education Delivery Systems. Burlington, VT: Cooperative Extension Service, The University of Vermont; 1 980. 39. Splett PL. The Nutrition Practitioner's Guide to Cost Effectiveness Analysis of Nutrition Interventions. Arlington, VA: National Center for Education in Maternal and Child Health; 1 996. 40. Detsky AS, Naglie G. A clinician's guide to cost-effectiveness analysis. Annal Intern Med. 1 990; 1 1 3 : 1 47- 1 54. 4 1 . L' Abbe KA, Detsky AS, O'Rourke K . Meta-analysis in clinical research. Annal lntern Med 1 987; 1 07 :224-233 . 42. Luce BR, Elixauser A. Standards for Socioeconomic Evaluations of Health Care Products and Services. Berlin: Springer-Verlag. 1990. 43 . Greene VL, Lovely lviE, Ondrich n. The cost-effectiveness of community services in a frail elderly population. Gerontologist. 1 993;33 : 1 77- 1 89. 44 . Kwan-Gett TSC, Whitaker RC, Kemper KJ. A cost-effectiveness analysis of prevaccination testing for Hepatitis B in adolescents and preadolescents. Arch Pediatr Adolesc Med. 1 994; 148 :9 1 5-920. 45. Chesebro MJ, Everett WD. A cost-benefit analysis of colposcopy for cervical squamous intraepithelial lesions found on Papaniocolaou smear. Arch Fam Med l 996;5 : 576-58 1 . 46. Neri M, Migliori GB, Spanevello A, Berra D, Nicolin E, Landoni CV, Ballardini L, Sommaruga M, Zanon P. Economic analysis of two structured treatment and teaching programs on asthma. Allergy. 1 996;5 l :3 1 3-3 1 9. 77 47. Sple tt PL. Phase I : Status report of existing data on the effectiveness, cost, and cost effectiveness of nutrition care services. J Am Diet Assoc. 1991;9 l (suppl) :S9-S14. 48 . Gilbride JA, Parks SC, Palakurthi RR. Attitudes and opinions of dietetics professionals toward cost-benefit and cost-effectiveness analyses . J Am Diet Assoc. l 994;94:386-389. 49 . Splett PL, Ca ldwell HM, Holey ES, Alton IR. Prenatal nutrition services : A cost analysis. J Am Diet Assoc. l 987;87 :204-208 . 50 . Splett PL. The Cost-effectiveness of Education for Cholesterol Reduction: A Three State Investigation. Minneapolis, :MN: The Univers ity of Minnesota ; 1990 . Dissertation. 51 . Montgomery DL, Splett PL. Economic benefit of breast-feeding infants enrol led in WIC . J Am Diet Assoc. l 997;97 :379-385. 52. Brauer PM, Imes S, Thomson ABR. Economic impact of nutrition counseling in patients with Chrohn 's disease in Canada. J Can Diet Assoc. l 988 ;49 :236-240. 53 . Simco rvID, Conklin MT. Focusing on the effectiveness side of the costeffectiveness equation. J Am Diet Assoc. l 989 ;4:485-487. 54. Census of Population and Housing, 1990, Individual States and Earlier Censuses. Washington, DC : US Dept of Commerce, Dept of Census ; 1990 . Summ�ry Tape File 3A, Individual States and earlier censuses. 55 . Census of Population and Housing, 1990: Tennessee and Current Population Reports. Washington, DC : US Dept of Commerce, Dept of Census ; 1990 . Summary Tape File 3A, Series P-90, No . 18 1, and earlier editions. 56. Clancy K, Bowering J. The need for emergency food : Poverty problems and poli cy responses . J Nutr Educ. l 992 ;24(suppl) : 12S-l 7S. 51. Hunger 1994: Transforming the Politics of Hunger. Silver Spring, rvID: Bread for the World In stitute ; 1994. 58. Food Research and Action Center, Community Childhood Hunger Identification Project. A Survey of Childhood Hunger in the United States. Washington, DC : Food Research and Action Center ; 199 1. 78 59. Wehler CA, Scott RI, Anderson JJ. The Community Childhood Hunger Identification Project : A model of domestic hunger - demonstration project in Seattle, Washington . J Nutr Educ. l 992;24(suppl) :29S-35S. 60 . Life Sciences Research Office. Nutrition Monitoring in the United States: An Update Report on Nutrition Monitoring. Washington, DC : Life Sciences Research Office , US Public Health Service, US Dept of Health and Human Services ; 19 89. DlffiS Publication No . (PHS) 89- 1255. 6 1. Leidenfrost N. Definitions Concerned with Food Security, Hunger, Undemutrition and Poverty. Washington , DC : Cooperative Extension System, US Dept of Agriculture ; 1993. 62. Campbell CC. Food insecurity : a nutritional outcome or a predictor variable? J Nutr. 199 1 ; 1 2 1 :408-4 1 5. 63. Frazao E . The American Diet: Health and Economic Consequences. Washington, DC : Economic Research Service, US Dept of Agriculture ; 1995. Agriculture Information Bulletin Number 71 1. 64. Hack M, Klein NK, Taylor HG. Long-term developmental outcomes of low birth weight infants. In: The Future of Children, Vol . 5. Los Altos, CA : Center for the Future of Children , The David and Lucille Packard Foundation ; 1995. 65 . CDC. Economic costs of birth defects and cerebral palsy - United States , 1992. MMWR. l 995;44:694-699 . 66 . Pipes PL, Trahms CM. Nutrition : Growth and development. In : Pipes PL, Trahms CM, eds. Nutrition in Infancy and Childhood St . Louis , MS: Mosbyy ear Book, Inc . ; 1993 . 67. LozoffB, Jimenez E, Wolf AB. Long-term developmental outcome of infants with iron deficiency. N Engl J Med. 199 1 ;325 :687-694. 68. Ryan AS, Pratt WF, Wysong JL, Lewandowski G, McNally JW, Krieger FW. A comparison of breast -feeding data from the National Surveys of Family Growth and the Ross Laboratories Mothers Surveys . Am J Public Health. 199 1 ;8 1 : 1049-1052. 69 . Gerrior S. Does the 1983 Thrifty Food Plan provide a nutritionally adequate diet at the cost level currently used? Fam Econ Nutr Rev. 1995 ;8 :2- 16. 79 70. Crockett EG, Clancy KL, Bowering J. Comparing the cost of a Thrifty Food Plan market basket in three areas of New York State. J Nutr Educ. 1 992;24(suppl) :7 l S-78S. 7 1 . 1 995 Farm Bill : Guidance of the Administration: Section 5 : Food and Nutrition. Washington, DC: US Department of Agriculture; 1 995 . 72. Levedahl JW, Ballenger N, Harold C. Comparing the Emergency Food Assistance Program and the Food Stamp Program. Washington DC: Economic Research Service; 1994 . Agricultural Economic Report No. 689. 73 . Rank I\1R, Hirschl TA. The link between population density and welfare participation. Demography. l 993 ;30:607-622. 74. Poverty in the United States: 1991. Washington, DC: US Bureau of the Census; 1 992. Current Population Report, Series P-60, No. 1 8 1 . 75 . Lino M. Income and spending of rural single-parent families. Fam Econ Nutr Rev. 1 995;8 : 1 7-26. 76. Emmons L. Food procurement and the nutritional adequacy of.diets in lowincome families. J Am Diet Assoc. 1 986;86: 1684- 1 693 . 77. Campbell CC, Desjardins E. A model and research approach for studying the management of limited food resources by low income families. J Nutr Educ. 1 989;2 1 : 1 62- 1 7 1 . 78. Boya YO. Consumer usage of unit pricing. J Consumer Studies Home Econ. 1987; 1 1 : 279-293 . 79. Bradbard S, Michaels EF, Fleming K, Campbell M. Understanding the Food Choices of Low Income Families: A Summary of Findings. Alexandria, VA: Food and Consumer Service, Office of Analysis and Evaluation, US Dept of Agriculture; 1 997. 80. Kaufman PR, MacDonald JM, Lutz SM, Smallwood DM. Do the Poor Pay More for Food? Item Selection and Price Differences Affect Low-Income Household Food Costs. Washington, DC: Food and Rural Economics Division, Economic Research Service, US Dept of Agriculture; Agricultural Economic Report No. 759; 1 997. 8 1 . Lino M, Guthrie J. The food situation of families maintained by single mothers: expenditures, shopping behavior, and diet quality. Family Economics Rev. 1 994;7:9-2 l . 80 82. Nicklas TA, Webber LS, Kaschak ML, Berenson GS. Nutrient adequacy of low fat intakes for children: The Bogalusa Heart Study. Pediatr. l 992;89 :22 1 -228. 83 . Lutz Z, Blaylock J, Smallwood D . Household characteristics affect food choices. Food Review. May-August, 1 993 . 84. Krebs-Smith SM, Cook A, Suhar AF, Cleveland L, Friday J, Kahle LL. Fruit and vegetable intakes of children and adolescents in the United States. Arch Pediatr Ado/es Med 1 996 : 1 50: 8 1 -86. 85 . Kennedy E, Goldberg J. Review of What American Children Are Eating. Washington, DC: Center for Nutrition Policy and Promotion, US Dept of Agriculture; 1 995. 86. Tippett KS, Mickle SJ, Roidt L. Food and nutrient intakes of low-income women and children in metro/nonmetro areas, 1 985/86. Family Economics Review 3 : 1 2-1 5 ; 1990. 87. Cook JT, Martin KS. Differences in nutrient adequacy among poor and non-poor children. Medford, MA: Center on Hunger, Poverty and Nutrition Policy, Tufts University; 1995. 88. Radimer K, Olson CM, Greene JC, Campbell CC, Habicht JP. Understanding hunger and developing indicators to assess it in women and children. J Nutr Educ. l 992;24:3 5-45 . 89. Brief el RR Woteki CE. Development of food sufficiency questions for the third National Health and Nutrition Examination Survey. J Nutr Educ. l 992;24(suppl) :24S-28S. 90. Life Sciences Research Office. Third Report on Nutrition Monitoring in the United States. Vol I . Washington, DC: Life Sciences Research Office, Federation of American Societies for Experimental Biology; 1995. 9 1 . Treiman K, Freimuth V, Damron D, Lasswell A, Anliker J, Havas S , Langenberg P, Feldman R. Attitudes and behaviors related to fruits and vegetables among low-income women in the WIC program. J Am Diet Assoc. 1 996;28 : 1 49- 1 56. 92. Reicks M, Randall JL, Haynes BJ. Factors affecting consumption of fruits and vegetables by low-income families. J Am Diet Assoc. 1 994;94 : 1 3 09- 1 3 1 1 . 93 . Oliveira V. Cost of food-assistance programs declined slightly in first half of 1 996. Food Review. l 996;September-December:26-33 . 8 1 94. Katzev AR, Katzev D. Evaluation of the 1993-94 Oregon Food Stamp Family Nutrition Education Program. Public Policy Research Report, 2432 NW Johnson, Portland, OR; 1994. 95. Butkus S . Mid-year Report of Family Nutrition Education Program. Puyallup, WA: Washington State University; 1995. 96. Fraker TM. The effects of food stamps on food consumption: A review of the literature. In: Current Perspective on Food Stamp Program Participation. Washington, DC: Food and Nutrition Service, US Dept of Agriculture; 1 990. 97. Allen J, Gadson K. Nutrient Consumption Patterns of Low-Income Households. Washington, DC: Economic Research Service, US Dept of Agriculture; 1 983. TB- 1 685. 98. Morgan KJ, Peterkin BB, Johnson S� Goungetas B. Food energy and nutrients per dollar's worth of food from available home food supplies. Home Econ Res J. 1 985; 14:24 1 -25 1 . 99. Lin BH, Guthrie J, Blaylock J. The Diets of America 's Children: Influence of Dining Out, Household Characteristics, and Nutrition Knowledge. Food and Consumer Economics Division, Economic Research Service, US Dept of Agriculture; 1 996. Agricultural Economic Report No. 746. I 00. Devaney B. Very Low Birthweight Among Medicaid Newborns in Five States: The Effects of Prenatal WIC Participation. Washington, DC: Prepared by Mathematica Policy Research, Inc for Food and Nutrition Service, Office of Analysis and Evaluation; US Dept of Agriculture; 1 992 . 1 0 1 . Stockbauer JW. WIC prenatal part icipation and its relation to pregnancy outcomes in Missouri : A second look. Am J Public Health. l 987;77:8 1 3-8 1 8 . 102. Owen AL and Owen GM. Twenty years ofWIC: A review of some effects of the program. J Am Diet Assoc. l 997;97 :777-782. 103 . Rush D, Sloan NL, Leighton J, Alvir JM, Horvitz DG, Seaver WB, Garbowski GC, Johnson SS, Kulka RA, Holt M, Devore JM, Lynch JT, Woodside MB, Shanklin OS . Longitudinal study of pregnant women. Am J Clin Nutr. l 988;48 :439-483 . 1 04. Bayder N, McCann M, Williams R, Vesper E. Final Report: WIC Infant Feeding Practices Study. Alexandria, VA: Prepared by Battelle Centers for Public Health for the US Dept of Agriculture, Office of Analysis and Evaluation, Food and Consumer Service; 1 997. 82 1 05 . Burghardt J , Gordon A, Chapman N, Gleason P , Fraker T. The School Nutrition Dietary Assessment: School Food Service, Meals Offered, and Dietary Intakes. Washington, DC: Prepared by Mathematica Policy Research Inc for the US Dept of Agriculture; 1 993 . 1 06. Gordon AR, McKinney P. Sources of nutrients in students' diets. Am J Clin Nutr. l 995 ;6 l {suppl) :232S-240S. 1 07. Gordon AR, Devaney BL, Burghardt JA. Dietary effects of the National School Lunch Program and the School Breakfast Program. Am J Clin Nutr. l 995 ;6 l {suppl):22 l S-23 1 S. 1 08 . St. Pierre RJ, Rezmovic V. An overview of the national Nutrition Education and Training Program evaluation. J Nutr Educ. 1 982; 14 : 6 1 -66. 1 09. Rossi PH, Freeman HE. Evaluation: A Systematic Approach. Beverly Hills, CA: Sage Publications; 1 982:267-297. 1 1 0. Government Accounting Office. Circular No. A-94. Washington DC: US Office of Management and Budget; 1 992. 1 1 1 . Warner KE, Luce BR. Cost-Benefit and Cost-Effectiveness Analysis in Health Care. Health Administration Press. Ann Arbor,· MI; 1 982. 1 12 . Nas T. Cost-Benefit Analysis: Theory and Application. Sage Publications, Inc. Thousand Oaks, CA; 1 996. 1 13 . Mishan EJ. Cost-Benefit Analysis: An Informal Introduction. Allen & Unwin, Inc. Winchester, MA; l 988. 1 14 . Field K, Thorogood M, Silagy C , Normand C , O'Neil l C, Muir J . Strategies for reducing coronary risk factors in primary care: Which is most cost effective? Br MedJ. 1 995;3 10 : 1 083- 1 084. 1 1 5 . Splett PL, Caldwell M . Costing Nutrition Services: A Workbook. Chicago, IL: Region V, US Dept ofHealth and Human Services; 1 985 . 1 1 6. Cost-Sharing or Matching, Subpart G, Chapter XXX, Title 7. In: Code of Federal Regulations. Washington, DC: Office of the Federal Register, National Archives and Records Administration; 1 992. 1 1 7. What We Eat in America: 1994-1996. Continuing Survey of Food Intakes by Individuals. Rockville, MD: Prepared by Westat, Inc for Human Nutrition Research Center, Agricultural Research Service, US Dept. Of Agriculture; 1 996. 83 1 1 8 . Thompson FE, Byers T. Dietary assessment resource manual. J Nutr. 1 994; 1 24:2245S-23 1 7S. 1 19. DeWalt KM, D' Angelo S, McFadden M, Danner W, Noland M, Ketchen JM. The use of itemized register tapes for analysis of household food acquisition patterns prompted by children. J Am Diet Assoc. l 990;90 :559-562 . 1 20. Gerace T. The descriptive sales slip: A dietary assessment tool on the horizon. J Am Diet Assoc. 1 986; 86 :5 1 5-5 1 6. 1 2 1 . Manchester A. 1 993 food spending picked up. Food Review. 1 995 :8 :44 . 122 . Beaton GH, Milner J , McGuire V, Feather TE, Little JA. Sources of variance in 24-hour dietary recall data: Implications for nutrition study design and interpretation. Carbohydrate sources, vitamins, and minerals. Am J Clin Nutr. 1 983; 37 :986-985 . 123 . Bowens J, Cox RH, Pratt C, Gaylord C . Development of a food practice checklist for use with low-literacy homemakers in EFNEP. J Family and Consumer Sciences. l 995;Summer:35-42. 1 24. Thorndike R. Applied Plychometrics. Boston, MA: Houghton Mifflin; 1 982. 125 . Eating Right Is Basic. Third Version. Lansing, MI: Michigan State University; 1 995 . 126. Guyatt G, Walter S, Norman G. Measuring change over time: assessing the usefulness of evaluative instruments. J Chronic Disease. 1 987;40 : 1 7 1 - 1 78 . 1 27. Perkin J . Design and use of questionnaires in research. In: Monsen ER, ed. Research: Successful Approaches. Chicago, IL: The American Dietetic Association, 1 992; 1 1 1 - 129. 1 28 . Cronbach L . Coefficient alpha and the internal structure of tests. Psychometrika. 1 95 1 ; 1 6 :297-334. 1 29. Fitz-Gibbon Morris. How to Design a Program Evaluation. Los Angeles, CA: Sage Publications; 1 987. 1 30. Campbell DT, Stanley JC. Experimental and Quasi-Experimental Designs for Research. Chicago, IL: Rand McNally; 1 963 . 1 3 1 . Udvarhelyi S, Colditz GA, Rai A, Epstein AM. Cost-effectiveness and costbenefit analyses in the medical literature. Anna Intern Med. 1 992; 1 1 6 :239-244. 84 1 32. Kassirer JP, Angell M. The journal's policy on cost-effectiveness analyses. N Engl J Med. 1 994 :33 1 :669-670. 1 33 . Ladewig H . Performance-based Budgeting and Cooperative Extension. College Station, TX: Texas Agricultural Extension Service, The Texas A&M University System; 1 995. 85 Part 2 A Cost-Benefit Analysis of the Expanded Food and Nutrition Education Program 86 INTRODUCTION Americans spend billions of dollars every year to improve the nutritional welfare of at-risk populations through public and private programs that provide food assistance and education (1) . Since resources for health and social services are limited, stakeholders are asking if those resources are spent for programs that have the greatest potential for achieving desired outcomes (2). Future funding will be determined by how well a program achieves outcomes in relation to costs compared to alternative approaches (2). Economic analysis has been used for years by federal agencies in the decisionmaking process as a way to choose among alternative approaches (3). Using evaluation procedures based on systematic and careful assessment of options, such as cost-benefit analysis, can help decision makers allocate resources more efficiently. In cost-benefit analysis, potential benefits are expressed in monetary terms so that alternatives can be compared. Decisions about accepting or rejecting a program are based on whether there is a net gain when costs are subtracted from outcomes (3-5). Although economic analysis has been in existence for years and has been used in education and health, few cost-benefit analyses have been conducted in nutrition (6). Cost-benefit analyses have been conducted for the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) using estimated savings in Medicaid expenses for low birth weight infants (7-9) and reduction in Medicaid expenditures for breast-fed infants on WIC (10). A second type of economic analysis, cost-effectiveness analysis, has been used to evaluate prenatal nutrition services (1 1 ), to compare the 87 effectiveness of treatment alternatives for reducing cholesterol ( 12), and to demonstrate the economic benefits of nutrition counseling for patients with Crohn's disease ( 13). The Expanded Food and Nutrition Education Program (EFNEP) is a educational program funded with federal dollars and implemented by state land-grant universities designed to improve the nutritional welfare of low income families. Although costeffectiveness analyses have been conducted that examined various delivery strategies in EFNEP (14-16), a review of literature reveals no published studies that examined the costs of implementing an ongoing EFNEP program in relation to benefits expressed in monetary terms. The objectives of EFNEP are to improve behaviors related to food selection and preparation, food resource management, and food safety, leading to the ultimate outcome of good health. To accomplish these objectives, participants enroll for a series of lessons consisting of information on planning nutritious meals and snacks using the Food Guide Pyramid ( 17), shopping for food, reading food labels, preparing and storing food safely, preparing foods from each food group, reducing dietary sodium and fat, eating during pregnancy, and feeding children. Lessons are delivered by paraprofessionals trained by Extension Family and Consumer Science faculty employed by the state's land-grant university. In Tennessee, the average number of lessons during the 1 997-98 reporting period was 1 1 .8 and the average length of time enrolled was from 3 to 6 months (1 8). After a series of at least six lessons, the participant graduates from EFNEP. Numerous surveys have suggested that low income families consume too little of some nutrients ( 19-21) and too much of others (22-23). As a result, they are at risk for 88 iron-deficiency anemia, poor prenatal outcomes, poor growth, and chronic disease, such as heart disease, cancer and stroke (24-27). These conditions consume a large portion of tax dollars for health care. The purpose of this study was to apply cost-benefit analysis to determine 1 ) if participation in EFNEP helps households use their resources wisely, 2) if participation in EFNEP helps households improve their nutrient intake, and 3) what it costs to improve these behaviors. The costs and benefits calculated in this research could be used to help decision-makers choose between spending resources on EFNEP or using resources for alternatives. 89 METHODS A detailed description of methods used in this study is in Appendix A. Subjects Subjects were individuals from low-income families living in 1 6 Tennessee counties served by the EFNEP program. All subjects were females 1 8 years of age or older. Only females were included, since the majority (89%) of EFNEP participants were female in Tennessee in 1997 ( 18). Subjects under 18 years of age were excluded because parental consent was required for participation. All subjects were eligible to participate in EFNEP, because they received benefits from a public assistance program or had incomes at or below the poverty level established by the Department of Health and Human Services (28). Paraprofessionals, who deliver nutrition education to participants, and Extension professionals, who supervise county EFNEP programs, recruited participants for the study. Many subjects were referred from agencies that serve low-income audiences. Others were solicited at sites where low-income families frequent (e.g., WIC clinics) and from door-to-door inquiries. Research Design The design used in this study was quasi-experimental, described by Campbell and Stanley (29) as the Nonequivalent Control Group Design. This design included three groups: the experimental group A (the group receiving nutrition education from EFNEP 90 who collected register receipts), the experimental group B (the group receiving nutrition education from EFNEP who estimated food expenditures from recall), and the control group C (a group who qualified for EFNEP, but delayed their participation until subjects in groups A and B completed their EFNEP education). The usual procedure for reporting food expenditures in EFNEP is to ask the participant to estimate how much money, food stamps, and WIC vouchers were spent on food in the month preceding enrollment and the month prior to graduation. Groups B and C followed this procedure, while group A kept cash register receipts for food purchases, or a record of expenditures when a receipt was not available (Appendix B4). Groups A and B received the same nutrition education intervention typically provided by EFNEP paraprofessionals. The only difference between treatment of groups A and B and the usual EFNEP intervention was the use of cash register receipts to determine food expenditures for group A. Group C delayed their nutrition education until subjects in groups A and B graduated so that there was no intervention during the course of the study. In addition to food expenditures, participants in all three groups were given a pretest, which consisted of a survey of demographic characteristics (Appendix B 1 ), a 24-hour dietary recall (Appendix B2), and a survey of food and nutrition behaviors (Appendix B3). The same records were administered a second time: at graduation for groups A and B and at the time they began receiving lessons for group C. Subjects were randomly assigned to one of the three groups, i.e., the first person recruited was assigned to group A, the second person recruited was assigned to group B, and the third person recruited was assigned to group C. In the event a subject declined to 9 1 participate in the group to which she was randomly assigned, she was enrolled in EFNEP, but not in the study. Sample Size A sample size of 3 84 was determined using guidelines published for simplerandom surveys (30-3 1 ). The sample size was based on population size, the permissible error and associated confidence level, and the population proportion to be estimated. In 1 996-97, the nwnber of females over 1 8 years of age who graduated from EFNEP within six months was 3,9 1 1 ( 18). A 0.05 confidence level was selected because the research was considered exploratory and a 0.0 1 level was considered too stringent. According to Wunsch (3 1 ), 0.05 is typically used in educational research. Based on percentages of participants who reported improvements in food and nutrition behaviors from data in the 1 996-97 EFNEP report (32-33), the researcher was interested in determining if at least 70% of participants in the study improved nutrition and food-related practices. Using these factors, it was determined that a sample size of 34 1 people would be representative of the population within an acceptable error limit and take attrition into account. Data Collection The Pilot Test Prior to beginning the study, the methodology for collecting outcome data was pilot tested in two counties with 20 participants. As a result of the pilot test, instructions for recruiting subjects for group C were modified. Initially, paraprofessionals were asked 92 to recruit subjects similar to those in groups A and B by asking them to recommend a friend or neighbor who would agree to be in the control group. Selecting controls from neighborhoods has been used to control for factors such as socioeconomic and ethnic variables (34-35). Participants in the pilot study did not provide names of friends or neighbors. Therefore, paraprofessionals were instructed to employ the usual strategies to recruit subjects, except subjects were to be recruited in sequence. When an individual should be in group C, she was asked if she would delay her EFNEP education for about 3 months. Fallowing the pilot test, the researcher explained the protocol to each paraprofessional and supervising EFNEP agent in 2 1 Tennessee counties. In addition to verbal instructions, paraprofessionals and agents were given written instructions (Appendix C). In accordance with The University Office of Research's policy on human subjects, paraprofessionals and agents were instructed to obtain informed consent from each participant. Copies of the consent forms are in Appendix D. Two kinds of data were collected in this study: cost and outcome. Most cost data were obtained from the University of Tennessee financial data base system. Outcome data were obtained from participants using pre- and posttests. Collecting Cost Data The ingredients method described by Levin (4) was used to estimate costs. According to Levin, any intervention has ingredients that have a value or cost. Once the ingredients are identified, their costs are determined, then the costs of all ingredients are 93 combined to estimate total costs of an intervention. This method also is useful when determining which parts of an intervention have the greatest cost burden. The ingredients identified for EFNEP included: personnel, equipment and materials, facilities, and other inputs. Personnel, equipment and materials were direct costs, while facilities and other inputs were indirect costs. Expenses associated with this research were collected or estimated using The University of Tennessee's financial data base system. A description of accounting standards is found in the document, "Cost Accounting Standards Board Disclosure Statement for Educational Institutions" (36). This document was submitted by The University of Tennessee to DHHS Office of Inspector General in 1 997 to meet the requirements of public law. Direct costs (i.e., personnel, materials, and equipment) were obtained from a monthly Grant and Contract Budget and Expenditure Report (Appendix E), an official accounting document for grants and contracts. All expenses charged to the adult EFNEP account numbers were collected in these reports. These were obtained from the Agricultural Extension's fiscal officer during the months of May, June, July, August, September, and October of 1 997. Monthly expenditures charged to the adult EFNEP accounts during the six-month time horizon are listed in Appendix F. All expenses were multiplied by two to determine total expenses for one year except for three items: I) group food and lodging, 2) seminar and conference registration, and 3) computer purchases. Expenses associated with group food and lodging, and seminar and conference registration, were for a biennial staff development conference for all faculty 94 and staff paid with EFNEP funds. Therefore, these expenses were divided by two to estimate costs for 12 months. The costs of computers and purchased during the six-month time frame were divided by five, the expected life of computers and printers (4). Indirect costs, such as office space, utilities, and other costs, were estimated using a rate of2 1 .79% of direct costs for expenses on campus and a rate of 10.79% for expenses off campus. The rates were developed for the Agricultural Extension Service by the controller's office of The University of Tennessee and the accountant for the Institute of Agriculture using cost data collected during 1997 and were submitted by The University of Tennessee to the US Department of Health and Human Services (DHHS) for approval. Approval was pending during the time of this research. All expenditures listed on the grant and contract report were used to determine indirect costs, except for equipment expenditures. Indirect costs were not included for equipment because depreciation is included in indirect costs. Collecting Outcome Data Outcome data were collected from the Family Record, the Dietary Recall Form, and the EFNEP Survey at entry to and exit from the program. These records are standard for the state program and provided the following information for each participant: 1 ) demographic characteristics, 2) food intake, 3 ) money and other resources ( e.g., food stamps) spent on food for one month, and 4) information about food and nutrition practices. Each record was administered by a paraprofessional who either conducted an individual interview with the subject, or provided instruction to a group of subjects. Each 95 paraprofessional was trained by a supervising Family and Consumer Science agent to administer the records following the protocol in the EFNEP Evaluation/Reporting System User's Guide (37). The Dietary Recall Form was used to record all foods and beverages consumed in a 24-hour period prior to enrollment in EFNEP and 24 hours prior to graduation. Foods obtained from the 24-hour recall were classified into food groups using USDA' s Food Guide Pyramid ( 1 7) by a computerized dietary analysis program included in the EFNEP national reporting system. Intakes of eight nutrients (protein, fat, carbohydrate, vitamin A, vitamin C, vitamin B6, iron, and calcium) and energy also were calculated using the computerized dietary analysis program. In addition to foods consumed, participants in group B were asked how much they spent on food during the month prior to enrollment and during the month before exiting the program. This included money, food stamps, and WIC vouchers. Participants in group A were asked to keep all cash register receipts for food for two weeks out of the month following enrollment and two weeks during the month prior to graduation. These weeks were the middle two weeks of their monthly spending cycle, i.e., the two weeks after they received their food stamps or cash payments. According to Joy (38), the middle two weeks of the monthly spending cycle are a reliable estimate of monthly food expenditures because these two weeks represent average expenditures. Expenditures during the first week following receipt of food stamps and cash payments tend to be very high, while expenditures during the last week of the cycle are very low. 96 When food receipts were not available, participants were asked to record the amount of money spent on food on a record (Appendix B4). According to Joy (38), participants need a method by which they can keep track of food expenditures that may not be recorded on a cash register receipt, such as meals in fast food restaurants and vending machines. Any non-food items and the sales tax on those items were subtracted from the total on the cash register receipts. Paraprofessionals gathered receipts and/or the record of expenditures from participants and submitted them to a supervising agent, who added the total expenses and multiplied the total by two to represent a month during each of the reporting periods. These amounts were recorded on the Dietary Recall Form. Food expenditures for groups B and C were obtained by participant recall for the month prior to enrollment and the month prior to leaving the program, which is the usual method for determining food expenditures in EFNEP. The purpose of collecting food receipts in group A was to provide a second method by which food expenditures could be measured. Since this cost-benefit analysis was based on the estimated savings in food expenditures as a result of participation in EFNEP, food expenditure data affected the results of the analysis. Therefore, it was important to consider how more than one method of collecting data affected the final calculation, the net present value (NPV). Using sensitivity analysis, it was possible to examine how differences in the way food expenditures were valued influenced the NPV. Because some variables have a great influence on the final calculation, it is recommended that every CBA model include sensitivity analysis. 97 The EFNEP Survey was used to collect data on food resource management practices and food selection and preparation practices. The survey consisted of 13 questions related to three areas: food resource management practices, food selection and preparation practices, and food safety practices. Since only two questions measured changes in food safety behaviors, food safety practices were not included as EFNEP outcomes in this study. Ten survey questions required a response to a Likert-type scale and corresponded to a numerical score of I to 5 (i.e., I = Do not do, 2 = Seldom, 3 = Sometimes, 4 = Most of the Time, 5 = Almost Always). Three questions asked how many times in the past two weeks or in the past month a behavior was practiced and had a possible score of O to 3 1 . One question required a yes or no response. Incentives to participate were provided for subjects in groups A and C, because they were asked to keep register receipts for food or a record of food purchased (group A) or to delay their participation in EFNEP (group C). Normally, EFNEP participants are not asked to keep a record of food expenses and are not asked to delay their participation. Subjects in group A were given colorful three-ring binders for Extension publications, while subjects in group C received one of the following tokens of appreciation: a meat thermometer, vegetable brush and peeler, measuring spoons, measuring cups for dry ingredients, measuring cup for liquid ingredients, or plastic cutting board. Statistical Analyses Statistical analyses were conducted using The Statistical Analysis System (SAS) for Windows, Release 6. 12 ( 40). Frequency tables were constructed for each of the three 98 groups and the total sample for variables with nominal measurements, such as gender, race, residence, participation in public assistance programs, and type of instruction. Frequency tables also were constructed for the total number of persons in a family, number of adults other than the participant in a family, number of children in a family, the ages of children, and the number of lessons received. Descriptive statistics were used for measurements such as age, number of children, total family size, servings from food groups, nutrient intakes, food expenditures, and behaviors from the EFNEP survey. Food Expenditure Data The difference between the amount of money or other resources spent on food at program entry and program exit was calculated for each of the three groups. The Analysis of Variance (ANOV A) procedure, a parametric test, was used to compare food expenditures of the groups. Even though data were not normally distributed according to a statistical test for normality, the distribution was symmetrical . According to Schlotzhauer and Littell ( 41 ), normality also can be assumed when data on the total population is available and the sample is representative of the total population. ANOV A was used to determine if differences in sample means for each group were statistically significant. The independent variable group (A, B, or C) was used as the classification variable and the difference in food expenditure as the dependent variable. An alpha-level of 0.05 was chosen as the significance level. Tukey 's studentized range test was used to make pairwise comparisons between groups. 99 A covariate, how often the participant used a shopping list (Q 12 on the EFNEP survey), and the interaction of the covariate wi� the treatment group were measured for each of the three groups in an Analysis of Covariance (ANCOV A). How often a participant used a shopping list was included in the model to determine more precisely the effect of treatment group on food expenditures. When the ANCOV A showed an effect of the interaction of treatment group and how often shopping lists were used, correlation coefficients were computed to measure the strength of the relationship. Food, Nutrient, and Survey Data Statistical tests on nutrient intake and survey data were nonparametric because data were not normally distributed. An alpha-level of 0.01 was used for these tests to control the probability of making a type I error by rejecting the null hypothesis that there were no differences between groups. Differences between program entry and exit were used in tests for food and nutrient intakes. Scores for EFNEP Survey questions were analyzed as differences between program entry and exit or as average scores at exit depending on the type of data. Ratio measures were analyzed as differences between entry and exit. Ordinal measures were analyzed as average scores at exit. Once it was established that there were no statistical differences in food and nutrient intakes between program entry among the three groups, and no differences between the two experimental groups at program exit, they were combined for further analyses. Likewise, experimental groups were combined for survey data, such as planning meals ahead of time. The two experimental groups were combined because they 1 00 received the same nutrition education intervention and the intent of the analysis was to establish that differences between subjects who received the intervention, and those who did not, were due to the intervention. The Kruskal-Wallis Test for comparing two groups (i.e., the combined ,experimental groups and the control group) was used for both food and nutrient intakes and survey data. One question on the survey required a yes or no response; therefore, a chi-square test was used to determine if average responses from groups were significantly different at program exit. Comparing Costs and Benefits Once costs and outcomes were identified and quantified, the benefits of implementing EFNEP were compared to the cost. This was accomplished in two ways: 1 ) using a single measure of effectiveness and 2) using multiple effectiveness measures in an array. With the single measure of effectiveness, the net present value (NPV) was calculated using the amount of money and other resources (e.g., food stamps, WIC vouchers) saved on food expenditures as a result of participation in EFNEP and the cost of implementing EFNEP. NPV was calculated using the formula: where r = discount rate t = time period T = time frame NPV= L (Benefits - Costs) t = O (1 + r)' The amount saved on food expenditures was the difference between food expenditures by each family at the beginning of the study (i.e., program entry) and at the 101 end of the study (i.e., program exit). For this research, sensitivity testing was applied to determine future benefits using three discount rates: 3, 5, and 7%. Discounting is used in economic analyses to adjust the future value of costs and benefits to current value. According to N as (3 ), testing more than one discount rate is useful because shadow pricing, the method recommended by Office of Management and Budget (0MB) for determining discount rates ( 42), is difficult and complicated due to the effect of taxation policies on interest rates. The Public Health Service's Panel on Cost-Effectiveness in Health and Medicine recommends that a 3% discount rate be used in cost-effectiveness analyses of health interventions ( 43 ). Five percent is the most common discount rate in health-care literature (44). Seven percent is the rate recommended by the 0MB for government projects because it "approximates the marginal pretax rate of return on an average investment in the private sector in recent years" ( 42, p. 7). Because there is uncertainty about how long participants practice food resource management behaviors that may lead to savings in food expenditures, two retention periods were tested using sensitivity analysis. The amount saved on food expenditures in one month was multiplied by 3 years and 5 years, based on the assumption that the knowledge gains and behavior changes would be sustained for at least three years as reported by Green et al. ( 45) and Torisky et al. ( 46), and for five years as reported by Nierman ( 4 7). Program cost was the average cost per unit of output, which was determined by dividing the total cost of implementing EFNEP for 12 months by the total number of participants that graduated during the same period. Future costs were not discounted 1 02 because providing EFNEP education to participants is a one time expense incurred over an average time of three to six months. Because the difference in food expenditures at program entry and exit was the single measure of effectiveness, two assumptions were tested using sensitivity analysis. The first assumption was that keeping receipts and records of food expenditures was a reliable way to determine family food expenditures. The second assumption was that participant recall of food expenditures without keeping receipts or records, the typical method used in EFNEP, was a reliable way to document family food expenditures. Each method was used to calculate the average savings on food to derive NPV s. In the array of measures, an array of outcomes was compared to average cost per program family. The advantage of this method is the ability to express diverse outcomes in relation to cost. As several researchers in the field of economic analyses have noted, it is difficult to reduce diverse outcomes to a single measure of effectiveness expressed in dollars (3-5). According to the Office of Management and Budget ( 42), even though all benefits can not be monetized, it is useful to identify as many benefits as possible and to quantify them using other summary measures. The array of outcome measures consisted of differences in food and nutrient intakes from program entry to program exit and average scores on behaviors related to food resource management practices and food selection and preparation practices at program exit for individuals in program families. 103 RESULTS Sample Four hundred seventy subjects from 16 counties were recruited for the study, 1 63 in group A, 1 59 in group B, and 148 in group C. Of the 4 70 who completed entry records, 444 (94.47%) completed exit records. Sixty-seven (1 5 .09%) of those who completed entry and exit records were excluded, because they failed to keep food receipts, a record of expenditures, or indicated they spent no money or other resources on food. Eight additional subjects were excluded because two were males and four were under 1 8 years old. Therefore, 371 subjects (78.94%) completed the study. Participation rates by county are shown in Table 2. Overall, participation rates were similar for all three groups. All subjects were female, 23 (6.20%) were pregnant, and 9 (2.42%) were nursing. The mean age of subjects was 3 1 years old and ranged from 1 8 to 72 years old. Mean ages of each group were similar: 32 years in group A, 3 1 years in group B, and 30 years in group C. Racial/ethnic characteristics and place of residence of subjects are presented in Table 3. The majority of the sample was either African-American or white, which was similar to the total population of EFNEP participants during the same reporting period, 1 997-98 (1 8). However, 55% of participants were African-American and 43% were white in the sample compared to 59% white and 39% African-American in the total population. Most participants lived in towns of less than 10,000 people and rural 104 Table 2. Participation rates by county for each group. Group A Group B Group C Total County Number Rate Number Rate Number Rate Number Rate Benton 0 0.00 6 66.67 3 42.86 9 34.62 Carroll 17 94.44 18 100.00 17 94.44 52 96.30 Carter 9 100.00 6 100.00 6 1 00.00 2 1 100.00 Davidson 0 0.00 20 95 .24 1 7 89.47 38 60.32 Giles 5 71 .43 1 12.50 8 6 1 .54 14 50.00 Greene 6 100.00 4 0.67 4 100.00 14 87.50 Hamilton 2 66.67 2 100.00 0 100.00 4 80.00 Hardeman 10 83 .33 12 100.00 9 75.00 3 1 86. 1 1 Haywood 12 100.00 12 100.00 1 1 9 1 .67 35 97.22 Henry 6 100.00 4 66.67 6 100.00 16 88.89 Johnson 5 100.00 0 0.00 0 0.00 5 62.50 Lincoln 4 100.00 4 66.67 6 100.00 14 87.50 Overton 6 100.00 2 100.00 5 100.00 1 3 100.00 I Putnam 7 58.33 8 66.67 0 0.00 1 5 48.39 I Shelby 30 100.00 30 100.00 29 96.67 89 98.89 Unicoi 3 75 .00 0 0.00 0 0.00 3 33 .33 Washington 0 0.00 1 33 .33 0 0.00 1 20.00 Total 12 1 74.23 129 8 1 . 13 12 1 8 1 .76 371 78.94 105 Table 3 . Demographic characteristics of the sample: race/ethnicity and residence. Group A Group B Group e Total n=l21 n=129 n=l21 n=371 Freq. Percent Freq. Percent Freq. Percent Freq. Percent IRAcE I African- 60 49.59 77 59.69 69 57.02 I 206 55.53 American ! I White 59 48.76 50 38.76 50 4 1 .32 159 42.86 American 1 0.83 1 0.78 0 0.00 2 0.54 Indian/ Alaskan Hispanic- 1 0.83 1 0.78 1 0.83 I 3 0.8 1 American I I I Asian or 0 0.00 0 0.00 1 0.83 1 0.27 Pacific I Islander I RESIDENCE I Towns 58 47.93 56 43 .4 1 56 46.28 170 i 45.82 <10,000 I and rural Central 38 3 1 .40 57 44. 1 9 49 40.50 144 38.8 1 cities over I 50,000 Towns & 24 19.83 14 10.85 16 13 .22 54 14.56 cities 10,000 to 50,000 Farm 1 0.83 2 1 .55 0 0.00 3 0.8 1 Suburbs of 0 0.00 0 0.00 0 0.00 0 0.00 cities over 50,000 I 106 communities (45.82%) or central cities over 50,000 (38.81%). In the 1 998 Adult Participant Profile for EFNEP participants (1 8), 40% lived in the small towns and rural communities and 42% lived in the large cities. The composition of subjects' households is shown in Table 4. The average family size was 3 .43 persons and was similar for groups A, B and C, respectively. Most families in each of the groups had from one to three children, which was comparable to the average number of children reported for all EFNEP participants in 1 997-98 (1 8). Most of the children (72.75%) reported by subjects in the sample were 1 0 years old or younger. This was consistent for all three groups. Over half of the participants in the sample were single-female households, while about a third reported a spouse or another adult. Fewer than 1 0% reported more than one other adult living in the household. Participation in public assistance programs was similar for groups A, B, and C, as shown in Table 5. Participation also was consistent with that reported by all EFNEP participants in 1 997-98 (1 8). The majority (60. 1 1 %) of the sample received food stamps. Over one third participated in the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) or the Commodity Supplemental Food Program (CSFP). Almost one third receiv�d money from the Temporary Assistance Program to Needy Families (T ANF), a cash assistance program. In addition to child nutrition, The Emergency Food Assistance Program (TEFAP), and Head Start, over 20% of subjects received public assistance from other sources, such as public housing and energy assistance. 1 07 Table 4. Demographic characteristics of the sample: household composition. Group A Group B Group e I Total n=121 n=l29 n=121 n=371 Freq. Percent Freq. , Percent Freq. Percent Freq. Percent I TOTAL NUMBER OF FAMILY MEMBERS I I 7 5.79 6 4.65 3 2.48 1 6 4.3 1 2-4 86 7 1 .07 98 75 .97 97 80. 17 281 I 75 .74 5-7 20 16.53 24 1 8.60 17 14.05 I 61 16.44 8- 10 8 .6.61 17 14.05 4 3.3 1 1 3 3 .50 I TOTAL NUMBER OF CHILDREN PER FAMILY I 0 14 1 1 .57 19 14.73 1 3 10.74 46 12.40 1-3 88 72.73 96 74.42 94 77.69 278 74.93 4-6 17 14.05 13 10.08 1 3 10.74 43 I 1 1 .59 I 7-9 2 1 .65 I 0.78 I 0.83 4 1 .08 I AGES OF CHILDREN I less than I 7 3 . 1 8 14 6.0 1 8 3.73 29 4.34 year 1 -5 years 84 38. 1 8 105 45.06 8 1 37.67 270 40.42 6- 10 years 57 25 .92 67 28.76 63 29.30 1 87 27.99 1 1-14 years 40 1 8. 1 8 20 8.58 30 13 .95 90 13 .47 15-1 8 years 29 13 . 1 8 26 1 1 . 1 6 3 1 14.42 86 12.88 19 years 3 1 .36 I 0.43 2 0.93 6 0.90 Total 220 100.00 233 100.00 2 1 5 100.00 668 I 1 00.00 I TOTAL NUMBER OF OTHER ADULTS I 0 68 56.20 79 6 1 .24 70 57.85 227 58 .49 I 42 34.7 1 36 27.9 1 4 1 33.88 1 1 9 32.09 2 5 4.13 9 6.98 5 4. 1 3 1 9 5 . 12 3 6 4.96 4 3 . 10 5 4. 13 1 5 4.04 4 0 0.00 I 0.78 0 0.00 I 0.27 1 08 Table 5 . Participation in public assistance programs. Group A Group B Group e Total Type of n=l21 n=l29 n=l21 I n=371 Assistance Freq. Percent Freq. Percent Freq. Percent Freq. Food 73 60.33 83 64.34 67 55.37 223 Stamps WIC/CSFP 1 40 33 .06 50 38.76 43 35.54 133 I TANF2 42 34.7 1 44 34. 1 1 36 29.75 122 Child 36 29.75 3 1 24.03 40 33 .06 107 Nutrition I Other 26 2 1 .49 23 17.83 27 22.3 1 I 76 TEFAP3 20 16.53 17 13 . 1 8 1 7 14.05 54 Head Start 1 0 8.26 12 9.30 12 . 9.92 34 1Special Supplemental Nutrition Program for Women, Infants, and Children (WIC)/Commodity Supplemental Food Program (CSFP) 2Temporary Assistance to Needy Families (T ANF) 3The Emergency Food Assistance Program (TEFAP) Percent 60. 1 1 35.85 32.88 28.84 20.49 14.56 9. 16 Over 96% of participants received six more lessons (Table 6). The average number of lessons received was 13 .27 for group A and B (13 . 1 3 and 1 3 .40, respectively). This was comparable to the average number of lessons reported for the total Tennessee EFNEP population in 1997-98 (1 8). Since the first six lessons taught contain basic food and nutrition information, they are considered the minimum number necessary to graduate from EFNEP. Almost 98% (244) of participants received six or more lessons. I I I The majority ( 6 1 . 1 6%) of the sample received individual lessons with the program assistant (Table 7). In comparison, only 19% of the total population of EFNEP participants received individual lessons during 1997-98 (1 8). 1 09 Table 6. Number oflessons received by subjects in group A and B. I Group A Group B Total Number of n=121 n=129 n=250 Lessons Frequency Percent Frequency Percent Frequency Percent I <6 2 1 .65 4 3 . 10 6 2.40 �6 1 19 98.35 125 96.90 244 97.60 Table 7. Type of instruction received by subjects in group A and B. Group A Group B Total Type of n=121 n=129 n=250 Instruction Frequency Percent Frequency Percent Frequency Percent Group 41 33 .88 57 44. 19 98 39.20 Individual 74 6 1 . 16 69 53.49 143 57.20 Group and 6 4.96 3 2.33 9 3 .60 Individual Program Costs Direct costs for May, June, July, August, September, and October of 1 997 are listed in Appendix F. When direct costs for personnel, equipment, and materials were summarized for six months and multiplied by two to estimate yearly expenditures, total costs were $1 ,337,795 .71 (Table 8). The largest portion of total dollars spent was $1 ,276,836. 1 1 for the first ingredient, personnel. Personnel costs included $888,61 0.94 for professional salaries (state specialists, Extension agents, and program leaders), clerical salaries ( secretaries paid monthly), and all paraprofessionals (including paraprofessionals and secretaries classified as paraprofessionals who are paid at an hourly rate). 1 1 0 Expenditures on benefits totaled $3 1 7, 1 1 6.70 and included longevity, retirement, Social Security, Unemployment Compensation, Workers Compensation, group insurance, and 401 K matching. Food and lodging expenditures typically are considered part of travel expenses. However, group food and lodging was charged on one master bill and listed separately. Since group food and lodging expenses were incurred for a biennial staff development conference, expenses were divided by two to estimate yearly expenses. The cost of equipment and materials, was $60,959.60. This included the cost of the following: printing, duplication, and binding; computers; maintenance and repairs of equipment; operating supplies; photography; and subscriptions. Facilities and other program inputs were considered as indirect costs. Total indirect costs for 12 months in 1997-98 were $ 1 76,052. This included $48,842 for the state portion of the budget and $ 127,2 10 for the county portion of the budget. Indirect costs per month are listed in Appendix F. Total costs to implement EFNEP for 1 2 months in 1997-98 were $ 1 ,5 1 3,847.71 . A total of 3,899 families graduated from EFNEP in12 months (1 8). Therefore, the program cost for 12 months was $388.26 per family that graduated. Program Benefits Comparison of Experimental Groups and Control Group at Entry Average entry scores for four questions on food resource management practices and five questions on food selection and preparation practices from the EFNEP Survey 1 1 1 Table 8. Direct costs for 12 months. I Ingredient I Ex�enditures {dollars} I PERSONNEL Salaries 888.6 10.94 Benefits 3 1 7, 1 1 6.70 Travel 68,391 .72 I Seminar and Conference Registration 3 10.50 Group Food and Lodging1 2,406.25 Total Personnel 1 ,276,836. 1 1 I EQUIPMENT AND MATERIALS I' Printing, Duplicating, and Binding 1 7,654.86 I Computers 1 0,200.00 Maintenance and Repairs 16,383 .96 Supplies 1 6,692.36 I Photography 8 .48 Subscriptions 1 9.94 Total Equipment and Materials 60,959.60 I TOTAL DIRECT COSTS I 1 ,337,795.7 1 1Biennial staff development conference (e.g., total cost divided by two years). 1 12 are shown in Table 9. Results from the K.ruskal-Wallis Test for comparing two groups showed that scores for the combined experimental group and control group were not significantly different for these practices. However, there was a significant difference between the combined experimental group and the control group in how often participants made shopping lists. At program entry individuals in the combined experimental group more often (P<0.0 1 ) made shopping lists than did individuals in the control group. At program entry, 41 subjects (1 1 .20%) reported cutting the size of their children's meals because there was not enough money for food. Percentages were comparable for the experimental and control groups. Comparison of Experimental Groups and Control Group at Exit Food expenditures. Subjects in both experimental groups saved money after participation in the program. Subjects in group A saved an average of $ 10.36 per month on food expenditures based on receipts and food expenditure records. Those in group B saved an average of $ 19.53 based on recall. However, subjects in group C spent $5.52 more on food at program exit than at program entry based on recall. The ANOV A procedure showed the differences in food expenditures between program entry and exit for groups A, B, and C were not significantly different. However, because there was a significant difference between the experimental groups and the control group in how often they made shopping lists at program entry, this factor was 1 1 3 Table 9. Average scores for food and nutrition related practices at program entry for the combined experimental group and control group. 1 Group A and B Group e Question (n=248) (n=l19) I FOOD RESOURCE MANAGEMENT P�CTICES Planning meals 2.99 ± 1 . 1 7 3 .26 ± 1 .08 Comparing prices 3 .57 ± 1 .32 3 .62 ± 1 .26 I I Run out of food 2. 1 8 ± 1 .28 2.02 ± 1 .24 I I FOOD SELECTION AND PREPARATION PRACTICES Healthy food choices 3 .48 ± 1 .09 3 .73 ± 1 . 10 Adding salt 2.52 ± 1 .40 2.40 ± 1 .36 Labels with less salt 2.3 1 ± 1 .28 2.07 ± 1 .23 Labels with less fat 2.52 ± 1 .36 2.33 ± 1 .38 Food and nutrition needs 2.85 ± 0.70 2.92 ± 0.64 1Scores on the EFNEP Survey included : l=Do Not Do, 2=Seldom, 3=Sometimes, 4=Almost Always, 5=Always I included in the ANOVA as a covariate. When the covariate, how often the subject shopped with a list, and the interaction of the covariate with the treatment group were measured for each of the three groups in the ANCOV A, there was a significant difference between group B and group C (P<0.05). The interaction between how often the subject kept a shopping list and the group they were in did affect food expenditures. Those who more often kept a shopping list and were in group B, saved more on their food expenditures than those who kept a shopping list less often and were in group C. However, keeping a shopping list alone did not significantly affect food expenditures. 1 14 Whether or not keeping a shopping list affected food expenditures depended on the group in which the subject participated. All participants received education from the same competency-based curriculum, "Eating Right is Basic" (48). However, because the number of lessons taught (Table 6) and type of instruction (Table 7) varied, these factors were included in the ANOVA to determine if they affected food expenditures. The number of participants who received less than six lessons was too small to determine if the number of lessons affected food expenditures. ANOV A showed no difference in food expenditures for those who received instruction in groups and those who received education individually. Food and nutrient intakes. Differences in the average number of meals and the number of servings from each food group consumed at program entry and program exit are shown in Table 10. The combined experimental group consumed significantly more vegetables, fruit, and bread (P<0.0 1 ). On average, subjects in this experimental group consumed at exit 1 .42 more servings of vegetables per day, 0.82 more servings of fruit per day, and 1 .02 more servings of breads per day compared to 0. 17 more servings of vegetables, 0.27 more servings of fruit, and 0.04 more servings of bread for the control group. There also were significant differences between the combined experimental group and the control group when differences in nutrient intakes at program entry and exit were examined (Table 1 1 ) . Subjects in the experimental groups consumed significantly more carbohydrate, iron, vitamin C, and fiber by program exit than did subjects in the control 1 1 5 Table 1 0. Comparison of average number of meals and servings from food groups at program entry and program exit for combined experimental group and the control group. Group A and Group B Group C (n=248) (n=l21) Number of Meals 0.38 ± 1 .23 0. 1 1 ± 1 .06 Meat 0.26 ± 2. 1 9 -0.03 ± 2 . 1 1 Dairy 0.60 ± 3 .33 0.23 ± 2.85 Vegetables 1 .42 ± 6.62** 0. 1 7 ± 9.60** Bread 1 .02 ± 3.71 ** 0.04 ± 5.98** Fruit 0.82 ± 3 .80** 0.27 ± 1 . 1 9* * Other 0.69 ± 16.69 0. 12 ± 1 8.41 * *P<0.01 group (P<0.01 ). Vegetables, fruit, and grains, which are important sources of these nutrients, increased as a result of EFNEP education. Mean vitamin A intake was significantly higher for the control group compared to the experimental groups. The difference between calcium intake from program entry to program exit was slightly greater for the combined experimental group compared to the control group, but not significantly so (P<0.05). Food selection and preparation practices. Average scores on nine practices are shown in Table 12. At program exit, subjects in the combined experimental group had higher mean scores on most practices than did subjects in the control group. Improvements were reported in resource management practices and food selection and preparation practices. Subjects in the combined experimental group improved their resource management practices by planning meals ahead of time, comparing prices, and 1 1 6 Table 1 1 . Comparison of the difference between average nutrient intakes at program entry and program exit for the combined experimental group and the control group. Group A and Group B Group C Nutrient (n=248) (n=121) Energy (kcal) 300.68 ± 1 1 64. 70 -25 .7 1 ± 1287.25 Protein (g) 1 6.09 ± 56. 72 1 .39 ± 49.30 Fat (g) 7.85 ± 59.95 0. 1 8 ± 57.62 Carbohydrate (g) 42.99 ± 145.58** -7.92 ± 1 93 .70** Iron (mg) 3 .00 ± 1 0.00** 0.06 ± 1 0.00** Calcium (mg) 256.77 ± 1026.52* 55 .76 ± 895 .76* Vitamin A (RE) 489. 12 ± 2279.22** 701 .60 ± 9375.66** Vitamin C (mg) 37.30 ± 1 1 1 .44** 4.38 ± 96.88* * Vitamin B6 0.39 ± 1 .29 -0.02 ±' 1 .68 Fiber (g) 4.20 ± 1 8.98* * -2.05 ± 22.35** *P<0.05, * *P<0.0 1 shopping with a list (P<0.01 ). In addition, subjects in the combined experimental group reported running out of food before the end of the month less often than did subjects in the control group (P<0.01). There was no difference between groups in whether or not they reported cutting the size of their children's meals because they ran out of food. Subjects in the experimental groups improved their food selection and preparation practices. Scores on behaviors for the combined experimental group were higher at program exit compared to scores for subjects in the control group. Subjects in the combined experimental group more often thought about healthy food choices, more often read food labels to select foods with less salt or sodium, more often prepared foods 1 17 Table 1 2. Average scores for food selection and preparation practices at program exit for the experimental groups and the control group. 1 Group A and B Group e Question (n=254) (n=125) I FOOD RESOURCE MANAGEMENT PRACTICES Planning meals 3 .96 ± 0.93** 3 .41 ± 1 .1 1 ** Comparing prices 4.41± 0.85* * 3 .86 ± 1 .1 3 * * Shop with a list 1 .68 ± 1 .05* * 1 .26 ± 1 .28** Run out of food 1 .50 ± 0.82** 1 .96 ± 1 .24** I FOOD SELECTION AND PREPARATION PRACTICES Healthy food choices 4.25 ± 0.91 ** I 3.75 ± 1 .04** Not adding salt 3 .02 ± 1 .1 9** 2.67 ± 1 .29* * Labels with less salt 3 .37 ± 1 .1 5** 2.57 ± 1 .30** Labels with less fat 3 .62 ± 1 .13** 2.69 ± 1 .34* * Food and nutrition needs 3.23 ± 0.55* * 2.80 ± 0.63 ** 1Scores on the EFNEP Survey included: 1 =Do Not Do, 2=Seldom, 3=Sometimes, 4=Almost Always, 5=Always **P<0.01 II without added salt, and more often read food labels to select foods with less fat (P<0.01 ). More subjects in the combined experimental group also reported the food and nutrition rteeds of their families were being met compared to those in the control group (P<0.01 ), which could be indicative of better food resource management practices and food selection and preparation practices 1 1 8 Net Present Value and Array of Effectiveness Measures The NPV ofEFNEP related to savings in food expenditures was $600.52 in this study. This value was based on the assumption that benefits were retained for at least 5 years as reported by Nierman ( 4 7) at a discount rate of 7% recommended by 0MB ( 40). Participant recall was used to estimate food expenditures rather than register receipts for the following reasons: 1) there was no statistically significant difference between estimated expenditures collected by participant recall and expenditures collected from receipts and records and 2) all paraprofessionals reported difficulties collecting register receipts and/or records of food expenditures (i.e., subjects either forgot to keep receipts and records or provided receipts for purchases made before they were to collect receipts). Those who estimated food expenditures using receipts and records reported lower expenditures than those who estimated expenditures from recall. This information combined with observations reported by paraprofessionals suggested that individuals who estimated expenditures by keeping receipts and records may have underestimated actual expenditures. Sensitivity analyses were conducted on food expenditures to determine how changing the method of estimating expenditures, length of retention of benefits, and discount rate affected the NPV (Table 1 3). NPV was highest, $699. 1 0, when participant recall was used to estimate food expenditures at a 3% discount rate, and when food resource management practices were retained for 5 years. NPV was lowest, -$36.60, when register receipts and records were used to estimate food expenditures at a 7% discount rate, and when food resource management practices were retained for 3 years. If 1 19 Table 13 . Net Present Value (NPV) from savings in food expenditures as a result of participation in EFNEP. Retention of Benefits 3 Years 5 Years Discount Rate Discount Rate 3% 5% 7% 3o/o 5% 701o Using Food Receipts -25.29 -3 1 .22 -36.60 1 92.4 1 1 68.47 146.88 Kept by Participants (Group A) Using Participant 287.66 270.09 253.75 699. 1 0 647.49 600.52 Recall (Group B) the discount rate is 7% and practices are retained for 5 years, the NPV of EFNEP would be $ 146.88, when food expenditures are estimated from receipts and records, and $600.52 when food expenditures are estimated from recall. Both of the NPV calculations were positive. Therefore, the value of benefits exceeded the cost of implementing EFNEP using either method of collecting food expenditures. However, the magnitude of benefits was affected by whether participants kept receipts and records to estimate expenditures or used recall. An array of effectiveness measures for food and nutrient intakes is presented in Table 14. These include improvements in food and nutrient intakes and behaviors that significantly improved for individuals receiving EFNEP education compared to those in the control group. As a result of participation in EFNEP, subjects increased their intake of vegetables, breads, fruit, carbohydrate, iron, vitamin A, vitamin C, and fiber. Table 1 5 shows an array of effectiveness measures for food resource management and food selection and preparation practices. EFNEP participants improved resource 120 Table 14. An array of food and nutrient outcomes for the combined experimental group that improved as a result of participation in EFNEP. Benefit Average Change1 Increased Servings from Food Groups Vegetables 1.42 Bread 1.02 Fruit 0.82 Increased Nutrient Intakes Carbohydrate(g) 42.99 Iron (mg) 3.00 Vitamin C (mg) 37.30 Fiber (g) 4.20 1The difference between number of servings or nutrient intake at program entry and exit. choose foods with less salt and fat. Table 15 . An array of food resource management, food selection, and food preparation outcomes for the combined experimental group that improved as a result of participation in EFNEP. Benefit Average Score1 Improvements in Food Resource Management Practices Planning meals ahead of time 3 .96 Comparing prices before shopping 4.41 Shopping with a list 1.68 Running out of food before end of 1.50 month less often Improvements in Food Selection and Preparation Practices Making healthy food choices 4.25 Adding salt less often when cooking 3 .02 Reading labels for less salt 3 .37 Reading labels for less fat 3 .62 Meeting food and nutrition needs 3 .23 1 1 =Do Not Do, 2=Seldom, 3=Sometimes, 4=Almost Always, 5=Always 121 I I management practices by improving meal planning, comparing prices before shopping, and shopping with lists. As a result families ran out of food less often compared to families in the control group. Because of participation in EFNEP, families learned to make healthy food choices, added salt less often when cooking, and read food labels to choose foods with less salt and fat. In general, subjects felt their food and nutrition needs were being met more often than they did before participating in EFNEP. 122 DISCUSSION Does participation in EFNEP improve the ability of participants to manage resources wisely? Results from this study showed that participation in EFNEP did help households manage their food resources wisely. Participants who kept food receipts and/or a record of expenditures reported a savings of $ 10.36 a month and participants who were asked to recall food expenditures saved $19.53 a month. In comparison, families that did not receive EFNEP education and estimated food expenditures from recall, reported spending $5.52 more a month. Individuals who kept a shopping list and estimated food expenditures from recall saved significantly more on their food expenditures (P<0.05) than did the individuals who estimated food expenditures from recall and did not receive nutrition education from EFNEP. In addition, participants who received education from EFNEP improved their food resource management practices. Participants reported they more (P<0.01 ) often planned meals ahead of time, more (P<0.01) often compared prices when they shopped, more (P<0.0 1 ) often felt their food and nutrition needs were being met, and reported less (P<0.0 1) often running out of food than did participants who did not receive EFNEP education. Does participation in EFNEP improve the nutritional status of families? Participation in EFNEP did improve the nutritional status of families despite spending less resources on food, as shown by improvements in food and nutrient intakes and nutrition behaviors. Families that received nutrition education consumed significantly (P<0.01 ) more servings of vegetables, bread, and fruit than did families in the control group. 123 Therefore, EFNEP participants consumed greater (P<0.01 ) amounts of carbohydrate, iron, vitamin A, vitamin C, and fiber and possibly more (P<0.05) calcium than individuals who did not receive nutrition education. In addition to improved food and nutrient intakes, those who received EFNEP education significantly (P<0.01 ) improved their food selection and preparation practices compared to those who did not receive education. EFNEP participants reported making healthy food choices more often, adding salt to foods less often, and more often reading food labels to choose foods with less salt and fat than individuals who did not receive EFNEP education. In general, they felt their food and nutrition needs were being met more often than did individuals who did not receive education. How much did it cost to improve food resource management practices? The NPV ofEFNEP related to savings in food expenditures by participants was $600.52. Once future benefits were discounted and costs were subtracted, EFNEP families and society realized a savings of $600.52 because money saved could be used to buy other goods and services. This was determined based on following assumptions: I ) behaviors practiced as a result of participation in EFNEP would be maintained for at least five years, 2) future benefits would be discounted at a 7% rate, and 3) food expenditures were estimated by participant recall. One criticism of CBA is that because of the uncertainties involved in identifying and valuing costs and benefits, the same intervention can produce very different results. Therefore, potential users become confused or suspicious that the analyses can be manipulated to support any conclusion ( 49). Due to the uncertainties, assumptions must 124 be stated explicitly and sensitivity analyses must be conducted using different data or methods to test the robustness of the results. For this study, three sensitivity analyses were conducted. One analysis was conducted using different methods of estimating food expenditures, a second analysis used three different discount rates, and a third analysis used two periods for retention of benefits. A very critical asswnption made in this study was that the amount participants reported for food expenditures was accurate. Any uncertainty in food expenditures could change the NPV calculation. The two methods used to collected food expenditure data revealed that participants who reported expenditures from recall reported greater savings than did participants who kept food receipts and/or a record of expenses ($ 19.52 and $ 10.36, respectively). Despite what would appear to be a noticeable difference, the difference was not statistically significant. Average food expenditures for both groups had large standard deviations, which made it difficult to detect a significant difference . However, although the difference was not significant, it did affect the NPV calculation. In this study, sensitivity analysis showed the NPV ranged from -$36.60 to $699 . 10 depending on the values used in the calculation. When participants estimated food expenditures using recall, the NPV calculation was positive at all three discount rates and when benefits were retained for 3 or 5 years. When participants used register receipts and records to estimate food expenditures, NPV calculations were positive at all three discount rates, if benefits were retained for 5 years. However, when benefits were retained for only 3 years, the NPV calculation was negative at all three discount rates. Therefore, both the 125 method used to collect food expenditure data and the length of time participants retained behaviors learned as a result of EFNEP education had an important effect on the NPV. Another criticism of CBA is that often it does not include a control group and, therefore, assumes benefits can be attributed to the intervention. In this research, food expenditure data, food and nutrient intakes, and behavior data were collected for individuals receiving education from EFNEP and for a control group similar in socioeconomic status and other demographic factors. There was a significant (P<0.05) difference in food expenditures between the group reporting from recall only and the control group. Food and nutrient intakes and behaviors significantly (P<0.0 1) improved for those receiving nutrition education compared to the group that did not receive nutrition education. Therefore, it can be inferred that changes occurred due to EFNEP education. A third criticism of CBA is that often it fails to account for all the benefits of an intervention and, therefore, may underestimate the value of a program. This cost-benefit analysis used a single effectiveness measure . Ideally, cost-benefit analysis should include the value of all outcomes. However, using a single effectiveness measure is a common approach to cost-benefit analysis in health literature. Cost-benefit analyses of WIC used the savings in Medicaid benefits for infants during the first six months of life as a single measure of effectiveness (7-9). The researchers did not include other measures of effectiveness, such as improvements in iron intake ( 50) or savings due to increased incidence of breastfeeding ( 5 1) that have been documented in WI C evaluations. The practice of using the single measure effectiveness often is used because it is difficult to value all outcomes of an intervention in monetary terms. In a nutrition education program 126 such as EFNEP, it is more difficult to value the ultimate outcomes ( e.g., improved access to nutritious food and improved health) compared to the intermediate outcomes, such as improved food resource management behaviors. The problem is that there are no established methodologies for estimating ultimate benefits in dollars, particularly the intangible benefits, such as reduction in pain and suffering. In recent years, estimation techniques have been explored that address the problems of valuing outcomes for which market prices do not exist. According to the Centers for Disease Control and Prevention (39), these techniques include the use of expert opinion, past legislative policy decisions, the use of court awards to estimate intangibles such as pain and suffering, the cost-of-illness approach, and the willingness-topay approach. Of these five approaches, the cost-of-illness and the willingness-to-pay approaches can be validated more easily than the other techniques. The cost-of-illness approach attempts to determine the economic cost of disease by adding medical and nonrn.edical costs of disease and productivity losses because of morbidity or premature mortality. The willingness-to-pay approach attempts to measure the value individuals place on reducing the risk of death or illness. The most frequent way this is accomplished is through contingent-valuation studies using surveys of individuals. However, adequate survey instruments, which are required for this approach, are difficult to find in the literature. Methods that have appeared in health care literature have been characterized as poor, which makes it difficult to distinguish between good and bad CBA studies ( 49). The problem with using a single measure of effectiveness is the importance placed on one benefit, in this case, the savings in food expenditures. A major limitation of this 127 study was reliability of the food expenditure data. Despite a carefully explained protocol and periodically checking with paraprofessionals on collecting food expenditures, 15% of the group estimating food expenditures using receipts and records failed to keep receipts and records as instructed. Keeping some expenditures on a record did help participants remember what they spent because they often did not have a receipt. The process of collecting and summarizing expenditures also was arduous for EFNEP paraprofessionals and the supervising agent. Participants were told what two weeks of the month to collect receipts and records, which could vary by participant. Paraprofessionals had to remember to prompt participants to keep receipts at the designated time. The supervising agent had to determine which items on receipts were food purchases and sales tax had to be determined for food items only. The finding that there was no significant difference between estimates of food expenditures from receipts and records and from participant recall suggested that using participant recall would be as accurate as keeping receipts and records and less burdensome for participants and staff. The purpose of conducting a cost-benefit analysis is to provide decision makers with information needed to make choices among alternative interventions. Alternative interventions need not have common objectives, as in cost-effectiveness analysis. How does E FN EP compare to other interventions using cost-benefit analysis? The NPV determined for EFNEP in this study, $600.52 per participant, compared favorably with a cost-benefit analysis of diet counseling for individuals with Crohn's disease (13). Brauer et al. reported a net benefit to society of $163 .90 per patient who received diet counseling. 128 Despite the growing number of cost-benefit analyses in health care literature, few have used net present value to present findings. If results from this study were expressed as a benefit-cost ratio, the ratio would be 3 .03 : 1 . A savings of $3 .03 was realized on food expenditures for every dollar spent to implement EFNEP. Benefit-cost ratios reported for WIC (7-9), which varied from 1 .77: 1 to 3 . 1 3 : 1 , were comparable to the benefit-cost ratio for this study ofEFNEP. Benefit-cost ratios for the use offolic acid fortification of grain in preventing neural tube defects were slightly greater. Romano et al. reported a 4.3 : 1 ratio for low level fortification and a 6. 1 : 1 ratio for high level fortification (53). 129 APPLICATIONS This study and numerous others (54-56) provide evidence that EFNEP does accomplish the objectives for which it is intended, i.e. , to help low-income families acquire the knowledge, skills, attitudes, and behaviors needed to consume nutritionally sound diets. From the cost-benefit analysis conducted for this research, there also is evidence that the value of the benefits exceed implementation costs. A one time cost per participant could result in hundreds of dollars in savings for society provided that behaviors that improve as a result of EFNEP are sustained. An important assumption made in this study was that the behaviors needed to reduce food expenditures practiced at the time of graduation from EFNEP would be maintained for at least five years. What this economic analysis of EFNEP means to individuals making policy decisions regarding the benefits of allocating funds to EFNEP is that even with the current state of knowledge about valuing benefits, the monetary value of EFNEP in terms of savings in food expenditures exceed the monetary costs provided the participant retains behaviors learned for five years. As reliable methods of estimating other benefits evolve, this should further increase the monetary value of EFNEP. Additional cost-benefit analyses of EFNEP and other nutrition education interventions are needed to establish the monetary value of other benefits, such as prevention of chronic disease. To prevent or delay the onset of chronic diseases, recommended food selection and preparation practices must be maintained much longer than five years. To show that EFNEP can save society money by preventing chronic disease, changes in food and nutrition behaviors need to be measured over several years. 1 30 To this date, no studies have been published that examine the retention of EFNEP benefits for more than five years. With reliable methods of valuing benefits and documentation of the effectiveness of nutrition education, including both intermediate and long term outcomes, nutrition educators will have the tools necessary to apply economic analyses in a variety of settings. 1 3 1 REFERENCES 1 . Oliveira V. Cost of food-assistance programs declined slightly in first half of 1 996. Food Review. 1 996;September-December:26-33 . 2. Splett PL. The Nutrition Practitioner's Guide to Cost Effectiveness Analysis of Nutrition Interventions. Arlington, VA: National Center for Education in Maternal and Child Health; 1 996. 3. N as T. Cost-Benefit Analysis: Theory and Application. Sage Publications, Inc. Thousand Oaks, CA; 1996. 4. Levin HM. Cost-Effectiveness: A Primer. Newbury Park, CA: Sage Publications, Inc.; 1983. 5. Warner KE, Luce BR. Cost-Benefit and Cost-Effectiveness Analysis in Health Care. Health Administration Press. Ann Arbor, MI; 1 982. 6. Splett PL. Phase I: Status report of existing data on the effectiveness, cost, and cost effectiveness of nutrition care services. J Am Diet Assoc. 1 99 1 ;9 l (suppl) :S9-S 14. 7. Buescher PA, Larson LC, Nelson MD, Lenithan, AJ. Prenatal WIC participation can reduce low birth weight and newborn medical costs: a cost-benefit analysis of WIC participation in North Carolina. J Am Diet Assoc. 1993;93 : 1 63-166. 8. The Savings in Medicaid Costs for Newborns and Their Mothers from Prenatal Participation in the W/C Program. Washington, DC: Prepared by Mathematica Policy Research Inc for Food and Nutrition Service, Office of Analysis and Evaluation, US Dept of Agriculture; 1 99 1 . 9. Schramm WF. WIC prenatal participation and its relationship to newborn Medicaid costs in Missouri : a cost/benefit analysis. Am J Public Health. 1 985;75 :85 1 -857. 10. Montgomery DL, Splett PL. Economic benefit of breast-feeding infants enrolled in WIC. J Am Diet Assoc. 1997;97:379-385. 1 1 . Splett PL, Caldwell HM, Holey ES, Alton IR. Prenatal nutrition services: A cost analysis. J Am Diet Assoc. 1 987;87:204-208. 1 32 12. Splett PL. The Cost-effectiveness of Education for Cholesterol Reduction: A Three State Investigation. Minneapolis, MN: The University of Minnesota; 1 990. Dissertation. 13 . Brauer PM, Imes S , Thomson ABR. Economic impact of nutrition counseling in patients with Chrohn's disease in Canada. J Can Diet Assoc. 1988;49:236-240. 14. Enyart Y. Evaluation of Limited One-to-One Nutrition Lessons Plus Mailed Lessons and Follow-up Telephone Contacts. Columbia, MO: Prepared by The University of Missouri for the Cooperative Extension Service, US Dept of Agriculture; 1 984. Cooperative Agreement Number: 1 2-05-300-550. 1 5. Jackson RW. Cost Efficiency and Effectiveness o/Two Nutrition Education Methods for Food Stamp Recipients. Burlington, VT: Prepared by The University of Vermont for the Cooperative Extension Service, US Dept of Agriculture; 1 986. Cooperative Agreement Number: 12-05-300-550. 16. Neilan AM. Evaluating the Effectiveness o/Small Group Methods in Nutrition Education/or Young Mothers Receiving Public Food Assistance. Blacksburg, VA: Virginia Polytechnic Institute and State University; 1 985. 1 7. The Food Guide Pyramid. Washington, DC: Human Nutrition Information Service, US Dept of Agriculture, 1 992. Home and Garden Bulletin. 252. 1 8 . Summary of Adult Participant Profiles. EFNEP Reporting System. Knoxville, TN: The University of Tennessee Agricultural Extension Service; 1 998. 19. Cook JT, Martin KS. Differences in nutrient adequacy among poor and non-poor children. Medford, MA: Center on Hunger, Poverty and Nutrition Policy, Tufts University; 1 995. 20. Food Research and Action Center, Community Childhood Hunger Identification Project. A Survey o/Childhood Hunger in the United States. Washington, DC: Food Research and Action Center; 1 99 1 . 2 1 . Tippett KS, Mickle SJ, Roidt L. Food and nutrient intakes of low-income women and children in metro/nonmetro areas, 1985/86. Family Econ Rev. 1 990;3 : 12-1 5 . 22. Kennedy E, Goldberg J. Review of What American Children Are Eating. Washington, DC: Center for Nutrition Policy and Promotion, US Dept of Agriculture; 1995. 133 23. Life Sciences Research Office. Third Report on Nutrition Monitoring in the United States. Vol 1 . Washington, DC: Life Sciences Research Office, Federation of American Societies for Experimental Biology; 1 995. 24. Allen LH. Functional indicators and outcomes ofundemutrition. J Nutr. 1990; 120:924-932. 25. Food and Nutrition Board, Institute of Medicine. Nutrition During Pregnancy. Washington, DC: National Academy Press; 1991 . 26. Peterson KE, Chen LC. Defining undemutrition for public health purposes in the United States. J Nutr. 1990; 1 20:933-942. 27. The Surgeon General's Report on Nutrition and Health. Washington, DC: US Dept of Health and Hwnan Services; 1 988. DHHS (PHS) publication 88-50210. 28. Department of Health and Hwnan Services. Poverty Guidelines. Washington, DC: US Dept of Health and Hwnan Services; 1 998. 29. Campbell DT, Stanley JC. Experimental and Quasi-Experimental Designs for Research. Chicago, IL: Rand McNally; 1963. 30. Wang, MQ, Fitzhugh E, Westerfield RC. Determining sample size for simplerandom surveys. Health Values. 1 995; 1 9:53-56. 3 1 . Wunsch DR. Survey research: Determining sample size and representative response. Business Education Forum. 1 986:F ebruary:3 1 -34. 32. Diet Recall Summary Report. EFNEP Reporting System. Knoxville, 1N: The University of Tennessee Agricultural Extension Service; 1 997. 33. Behavior Checklist Summary Report. EFNEP Reporting System. Knoxville, 1N: The University of Tennessee Agricultural Extension Service; 1 997. 34. Bell BP, Goldoft M, Griffin PM, Davis MA, Gordon DC, Tarr Pl, Bartleson CA, Lewis JH, Barrett TJ, Wells JG, Baron R, Kobayashi J. A multistate outbreak of Escherichia coli 0157:H7-associated bloody diarrhea and hemolytic uremic syndrome from hamburgers. J Am Med Assoc. 1 994;272; 1 349-1 353. 35. Wacholder S, McLaughlin JK, Silverman DT, Mandel JS. Selection of controls in case-control studies. Am J Epidemiol. 1 992; 1 35 : 1 0 19- 1028. 1 34 36. The University of Tennessee. Cost Accounting Standards Board of Disclosure Statement. Knoxville, TN: The University of Tennessee; 1 997. 37. EFNEP Evaluation/Reporting System User's Guide, Version 3. 0. Laurel, MD: Cooperative Extension System, US Dept of Agriculture; 1 993. 38. Joy AB. Development of a simplified methodology to measure the effect of food purchasing practices on nutrient costs. Presented at: Cooperative State Research, Education and Extension Service Meeting, Nutrition Education for Diverse Audiences. February 10-14, 1 997; Houston, TX. 39. Clemmer B, Haddix AC. Cost-benefit analysis. In: Haddix AC, Teutsch SM, Shaffer PA, Du:fiet D, ed. Prevention Effectiveness: A Guide to Decision Analysis and Economic Evaluation. New York: Oxford University Press; 1 996:85- 1 02. 40. Statistical Analysis System. Release 6. 12. Cary, NC: SAS® Institute, Inc; 1 996. 41 . Schlotzhauer SD, Littell RC. SAS® System for Elementary Statistical Analysis. Cary, NC: SAS® Institute, Inc. ; 1 987. 42. Government Accounting Office. Circular No. A-94. Washington DC: US Office of Management and Budget; 1 992. 43 . Shaffer PA, Haddix AC. Time preference. In: Haddix AC, Teutsch SM, Shaffer PA, Du:fiet D, ed. Prevention Effectiveness: A Guide to Decision Analysis and Economic Evaluation. New York: Oxford University Press; l 996;76-84. 44. Krahn M, Gafni A. Discounting in the economic evaluation of health care interventions. Med Care. 1 993;5 :403-41 8. 45 . Green LW, Wang VL, Ephross PH. A 3-year longitudinal study of the impact of nutrition aides on the knowledge, attitudes, and practices of rural poor homemakers. Am J Public Health. l 974;64:722-724. 46. Torisky D, Hertzler A, Mifflin B. Evaluation of the Expanded Food and Nutrition Education Program (EFNEP) in Selected Areas of Virginia: Extent and Retention of Dietary Improvement and Related Family Factors. Blacksburg VA: Virginia Polytechnic Institute and State University; 1 987. 47. Nierman LG. A Longitudinal Study of the Retention of Foods and Nutrition Knowledge and Practices of Participants from Michigan Expanded Food and Nutrition Education Program. East Lansing, MI: Michigan State University; 1 986. 1 35 48. Eating Right is Basic. Third Version. Lansing, MI: Michigan Cooperative Extension Service, Michigan State University; 1 995. 49. Russell LB, Gold MR, Siegel JE, Daniels N, Weinstein MC. The role of costeffectiveness analysis in health and medicine. JAMA. 1996;276: 1 1 71 - 1 1 77. 50. Rush D, Sloan NL, Leighton J, Alvir JM, Horvitz DG, Seaver WB, Garbowski GC, Johnson SS, Kulka RA, Holt M, Devore JM, Lynch JT, Woodside MB, Shanklin DS. Longitudinal study of pregnant women. Am J C/in Nutr. 1 988;48 :439-483 . 5 1 . Bayder N, McCann M, Williams R, Vesper E. Final Report: WIC Infant Feeding Practices Study. Alexandria, VA: Prepared by Battelle Centers for Public Health for the US Dept of Agriculture, Office of Analysis and Evaluation, Food and Consumer Service; 1997. 52. Obrien B, Gafni A. When do the "dollars" make sense? Toward a conceptual framework for contingent valuation studies in health care. Med Decision Making. 1 996; 16 : 288-299. 53 . Romano PS, Waitzman NJ, Scheffler RM, Randy DP. Folic acid fortification of grain: an economic analysis. Am J Public Health. 1 995 ;85:667-676. 54. Del Tredici AM, Omelich CL, Laughlin SG. Evaluation study of the California Expanded Food and Nutrition Education Program: 24-hour food recall data. J Am Diet Assoc. 1 988;88: 1 85-1 90. 55 . Amstutz MK, Dixon DL. Dietary changes resulting from the Expanded Food and Nutrition Education Program. J Nutr Educ. 1 986; 1 8 :55-60. 56. Cox RH, Gonzales-Viligar MCRV, Novascone MA, Silva-Barbeau I. Impact of a cancer intervention on diet-related cardiovascular disease risks of white and African-American EFNEP clients. J Nutr Educ. 1 997;28:209-21 8. 1 36 APPENDICES 137 Appendix A Extensive Methodology 138 METHODS This appendix describes detailed methods used in this study. This research was a cost-benefit analysis of EFNEP, an evaluation of the effectiveness of EFNEP combined with an analysis of costs in a prospective study. The methods used to determine costs, benefits, and calculating the net present value (NPV) are described in this appendix. The Expanded Food and Nutrition Education Program Participants in this study were either enrolled in the Expanded Food and Nutrition Education (EFNEP) or would qualify as participants in EFNEP. EFNEP is a nutrition education program for low-income families and youth implemented by the United States Department of Agriculture's Cooperative State Education, Research, Education, and Extension Setvice (CSREES) with state land-grant universities and county governments. Each of the 50 states and territories receive federal formula funding to implement EFNEP based on the number of low-income residents. Funding began in 1969 and is appropriated by Congress on an annual basis. In 1998, there were 26 counties in Tennessee that provide EFNEP. Nutrition education is delivered to participants by paraprofessionals supervised and trained by Extension Family and Consumer Science agents. Each participant receives a series of lessons from the curriculum, "Eating Right is Basic" ( 1 ). These lessons consist of six basic units in foods and nutrition that include information on meal planning, shopping for food, the Food Guide Pyramid (2), reading food labels, and food safety. In addition to these units, additional lessons are available on each of the food 139 group s in the Food Guide Pyr amid , nutrition during preg nancy, brea stfeeding , feeding c hildren, and reducing d ieta ry fat and sod ium. Par tic ip ants rec eive instruc tio n ind ividua lly, in gro up s, or a co mbina tion of both. Type of Research Design T he de sign used in this re se arc h wa s qua si-expe rime ntal, de sc ribed by C ampbell and Stanley ( 3) a s the None quiva le nt Contro l Group De sign ( Tab le 16). This de sign included three group s: the exper imenta l group A ( the group rece iving nutritio n educ atio n fro m EFNEP who collec ted re gister rece ip ts and/or record s of food p urc ha se s), the expe rime ntal group B (the group rece iving nutrition educa tio n from EFNEP who e stima ted food expe nditure s fro m reca ll), and the contro l group C ( a group who de layed their p artic ip ation for 3 to 6 months and e stimated food expenditure s from reca ll). Tab le 16. T he no nequiva le nt contro l group de sign. Time Group 1 (pretest) 2 (posttest) Experimental Group A (EFNEP participants 0 X 0 with register receipts) ------------------- - -----------------------------------------�--------------------Experimental Group B (EFNEP participants 0 X 0 estimating expenditures by recall) Control Group (Individuals not 0 0 receiving EFNEP education). 140 I A pretest consisting of three records, the Family Record (Appendix B 1 ), Dietary Recall Form (Appendix B2), and EFNEP Survey (Appendix B3), administered to all three groups was used to establish the comparability of the experimental groups and control group and to provide baseline data. A posttest, using the same records, was administered to groups A and B at the time the participant graduated. The same posttest was given to group C at about the same time groups A and B graduated so that a similar length of time passed between the pre- and posttests for all three groups. Participant Selection Recruitment Paraprofessionals recruited study participants as they typically would in EFNEP. Participants were recruited in a variety of ways including solicitation at sites where lowincome families frequent (e.g., WIC clinics) and referrals from agencies that work with low-income individuals and families. Occasionally, participants were recruited door to door using referrals from other EFNEP participants. There are two audiences in EFNEP, adult and youth. Only participants who qualify for the adult portion of EFNEP were recruited for this study. According to USDA, adult participants are defined as, "low-income homemakers/individuals living in either rural or urban areas, who are responsible for planning and preparing the family's food" (4, p. 7). The. Federal Poverty Income Guidelines published annually (5) or participation in any public assistance program for low-income individuals are used to determine who is low-income. 141 Paraprofessionals were instructed to recruit women over 1 8 years old. Although males do participate in EFNEP, women represent the majority of adult participants. In addition, paraprofessionals were instructed to exclude individuals under 1 8 years old because of difficulties associated with obtaining parental consent. Participants who were enrolled for less than two months also were excluded so that there was adequate time to collect food expenditure data for one month at entry and one month at exit. The original proposal for the study described using a neighbor/friend recruiting approach ( 6-7). Each time a person was enrolled in EFNEP they were asked to identify a neighbor or friend who was similar to them in race/ethnicity, age, income, and family characteristics. None of the paraprofessionals in the pilot study were able to recruit the comparison group using neighbor/friend referrals. Participants they enrolled were not able to provide the names of neighbors and friends who were similar to them who might qualify for EFNEP. Paraprofessionals described the process of matching the control group to the experimental groups as so difficult that they could not concentrate on providing education to their EFNEP participants. Because of the difficulty using the neighbor/friend recruiting approach, the comparison group was obtained by asking individuals who were willing to enroll in EFNEP to delay the start of their lessons. Or, individuals agreed to complete the pre- and posttests but did not choose to participate in EFNEP. 142 Enrollment When participants were enrolled in EFNEP, they were randomly assigned to one of the three groups. Each paraprofessional was instructed to recruit 1 8 participants. Six of the participants were enrolled in group A, six in group B, and six in group C. The first person recruited was assigned to group A, the second to group B, and the third to group C. If subjects refused to participate in the group assigned, they were given the opportunity to participate in EFNEP, but not in the study. As a token of appreciation for keeping food receipts and/ or a record of expenditures, individuals in group A were given a three-ring binder for EFNEP handouts they received from the paraprofessionals. Because they agreed to delay their participation, individuals in group C were given a token of appreciation that included one of the following items: a measuring cup for liquid ingredients, a set of measuring cups for dry ingredients, a vegetable brush and peeler, a set of three spatulas, a meat thermometer, and a plastic cutting board. These individuals were asked to choose one of the items during their first contact with the paraprofessional and were given the item they selected during the second contact when they completed the posttest. Sample Size The desired sample size of 341 estimated from guidelines for simple-random surveys (8-9). The sample size was based on population size, the permissible error and associated confidence leveL and the population proportion to be estimated. In 1 996-97, the number of females over 18 years of age who graduated from EFNEP within six 143 months was 3,276 (9). A 0.05 confidence level was selected because the research was considered exploratory and a O.0 1 level was considered too stringent. Based on percentages of participants who reported improvements in food and nutrition behaviors from data in the 1996-97 EFNEP (9), the researcher was interested in determining if at least 70% of participants in the study improved nutrition and food-related practices. Using these factors, it was determined that a sample size of 341 would be representative of the population within an acceptable error limit and take attrition into account. Each paraprofessional was assigned the same number of participants to ensure that the sample represented the population of EFNEP participants in Tennessee. Eighteen participants per paraprofessional also was considered a manageable number within the time frame of the study. At the conclusion of the study, only 470 participants had been recruited and 371 completed the required records. Only 16 of the 21 counties participating in the study recruited participants. The five counties that did not recruit participants stated they could not find individuals willing to participate. Time Horizon The time horizon for this investigation was six months based on how much time it takes for most EFNEP participants to complete the program. According to the 1996-97 Summary of Adult Participation Profiles for Tennessee (9), 80% of homemakers who graduated from EFNEP did so in six months or less. Six months allowed adequate time for multiple contacts. The average number of lessons in the 1996-97 Summary of Adult 144 Participation Profiles was 12. Almost half ( 49%) of those who completed the program had 14 to 20 educational contacts and 40% had 7 to 13 contacts. Cost data also were collected over the same period as benefits as recommended by Splett (1 1). Data Collection Data for this research were collected for costs and benefits. Data Collection for Costs Cost determination was completed in a series of steps that included: defining the service to be costed, defining activities related to implementing the program, specifying how the costs will be measured, gathering the cost data, and quantifying the costs. Before costs can be determined for any program, it is necessary to define the services. The services costed in this investigation included all nutrition education services provided to participants enrolled in the adult EFNEP program. These services included the lessons provided by paraprofessionals, foods and equipment needed for food demonstrations, publications and printed materials given to participants, audiovisuals used for education, computer dietary analyses and summaries of the EFNEP surveys, and any other materials needed for providing nutrition education. After defining the services to be costed, the next step was to define all of the ingredients or activities necessary to implement the service so they can be valued in monetary terms. Each activity was listed with personnel and materials needed as shown 145 in Table 17. Listing all required personnel and materials helped to identify the resources that were to be quantified. After all costs were identified, they were measured. For this investigation, direct costs were measured in several ways depending on the kind of expense. First, as many direct costs as possible were be obtained from a Grant and Contract Budget and Expenditure Report (Appendix E). These included all personnel costs, such as professional and paraprofessional salaries, salaries for support personnel (i.e., secretaries), staff benefits, and travel. The number of volunteer hours were estimated by the agent and paraprofessionals and entered into the EFNEP Reporting System. The market value of volunteers was calculated at the same rate per hour as the salaries of paraprofessionals, $5. 66 per hour. In addition to personnel costs, equipment, food supplies for demonstrations, office supplies, communication costs (i.e., long distance credit card calls, postage, telecommunications), maintenance and repairs, computer software, and other operating supplies were obtained from the monthly ledger sheets. Monthly printing costs charged to the adult EFNEP program also were listed on the report. However, not all the materials printed were used during the six-month period of investigation. Actual printing costs were estimated from a record of publications and handouts paraprofessionals and supervising Family and Consumer Science agents kept for six months. Cost of equipment used exclusively by EFNEP, such as computers and printers, was recorded from the Grant and Contract Budget and Expenditure Report. The annual cost of computers and printers was determined by dividing actual costs by five, the average number of years computers 1 46 Table 17. Activities related to implementation of EFNEP. Material Resources and Activity Responsible Person Equipment Required i Hiring professionals Administration (Personnel Telephone (including e-mail) Posting job announcement Assistant, Assist./ Assoc. Deans, Office supplies Interviewing applicants Section Leader, District ! Travel Completing personnel records Supervisor, county Extension Postage Leader, secretaries, personnel, Office equipment ( computer, business office) printer, copy machine) Hiring paraprofessionals Extension Home Economist, Telephone (including e-mail) Posting job announcement county Extension Leader, Office supplies Interviewing applicants secretaries, Administration Postage Completing personnel records (personnel, business Office equipment ( computer, office) printer, copy machine) Staff Development for Extension Specialists, speakers Telephone (including e-mail) professionals at professional meetings and Office supplies New Worker Orientation training, secretaries Travel I lnservice training Office equipment ( computer, Professional meetings and printer, copy machine) training Postage Registration fees Educational materials (books, etc.) Staff Development for Extension Home Economists, Office supplies paraprofessionals Extension Specialists, speakers Travel New Worker Orientation from other agencies, Office equipment ( computer, Inservice training secretaries printer, copy machine) Postage :1 Educational materials (publications, books, etc.) Food supplies 147 Table 1 7. (continued) Material Resources and Activity Responsible Person Equipment Required Recruiting Families Ex1ension Specialists (EFNEP Telephone (including e-mail) Collaboration with state Coordinator, EFNEP Office supplies agencies Nutritionist, Section Leader), Travel Collaboration with commW1ity county Home Economists, Postage agencies paraprofessionals, a�nistration Office equipment Door-to-door solicitation ( computer, printer, copy machine) Printed materials (pamphlets, etc.) Dietary Assessments paraprofessionals Office supplies Family Record Printed materials (records, etc.) 1 Food Recall I EFNEP Survey Teaching Families paraprofessionals Telephone Office supplies Travel Postage Office equipment (computer, printer, copy machine) Food supplies Educational materials ( curriculum, handouts, publications, etc.) Data Entry secretaries Office supplies Computer Interpreting and Extension Specialists (EFNEP Office supplies Summarizing Reports Coordinator, EFNEP Office equipment (computer Nutritionist), Extension Home and printer) Economists, paraprofessionals Telephone Postage 148 are used before they are replaced ( 12). Annual maintenance costs were added to the cost of computers. Monthly indirect costs for items, such as office space and utilities, were calculated from a rate determined by the accountant of the Institute of Agriculture and The University of Tennessee' s controller' s office using cost data collected during 1997. According to Splett and Caldwell ( 1 3), indirect expenses can be determined from a standard rate established by the agency. The University of Tennessee Institute of Agriculture proposed a standard rate that was used in this investigation to estimate the cost of buildings and office space, maintenance, heat and lights, administration, accounting, and personnel services. The total .direct costs were multiplied by the standard rate for indirect services to determine the total indirect costs. A rate of 2 1 . 79% for expenses on campus and a rate of 1 0. 79% for expenses off campus were multiplied by the monthly costs of implementing EFNEP to estimate indirect expenditures. Indirect costs were not calculated for equipment because depreciation was part of the indirect cost. These rates were proposed and submitted to the Department of Health and Human Services (DHHS) for approval during the time of this research. After monthly costs were measured for the time horizon of six months, they were multiplied by two to represent 12 months. One exception was the annualization of computers and printers. Another exception was the cost of a biennial conference, which was divided by two, the cost of the conference for one year. 149 Data Collection for Benefits Prior to actual data collection, a pilot test was conducted to determine if any procedures needed modification. Two counties that did not participate in the study served as pilot counties. Procedures were explained to 4 paraprofessionals who were asked to recruit 20 subjects, 5 subjects per paraprofessional. As a result of the pilot test, the procedure for recruiting the control group was modified as described earlier. Participants in the pilot study did not provide names of friends or neighbors. Because of these difficulties, paraprofessionals were instructed to employ the usual strategies to recruit subjects who would agree to delay their lessons until those in groups A and B completed their instruction. As a result of the pilot test, oral instructions were written as a protocol that EFNEP staff could refer to if they had questions (Appendix C). As part of the protocol, agents were instructed to add the value of foods on WIC vouchers. These values were included in the protocol. Following the pilot test, all supervising agents and paraprofessionals received training by the researcher to make sure protocol was followed and recalls and other records were as accurate as possible. This training was more in-depth compared to the usual training for paraprofessionals provided by agents. Training consisted of the detailed protocol for administering the records and submitting the required data in Appendix C. In addition, the researcher visited all counties participating in the study during the data collection period to monitor procedures. Data collection began in March 1997 and continued until January 1998. At the end of the data collection period, all data from outcome measures in the county database 150 files adult.db f, cklist.db f, and reca ll.d bf. were cop ied to a d iskette. The d ataba se structures for these files are d escrib ed in the ma nual, EFNEP Evaluation/Reporting System User 's Guide (14 ). Eac h of the databa se files were impo rted into the Statistic al Analysis S ystem (S AS ) (1 5) for analysis. Records A ll mea sures of effec tiveness c ame fro m da ta co llec ted with three record s : the Fa mily Reco rd ( App endix B 1 ), Dieta ry Reca ll Form (App endix B2), and EFNEP Survey ( App end ix B3 ). T hese record s are p art of the nationa l EFNEP Reporting System and were d evelop ed b y a committee of nutritionists and sta te EFNEP lead ers appointed by p rogram lead ers at CSREES of US DA . All record s were p ilot tested b efore their relea se in 1993 and 1994 . Instruc tio ns for co mpleting these record s are in the EFNEP Evaluation/Reporting System User 's Guide (14). Reco rd s were co mpleted b y the p arap rofessio na l and/or the p articipant. Fo r individua l instruc tion, p arap rofe ssionals co mpleted reco rd s in a p ersona l interview. When p articip ants were taught in g roup s, pa rticipa nts co mp leted reco rd s with instruc tio n from the p arap rofessio na l. After the record wa s comp leted , the p arap rofessiona l checked the reco rd for ac curac y and missing info rmatio n for sub missio n to the sup ervising ag ent for entry into a co mp uter in the county Extensio n offic e. All co mp uter entry wa s comp leted by a secretary trained by the researcher and/or sup ervising ho me eco nomist. Instructio ns for co mp uter d ata entry are in the EFNEP Evaluation/Reporting System User 's Guide (14 ). 151 The Family Record. The purpose of the Family Record was to collect demographic information about the participant, such as age, gender, race and ethnicity, residence, household size, household composition, age of children, type of instruction (individual versus group), and participation in other federal assistance programs. The Family Record was completed on the first visit with the participant and updated as needed when the participant graduated or terminated from EFNEP. At that time, the number of lessons taught was added to the record with an exit date. The Dietaiy Recall Dietary Recall Form was used to calculate nutrient intakes in a 24-hour period and the amount of money or other resources spent on food for the past month. Each paraprofessional was instructed to take a detailed 24-hour dietary recall consisting of foods and beverages eaten, amount eaten, and the meal type (i.e. , morning, midmorning, afternoon, evening, late evening). After the recall was taken, the paraprofessional coded the foods for computer entry. Foods and amounts were coded by the paraprofessional using a food dictionary that corresponds with the national EFNEP food database. Foods, serving sizes and identification numbers are listed in the EFNEP Evaluation/Reporting System User 's Guide ( 14). Duplicate coding of a random sample ofrecalls by the researcher was used to determine reliability. In a sample of 64 records, approximately 1 8% had errors in coding. Secretaries in EFNEP counties entered the codes into the computer system. Double entry was conducted on a random sample of 64 recalls by the researcher to determine reliability. Approximately 9% of records had errors in data entry. 1 52 Foods obtained from the 24-hour recall are analyzed for the six nutrients (protein, fat, vitamin A, vitamin C, iron, and calcium) using a computerized dietary analysis program included in the national reporting system. Specific nutrients to be analyzed in the national EFNEP Reporting System were selected by state EFNEP Coordinators and Extension nutritionists during the development of the reporting system. The reporting system has a food database that consists of nutrient values and servings of food groups for 1 373 foods. Nutrient values for most foods were taken from USDA food data banks. Pennington's Food Values of Portions Commonly Used ( 16) and manufacturers were used for foods not listed in the national data banks. Each state also has the ability to add foods commonly eaten in the area. The amount of money spent on food per month was taken from question eight on the 24-Hour Recall. For group A, paraprofessionals asked each participant to keep register receipts for food expenditures for two weeks out of each month. Collection of register receipts began one week after the family received their food stamps or other income and continued for 14 days. Participants who did not keep receipts or who did not have receipts for places such as fast food restaurants and vending machines were asked to fill in the amount on a record form (Appendix B4). All receipts were given to the supervising agent who added the receipts and the amounts written on the record and multiplied the total by two, the amount estimated spent on food for one month. This method is recommended by Joy ( 17) who determined that low-income families spend the largest portion of their food dollars during the week immediately following receipt of food stamps or other income and the smallest portion during the week just before 1 53 receiving their food stamps or other income. Therefore, the remaining two weeks represent an average weekly amount that can be multiplied by two to estimate total monthly expenditures. The reason this method was used was to reduce the burden of collecting register receipts. According to Joy, this reduces the burden of keeping receipts and increases the reliability of data. Once the receipts and other food expenditures were totaled and multiplied by two, the dollar value ofWIC vouchers received were added to the total. The total amount was added to the 24-Hour Recall Form and entered into the computer by the EFNEP secretary. For groups B and C, paraprofessionals were instructed to ask the participant to recall what they spent on food in the past month. This amount included food stamps, money, and WIC vouchers. This was reported on the 24-Hour Recall Form and entered into the computer by the EFNEP secretary. EFNEP Survey. The EFNEP Survey was used to collect data on food resource management practices and food selection and preparation practices. In the EFNEP National Reporting System, questions on the EFNEP Survey are divided and summarized in three clusters: food resource management practices, nutrition practices, and food safety practices. The following questions from the survey were used in the study to measure food resource management practices: Number I : How often do you plan meals ahead of time? Number 2 : How often do you compare prices before you buy food? Number 9 : How often do you run out of food before the end of the month? 1 54 Number 1 1 : In the past month, how many times have you done major grocery shopping? Number 12 : Of those times (from number 1 1), how many times did you shop with a list? Number 13 In the past month, did you ever have to cut the size of your children' s meals because there was not enough money to buy food? Questions 3, 6, 7, 8, and 14 on the survey are considered measures of nutrition behaviors, which reflect practices in food selection and preparation. In addition to the foods on the dietary recall, these questions were used to measure food selection and preparation practices: Number 3 : Number 6: Number 7: When deciding what to feed your family, how often do you think about healthy food choices? In the past two weeks, how often have you prepared foods without adding salt? In the past two weeks, how often have you read food labels to select foods with less salt or sodium? Number 8: In the past two weeks, how often did you read food labels to select foods with less fat? Number 14 : How much do you agree with this statement? "The food and nutrition needs of my family are being met." Some questions were omitted from the analysis. Questions 4 and 5 were excluded because the relate to food safety practices. Although safe food handling is one objective 1 55 of EFNEP, it was difficult to measure food handling practices with two questions only. Question 10 was not included in the analyses because when imported into the SAS data set, most participants' responses appeared as missing data. Question 15 , "Most of the other needs of my family are being met," was excluded because the question may not relate to food and nutrition practices. Expression of Outcome Measures The outcome measures were expressed in the following ways: Food resource management practices. Food resource management practices were measured from data on food expenditures and from questions on the survey. Dollars saved on food expenditures was expressed in dollars calculated by subtracting the amount of money and other resources spent per family on food at program exit from program entry. Subjects in group A were instructed to keep register receipts and/or a record of food expenditures during the first month of participation in the EFNEP and in the last month of participation. Subjects in group B were instructed to recall food expenditures during the month prior to EFNEP enrollment and during the month prior to graduation. Subjects in group C also were instructed to recall food expenditures during the month prior to enrolling in EFNEP and the month prior to the time they actually began EFNEP education. Questions 1 , 2, 9, 1 1 , 12, and 13 were measures of food resource management practices. Questions 1 , 2, and 9 required one of five responses: do not do, seldom, sometimes, most of the time, and almost always. Therefore, responses were scores from 156 1 to 5. Questions 1 1 and 12 required a number from O to 14 and, therefore, were scores from O to 14. Question 13 was a yes or no response. Food selection and preparation. Changes in food selection and preparation were measured using the 24-hour food recall and questions related to nutrition practices on the survey. Food intake was expressed as the difference between numbers of servings from the five food groups and the sweets and fat group at program exit from mean servings at program entry taken from. the 24-hour dietary recall. Nutrient intakes were expressed as the difference between nutrients consumed at program exit and nutrients consumed at program entry. Nutrition practices were expressed as mean scores from 1 to 5 on question 3, 6, 7, 8, 14 on the survey. Relating Costs to Outcomes To conduct a cost-benefit analysis, benefits and costs must be expressed in monetary terms. Food expenditure was the only outcome measured in dollars and, therefore, was the only outcome included in the cost-benefit analysis. All other outcomes were expressed in an array of measures to demonstrate the effectiveness of EFNEP education. Findings from cost-benefit analyses frequently are presented as benefit-cost ratios. However, expressing findings as net present value (NPV) is the recommended method (1 8-20). NPV is calculated by subtracting discounted costs from discounted benefits using the following formula: 1 57 where r = discount rate t = time period T = time frame NPV= L @enefits - Costs) t = o ( 1 + r)' In this study, the benefit was the difference between food expenditures for one month at program entry and program exit. Cost was the amount of money spent per participant to implement EFNEP obtained by dividing the total cost of EFNEP for one year by the number of families who graduated in one year. The discount rate was 7% based on recommendations for federally funded programs by the Office of Management and Budget (20). The time period and time frame was five years, the longest period for which retention ofEFNEP benefits has been documented (21). Data Analysis Statistical analyses using SAS for Windows, Release 6. 12 ( 1 5) were conducted in this study to determine if changes in food resource management practices and food selection and preparation practices were the result of participation in EFNEP. Frequency tables were constructed for demographic variables such as gender, race, residence, participation in public assistance programs, total number of persons in the family, number of adults other than the participant in a family, number of children in a family, the ages of children, the number of lessons received, and type of instruction. Descriptive statistics were used for measurements such as age of participant, number of children, total family 158 size, servings from food groups, nutrient intakes, food expenditures, and behaviors from the survey. Mean food and nutrient intakes at program entry were compared for each of the three groups to determine if they were comparable. A nonparametric test was used because data were not normally distributed. Once it was determined that the three groups were comparable at program entry and the two experimental groups were similar at exit, the two experimental groups were combined into one group. Thereafter the KruskalWallis Test was used to compare the means of the two groups: the combined experimental group and the control group. An alpha level of 0.01 was used on all tests with food and nutrient data to control the probability of making a type I error by rejecting the null hypothesis that there were no differences between groups. Average scores for survey questions also were compared for each of the three groups using a nonparametric test to determine if the groups were comparable. Like the food and nutrient data, scores on survey questions were comparable for the groups; therefore, the experimental groups were combined for subsequent tests. The nonparametric test, Kruskal-W allis, was used to determine if there were differences in average scores for the two groups, i.e., the combined experimental group and the control group. An alpha level of 0.01 also was used for tests on survey data. Food ExpendiJure Data The difference between the amount of money or other resources spent on food at program entry and program exit was calculated for groups A, B, and C. Tests for 1 59 normality showed food expenditure data were not normally distributed. However, a histogram showed data were symmetrical. Because of the symmetry of data and because the sample was representative of the total population of EFNEP participants in Tennessee, the parametric test, Analysis of Variance (ANOVA) was used to determine if there were differences among sample means for the three groups (22). The General Linear Model (GLM) was performed using the independent variable (group A, B, and C) as the classification variable and the difference in food expenditures as the dependent variable used in the model. An alpha level of 0.05 was chosen as the significance level. When significant differences among the three groups were detected, the Tukey' s Studentized Range Test for more than two groups was used to identify which groups were different. Analysis of Covariance (ANCOVA) was used to determine if there was an interaction among the dependent variable group and how often a participant used shopping lists when she did major shopping. This particular covariate was tested in the interaction because at program entry the subjects in the combined experimental group more often made a shopping list compared to subjects in the control group. Sensitivity Analysis Sensitivity analysis uses mathematical calculations that test the degree of influence different factors have on the outcome of the cost-benefit analysis. Sensitivity analysis was conducted in this study by varying the method by which food expenditures were determined (i.e., participant recall versus collecting register receipts and/or records 1 60 of expenditures). A sensitivity analysis was conducted by varying the time frame behaviors learned in EFNEP were retained, either 3 years or 5 years. In a third sensitivity analysis, the discount rate was varied to determine the effect of a 3%, 5%, and 7% rate on the NPV. 1 6 1 REFERENCES 1 . &ting Right is Basic. Third Version. Lansing, :MI : Michigan Cooperative Extension Service, Michigan State University; 1995. 2 . The Food Guide Pyramid. Washington, DC: Human Nutrition Information Service, US Dept of Agriculture, 1992. Home and Garden Bulletin. 252. 3 . Campbell OT, Stanley JC. Experimental and Quasi-Experimental Designs for Research. Chicago, IL: Rand McNally; 1963 . 4. Expanded Food and Nu'trition Education Program: Guide to Program Management and Supervision. Washington, DC: Cooperative Extension System, US Dept of Agriculture; 1983 . 5 . Department of Health and Human Services. Poverty Guidelines. Washington, DC: US Dept of Health and Human Services; 1998 . 6. Bell BP, Goldoft M, Griffin PM, Davis MA, Gordon DC, Tarr PI, Bartleson CA, Lewis Ill, Barrett TJ, Wells JG, Baron R, Kobayashi J. A multistate outbreak of Escherichia coli 0157:H7-associated bloody diarrhea and hemolytic uremic syndrome from hamburgers. J Am Med Assoc. 1994;272; 1349-1353 . 7. Wacholder S, McLaughlin JK, Silverman OT, Mandel JS. Selection of controls in case-control studies. Am J Epidemiol. 1992; 135 : 1 0 19- 1028. 8 . Wang, MQ, Fitzugh E, Westerfield RC. Determining sample size for simplerandom surveys. Health Values. 1995; 19 :53-56. 9. Wunsch DR. Survey research: Determining sample size and representative response. Business Education Forum. 1986:February:3 l-34. 10. Summary of Adult Participant Profiles. EFNEP Reporting System. Knoxville, TN: The University of Tennessee Agricultural Extension Service; 1 997. 1 1 . Splett PL. The Nu'frition Practitioner's Guide to Cost Effectiveness Analysis of Nu'frition Interventions. Arlington, VA: National Center for Education in Maternal and Child Health; 1 996. 12. Levin HM. Cost-Effectiveness: A Primer. Newbury Park, CA:Sage Publications, Inc. ; 1983 . 162 13 . Splett PL, Caldwell M. Costing Nutrition Services: A Workbook. Chicago, IL: Region V, US Dept of Health and Human Services; 1 985. 14. EFNEP Evaluation/Reporting System User's Guide, Version 3. 0. Laurel, l\.ID: Cooperative Extension System, US Dept of Agriculture; 1993 . 1 5 . Statistical Analysis System. Release 6. 12. SAS® Institute Inc. Cary: NC. 1 996. 1 6. Pennington JAT. Food Values of Portions Commonly Used. JB Lippincott Co. Philadelphia, PA; 1994. 1 7. Joy AB. Development of a simplified methodology to measure the effect of food purchasing practices on nutrient costs. Presented at: Cooperative State Research, Education and Extension Service Meeting, Nutrition Education for Diverse Audiences. February 10-14, 1 997; Houston, TX. 18 . Nas T. Cost-Benefit Analysis: Theory and Application. Sage Publications, Inc. Thousand Oaks, CA; 1996. 19. Clemmer B, Haddix AC. Cost-benefit analysis. In: Haddix AC, Teutsch SM, Shaffer PA, Dufiet D, ed. Prevention Effectiveness: A Guide to Decision Analysis and Economic Evaluation. New York: Oxford University Press; 1 996:85- 102. 20. Government Accounting Office. Circular No. A-94. Washington DC: US Office of Management and Budget; 1 992. 2 1 . Nierman LG. A Longitudinal Study of the Retention of Foods and Nutrition Knowledge and Practices of Participants from Michigan Expanded Food and Nutrition Education Program. East Lansing, :MI: Michigan State University; 1986. 22. Schlotzhauer SD, Littell RC. SAS® System for Elementary Statistical Analysis. Cary, NC: SAS® Institute, Inc. ; 1 987. 1 63 APPENDIX B Participant Records 1 64 Expanded Food and Nutrition Education Program Adult Enrollment Form (Family Record) Fill out for each family at ENTRY and again at EXIT. Keep in family file after it's reviewed by Agent and sent to Secretary for computer entry . . -. . � .. }�� .•. : .•. : . . :.: . . :. . . -.'. .. .i.:.:.�.i.�.:.i .... J.:.�:.:.'..:.:.'..: . . . J�·.· .. �.. �. · .. :.: ... . ·.= . . . :.:.:.: ... :.!.�.: . . i. . :.:.i.· . . �. .•. :.:.1.� . . :.: . . . :.:.:_._;_:,�.·.1.:�1 . . :.:J.:.� . . . :.:.·.:.: . . i.: ... l.l . . :.l.'..! .. '..: .... !.:.:.:.:.:.i ... . :.·.:.l . . l.:. ��;:;:::;:::::;:�:;:::;:::::;:::::::�:::::::·:::::::_'._:_: _ _ l.:.l.l.l 5. Enrolled In EFNEP before? (circle Y for Yes, N for No) y N 6. If Yes, did you receive a Certificate of Completion? Y N (First) (Ml) (Last) a) Name ___________________________ _ b) Street _______________________ _ c) City _____________ Zip _____ _ d) Phone 11. Race: Check the category you identify with __ 1-00 White (non-Hispanic} __ 2-00 Black (non-Hispanic} __ 3-00 Am Indian/Alaskan Native __ 4-00 Hispanic __ 5-00 Asian or Pacific Islander 12. Place of Residence: circle 1 Farm 2 Towns under 10,000 & rural non-farm 3 Towns & Cities 1 0,000 to 50,000 4 Suburbs of Cities over 50,000 5 Central Cities over 50.000 7. Age: __ 9. Pregnant: y N 8. Sex: F M 10. Breast-Feeding: y N 13. Total Household Income Last Month: $ ___ _ 14. Household Members: Children by Age List First Name of Children (through Age 19) Age (Years} 15. Number of Other Adults in Household (don't count Homemaker) 1) 2) 3) 4) 5) 6) 7) Bl.I. Family Record Form (continued on the next page) 1 65 18. Date: 19. Assistance programs that the Family Participates in at ENTRY: (circle) WIC/CSFP y N Food Stamps y N FDPIR y N Commodities y N Head Start y N Child Nutrition y N AFDC y N Other y N (Specify) Comments: - -( um plt h· E,it information 0111) " lu·n ka, in!! EF�EP Pro!,!mm ___ 2_0._E_xi_t _Da_te_: --------11 22. Did your family receive 21. Exit Reason: (circle) 1 Educational Objective Met 2 Returned to School 3 Took Job 4 Family Concerns 5 StaffVacancy 6 Moved 7 Lost Interest 8 Other assistance as the result of a referral or suggestion from EFNEP personnel? YN lfyes, check all that apply: WIC/CSFP Food Stamps FDPIR Commodities Head Start Child Nutrition AFDC Other (Specify) Bl.2. Family Record Form (continued) 166 HOMEMAKER'S 24-HOUR FOOD RECALL (Form A) (Use for Nutritional Method 1 - When Foods Will Be Pre-coded) :·�·-···.· · , · :· , , ·.··.··.··.··.· ··· ·.·.·.· ;; :;···· · ·, ' :· ·. : ·.··.· · · ·.·,·.·;: ,L,,, ,·.· ·.·,·.··.··.· : 2. Date Taken• 5. Pregnant D Yes D No 6. Nursing D Yes D No 7. Takes Nutritional Supplements D Yes o No If "Yes• List Type: 8. Money Spent on Food Last Month: $ ___ _ MEAL TYPE Morning = 1 MidMoming = 2 Noon = 3 MEAL TYPE Afternoon = 4 Evening = 5 Late Evening = 6 � �TIONS 1!111.1 sl = slice 10. What did homemaker eat and drink in the last 24 hours? (To be filled out by P•aprofessional or Homemaker) FOOD ITEMS AND DESCRIPTION (Usts all foods and beverages. List separately main ingredients in mixed dishes.) �N� �� IIIIHlli B2. Food Recall Form 1 67 Name Date EFNEP SURVEY Check the box after entry or exit Entry 0 Exit 0 This is a survey about ways you plan and fix foods for your family. As you read each question, think about the recent past. Some questions ask you to think about the past month . Others ask you to think about just the past two weeks. Please put a check in the box Do Not Seldom Some- Most of Almost that best answers each Do times the Time Always question . ( 1 ) How often do you plan meals ahead of time? (2) How often do you compare prices before you buy food? (3) When deciding what to feed your family, how often do you think about healthy food choices? (4) This question is about meat and dairy foods. How often do you let these foods sit out for more than two hours? (5) How often do you thaw frozen foods at room temperature? (6) In the past two weeks, how often have you prepared foods without adding salt? (7) In the past two weeks, how often did you read food labels to select foods with less salt or sodium? (8) In the past two weeks, how often did you read food labels to select foods with less fat? B3.1. EFNEP Survey Form (Front) 1 68 Please put a check in the box Do Not Seldom Some- Most of Almost that best answers each Do times the Time Always question. (9) How often do you run out of food before the end of the month ? ( 1 0) How often in the past two weeks have your children had times something to eat in the morning, within 2 hours of waking up? ( 1 1 } In the past month, how many times have you done times major grocery shopping ? ( 1 2} Of those times, how many times times did you shop with a l ist? ( 1 3 } In the past month, did you Check one: D yes ever have to cut the size of D no your children's meals because there was not enough money to buy food? ( 1 4} How much do you agree Check one: D Strongly Disagree with this statement? uThe food D Disagree and nutrition needs of my D Agree family are being met. " D Strongly Agree The previous questions asked you about foods and nutrition. Now please think about other aspects of family care (such as health, other money management, parenting, or personal growth . } How much do you agree with this statement: ( 1 5} Most of the other needs of Check one: my family are being met. D Strongly Disagree D Disagree D Agree D Strongly Agree B3.2. EFNEP Survey Form (Back) 169 I ,__. .....J 0 = � ,_. . � Q Q Q,. g � = Q,. .... = ... Q a. ... Q Begin Date: ----------(month, day and year) End Date: __________ _ (month, day and year - s_hould be two weeks from begin date) Directions: Please keep all receipts for food purchased in this envelope. Include receipts from everyone who lives together in your household and eats from your household food supply. If you do not have a receipt, fill in the circle under the name of the type of place you and your household purchased the food and and the amount of money spent in the box below. Use a separate line for each purchase. Supermarket Convenience Restaurant I Fast Food I Farmer's I Vending I Other Total Purchase Store Market Machine Amount Example 0 0 0 • 0 0 0 ODD.DD 0 0 0 0 0 0 0 ODD.DD 0 0 0 0 0 0 0 ODD. DD 0 0 0 0 0 0 0 ODD. DD 0 0 0 0 0 0 0 ODD . DD 0 0 0 0 0 0 0 ODD . DD 0 0 0 0 0 0 0 DOD . DD 0 0 0 0 0 0 0 ODD . DD 0 0 0 0 0 0 0 ODD . DD 0 0 0 0 0 0 0 ODD. DD - -I Q> D I i � D D D D D D D D D D D D D D D D D D D D D D D D D D D D D D D � 1 D D D D D D D D D D D D D D D D D D D D D D D D D . D D D D D D D I 1 D D D D D D D D D D D D D D D D E,-1 ID 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 I - -ij � i 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 z .. l$ ID .:ia 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 J I I ! 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 : I � -� s Q> 1 ! 0 0 0 0 0 0 0 8 rJ.i 0 0 0 0 0 0 0 0 0 -l$ I t 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 J5 B4.2. Food Expenditures Form (Back) 171 APPENDIX C Instructions for Conducting a Cost-Benefit Analysis 1 72 Instructions for Conducting an EFNEP Cost-Benefit Analysis What is cost-benefit analysis? What are program outcomes? How do we measure program outcomes? Introduction Cost benefit analysis is a type of program evaluation that examines the amount of money spent to achieve program outcomes. Program outcomes are what you try to achieve as a result of the program. The major outcomes we try to achieve in EFNEP are: 1 . Improved food resource management practices - the use of food resources (money, food stamps, WIC vouchers, etc.) to enable families to obtain enough nutritious food. 2. Improved nutrition practices - the consumption of foods that meet nutritional needs. 3 . Improved food safety practices - following the recommendations for safe food handling. All of these outcomes can lead to improved health for participants, the ultimate outcome of participation in EFNEP. Program outcomes are measured with the National EFNEP Reporting System using three records: 1) the Family Record, 2) the 24-Hour Food Recall, and 3) the EFNEP Survey. We collect information for all three of the major outcomes. 1 . Food resource management practices are measured by: a. question 8 on the 24-Hour Food Recall ("Money spent on food last month.") b. questions I, 2, 9, 1 1 , 12, 13 , and 14 on the EFNEP Survey ct.1. Instructions for Conducting a Cost-Benefit Analysis 173 Who wi l l be included i n the cost-benefit analysis? Who wi l l not be i ncluded i n the cost-benefit analysis? 3 groups of participants 2. Nutrition practices are measured by: a. the 24-Hour Food Recall b . questions 3 , 6, 7, 8, and 10 on the EFNEP Survey 3 . Food safety practices are measured by questions 4 and 5 on the EFNEP Survey. This cost-benefit analysis should show that for every dollar we spend on a participant, the participant saves X number of dollars on food. Since spending less money does not necessarily mean that a person is spending more wisely, we will use improvements in food resource and nutrition practices to show that participants save money and improve their diets. The following EFNEP participants will be included in the cost-benefit analysis: 1 . adults 2. females 3 . participants who are enrolled for more than 2 months 4. participants taught in groups and individually The following EFNEP participants will not be included in the cost-benefit analysis: 1 . participants under 1 8-years-old 2. males 3 . participants enrolled for less than 2 months Participants will be placed in three groups. i i i II II II Group A Group B Group C Cl.2. Instructions for Conducting a Cost-Benefit Analysis (continued) 1 74 How many people wi l l be enro l led i n the study? I . Group A - will enroll in EFNEP and receive education as usual, but will be asked to keep food receipts. 2. Group B - will enroll in EFNEP and receive education as usual. 3 . Group C - will enroll in EFNEP but will be asked to wait a few weeks to begin lessons. Each program assistant will be enrolling ll persons for the study. 6 people to Group A 6 people to Group B 6 people to Group C Procedures for Data Collection Enro l l i ng participants Incentives for Group C The procedure for enrolling participants will be as follows: I . Enroll new participant and assign them to Group A 2. Enroll a second participant and assign them to Group B 3 . Enroll a third participant and assign them to Group C It is important that you do not choose which group to assign the participant. They are assigned to a group by chance. If you cannot convince a person to collect the food receipts or a person does not want to delay lessons, then assign them to the next group. For example, if a person does not want to delay their lessons, assign them to Group A and assign the next person to group C. Your goal is to have 6 people in each group. Because Group C will be asked to delay their education, they will be offered a small token of appreciation. Ask them to select an item from the list you have been provided. Be sure their name, your name, the county, and Cl.3. Instructions for Conducting a Cost-Benefit Analysis (continued) 175 How wi I I you know when to begin lessons for Group C? Consent Forms Completing the EFNEP records the date are written on the selection form. Make a copy of the selection form and send it to Janie Burney as soon as possible to allow enough time to order the items. They are to be given their selection at the time you begin their lessons. You begin lessons with Group C when either a Group A or Group B participant completes their last lesson. The Group A or Group B participant should be enrolled at about the same time. You can tell the Group C participant that you will begin lessons at the time you believe the Group A or Group B participant will graduate. You will be given brightly-colored self-stick notes to attach to the Family Records of the participants in Group A and B that were enrolled at about the same time you enrolled a participant in Group C. Write on the note: "Begin lessons for when this participant graduates." Choose the time either the participant in Group A or the participant in Group B receives her last lesson. All 18 people enrolled in this study are required to sign a consent form. This form is to be signed by the participant and sent to the EFNEP agent who will forward them to Knoxville. You have a consent form for each group. Every program assistant and agent should sign the forms and include her address and phone number. Every participant should have a copy of the consent form to keep in case she needs to contact anyone concerning the study. To keep in touch with the participant in Group C during the time you are not visiting them, send them a postcard provided by the state EFNEP office each month. This will serve to remind them that you will be visiting with them agam. Participants in Group A, B, and C will complete the Family Record, 24-Hour Recall, and the EFNEP Survey when they are enrolled. These forms will be sent to the EFNEP agent Cl.4. Instructions for Conducting a Cost-Benefit Analysis (continued) 176 Entering data i n the computer How wi 1 1 the food receipts be co l lected? as soon as possible after completion, or, by the next time you are in the office. The same participants will complete the same records at the time they graduate (for Group A and B), or, when they begin lessons (for Group C). The records taken for Group C at the time they begin lessons will be entered in the computer as their exit records but will be considered their entry records. Write on the three EFNEP records whether the participant is in Group A, B or C. If a person does not want to be in the group she should be assigned, write this on the Family Record under the "Comment" section. For example, "Did not want to delay lessons. Participant assigned to next available group, Group A." Records will be entered by EFNEP secretaries as usual except separate directories will be used for each group. All participant recoreds entered during the study will be in the regular directory ( example: DATA9697), a directory created for Group A (example: GROUPA), a directory created for Group B (example: GROUPB), or a directory created for Group C (example: GROUPC). There will be a total of four directories. Three new directories must be created. Participants in Group A will be asked to keep their food receipts for two weeks out of the month. They will be asked to keep receipts during the first month they are enrolled in EFNEP and for a month after they complete their last lesson. Which two weeks they collect receipts depends on when they receive their food stamps or other source of income. For example: 1 . If they receive their food stamps ( or other income) at the first of the month, they begin keeping food receipts Cl.5. Instructions for Conducting a Cost-Benefit Analysis (continued) 177 What food receipts wi l l be co l lected? WIC vouchers at the beginning of the second week and continue through the third week. 2 . If they receive their food stamps (or other income) the second week of the month, they should begin keeping receipts one week later for a total of two weeks. 3 . If they receive income weekly or twice a month, they should keep food receipts the second and third weeks of the month. If at the time you enroll a person in Group A, they have received their food stamps or income and it is too late to collect their food receipts for the month, do not include them in the study. Food receipts should be collected for every member of the household. Receipts should be kept for supermarkets, smaller grocery stores, convenience stores, deli's, fast food restaurants, and other restaurants. Ask the participant to identify items on the receipts that are nonfood items. If a person forgets or is unable to provide a receipt (for vending machines, for example), ask them to write down where they purchased the food and how much they spent on the form provided. This includes food purchases for every person in the household. Ask them to keep all receipts in the University of Tennessee envelope provided. Send the envelope with receipts and other documentation of money spent to the EFNEP agent. According to the instructions for the National EFNEP Reporting System, the value of the vouchers received from WIC should be part of the total amount spent on food (item 8 on the Family Record). This has not been done consistently, if at all. For the purposes of this study, we will include the value of WIC vouchers for all homemakers who participate in Group A, B, and C. At the beginning Cl.6. Instructions for Conducting a Cost-Benefit Analysis ( continued) 1 78 Determining the total amount spent on food of the new reporting year in October, we will need to do this with all of our homemakers. Ask the homemaker who receives WIC vouchers in the family [pregnant woman, postwoman, infant (birth to 6 mos.), infant (6 to mos. to 1 yr.), child ( 1 to 5 yrs.)], nursing woman. The value of each vouchers for each person per month is shown below: Packa&e FamilI Member Value Jler month A and A+ Pregnant women $52 B Post-partum women $40 C Infants (birth to 6 mos.) $95 D Infants ( 6 mos to 1 yr.) $ 106 E and E+ Children ( 1 to 5 yrs.) $54 G and A+ Nursing women $63 For this study, write the type of family member receiving WIC on the 24-Hour Food Recall. The agent will add this to the total amount spent on food each month. This is to be done by the EFNEP agent. After the participant collects receipts and gives them to the program assistant with the form for purchases without receipts, they should be added together. Be sure to subtract purchases for nonfood items, such as soap and pet food. Once the nonfood items are subtracted, the tax charged will need to be figured by multiplying the county tax rate by the total amount spent on food. This is to be done by the EFNEP agent In addition to the receipts collected, the value of WIC vouchers must be added to the other food expenditures. Part of a cost-benefit study requires that the amount of Cl.7. Instructions for Conducting a Cost-Benefit Analysis (continued) 179 Expenses for pub I i cations/handouts Setting up data directories on the computer money spent for programming be documented to determine how much money we spend per person. Most expenses can be obtained from expenditure reports in Knoxville, however, expenses for publications and handouts need to be figured for participants in the study. For the purposes of this study, the EFNEP agent will keep a log of publications and handouts used with the participants in Group A, B, and C during the study. This would be a list of the publication or handout name and how many were distributed. At the end of the study, they will be forwarded to Janie Burney. In order to keep the records for the participants in the cost-benefit study separate from the other EFNEP participants, new directories must be set up on the computers. A directory for each county must be set up for Group A, B, and C. To set-up these directories, follow these steps: I . Go to File on the EFNEP menu bar. 2. Select 3. View/Update Data Directories under File menu. 3 . Press 4. Press Insert to add a directory 5 . Type GROUPA by Name 6. Type C:\EFNEP\GROUPA\ by data directory. 7. Press 8 . Press Y when it tells you C:\EFNEP\GROUPA\ does not exist and asks you if you want to create it. 9. Do not change the name of the Tables Directory. I O. Press Repeat steps 4 through 1 0 for Group B and Group C. For computers with more than one county or subgroup on the computer: Cl.8. Instructions for Conducting a Cost-Benefit Analysis (continued) 1 80 ' Name the new directories according to what county or subgroup they will be in. For example: Benton County might be BGROUPA and Henry might be HGROUPA These can be up to 8 characters. Cl.9. Instructions for Conducting a Cost-Benefit Analysis (continued) 1 8 1 APPENDIX D Participant Consent Forms 1 82 GroupA Date: Dear We would like to invite you to participate in a study. The study involves research about the Expanded Food and Nutrition Education Program (EFNEP), a nutrition education program taught by The University of Tennessee Agricultural Extension Service. We want to know if learning about foods and nutrition from EFNEP helps you save money on food, helps you improve the food choices you make, and helps you learn about preparing foods. Your participation is expected to last from three to six months. During that time you will be contacted about 12 times. This is the average number of times an EFNEP participant receives a lesson. The exact number of lessons will depend on how many you would like to receive. During this time a program assistant, will teach you about foods and nutrition. This can be done individually in your home or in group meetings with other homemakers. We will pick a regular meeting time that is convenient for you. There is no charge for EFNEP. When you sign up for EFNEP we will be asking you a few questions about yourself and your family, the kinds of foods you eat, and how you shop for food, plan your meals, and prepare your food. Your answers will be written down. This will help the program assistant and you decide on what you would like to learn about foods and nutrition. At this time we also will ask you to keep a record of the amount of money your family spends on food for one month by keeping your grocery receipts and writing down how much you spend on food away from home. When you finish all the lessons, we will ask you the same questions we asked when you signed up for EFNEP. And, again we will ask you to keep track of how much you spend on food for one month. This will help us learn if you shop, plan meals, or prepare foods any differently than you did when you signed up for EFNEP. If you agree to participate in this study, you will not be at risk, either physically or mentally. We will do no more than ask you some questions and teach you about foods and nutrition. Since your name and other infonnation will be written down on our records, we want to assure you that the information will be kept confidential. The records will be locked up in a cabinet or drawer at your county Extension office. No one but the program assistant, the EFNEP Home Economist who supervises EFNEP in your county, the EFNEP secretary who will put the information into a computer, and the researcher (Janie Burney) at the University of Tennessee will know your name. Twelve months after the study your records will be destroyed. This consent form will be kept in the researcher's office for three years after the study is completed. Dl.1. Consent Form for Group A (Front) 18 3 We anticipate that you will learn many things about foods and nutrition. By the time you finish your lessons, you may be feeding your family better with less money! As part of the study, we will give you a computer analysis about your diet. You will receive a notebook that you can put all of the handouts (with recipes) that you receive from the program assistant. Participation in this study is voluntary. You may refuse to participate at any time. It will not affect any assistance you may be receiving from other programs, such as WIC and food stamps. Should you need to contact anyone about the study, you may contact the researcher (Janie Burney), the program assistant, or your county EFNEP Home Economist. Their addresses and phone numbers are given below. Sincerely, Janie Burney EFNEP Coordinator 1 19 Morgan Hall PO Box 1071 Knoxville, TN 37901 1-423-974-7402 EFNEP Program Assistant EFNEP Home Economist I have read this consent form and agree to participate. Signature of Participant Dl.2. Consent Form for Group A (Back) 184 Group B Date: Dear We would like to invite you to participate in a study. The study involves research about the Expanded Food and Nutrition Education Program (EFNEP), a nutrition education program taught by The University of Tennessee Agricultural Extension Service. We want to know if learning about foods and nutrition from EFNEP helps you save money on food, helps you improve the food choices you make, and helps you learn about preparing foods. Your participation in the study is expected to last from three to six months. During that time you will be contacted about 12 times. This is the average number of times an EFNEP participant receives a lesson. The exact number of lessons will depend on how many your would like to receive. During this time a program assistant, will teach you about foods in nutrition. This can be done individually in your home or in group meetings with other homemakers. We will pick a regular meeting time that is convenient for you. There is no charge for EFNEP. When you sign up for EFNEP we will be asking you a few questions about yourself and your family, the kinds of foods you eat, and how you shop for food, plan your meals, and prepare your food. Your answers will be written down. This will help the program assistant and you decide on what you would like to learn about foods and nutrition. When you finish all the lessons, we will ask you the same questions we asked when you signed up for EFNEP. This will help us learn if you shop, plan meals, or prepare foods any differently than you did when you signed up for EFNEP. If you agree to participate in this study, you will not be at risk, either physically or mentally. We will do no more than ask you some questions and teach you about foods and nutrition. Since your name and other information will be written down on our records, we want to assure you that the information will be kept confidential. The records will be locked up in a cabinet or drawer at your county Extension office. No one but the program assistant, the EFNEP Home Economist who supervises EFNEP in your county, the EFNEP secretaiy who will put the information into a computer, and the researcher (Janie Burney) at The University of Tennessee will know your name. Twelve months after the study your records will be destroyed. This consent fonn will be kept in the researcher's office for three years after the study is completed. We anticipate that you will learn many things about foods and nutrition. By the time you finish your lessons, you may be feeding your family better with less money! As part of the study, we will give you a computer analysis about your diet. D2.1. Consent Form for Group B (Front) 1 85 Participation in this study is voluntaiy. You may refuse to participate at any time. It will not affect any assistance you may be receiving from other programs, such as WIC and food stamps. Should you need to contact anyone about the study, you may contact the researcher (Janie Burney), the program assistant, or your county EFNEP Home Economist. Their addresses and phone numbers are given below. Sincerely, Janie Burney EFNEP Coordinator 1 19 Morgan Hall PO Box 1071 Knoxville, TN 37901 1-423-974-7402 EFNEP Program Assistant EFNEP Home Economist I have read this consent form and agree to participate. Signature of Participant D2.2. Consent Form for Group B (Back) 186 Group e Date: Dear We would like to invite you to participate in a study. The study involves research about the Expanded Food and Nutrition Education Program (EFNEP), a nutrition education program taught by The University of Tennessee Agricultural Extension Service. We want to know if learning about foods and nutrition from EFNEP helps homemakers like you save money on food, improve the food choices they make, and learn to prepare foods. Your participation in the study is expected to last from three to six months. During that time you will be contacted two times. Once when you begin to participate in the study and once when you end your participation in the study. If you agree to participate, an EFNEP program assistant will ask you a few questions about yourself and your family, the kinds of foods you eat, and how you shop for food, plan your meals, and prepare your food. Your answers will be written down. In about three to six months, the program assistant will visit you again to ask you the same questions she asked the first time. At that time you will be invited to participate in the EFNEP program to learn about feeding your family nutritious foods even on a tight budget This is free. If you agree to participate in this study, you will not be at risk, either physically or mentally. We will do no more than ask you some questions. Since your name and other information will be written down on our records, we want to assure you that the information will be kept confidential. The records will be locked up in a cabinet or drawer at your county Extension office. No one but the program assistant, the EFNEP Home Economist who supervises EFNEP in your county, the EFNEP secretary who will put the information into a computer, and the researcher (Janie Burney) at The University of Tennessee will know your name. Twelve months after the study your records will be destroyed. This consent form will be kept in the researcher's office in Knoxville for three years after the study is completed. Participation in this study is voluntary. You may refuse to participate at any time. It will not affect any assistance you may be receiving from other programs, such as WIC and food stamps. You also may choose to sign up for EFNEP before the study is over. But, it would really help us learn more about how we can make EFNEP beneficial to you if you wait to sign up when the study is over. D3.1. Consent Form for Group C (Front) 1 87 Should you need to contact anyone about the study, you may contact the researcher (Janie Burney), the program assistant, or your county EFNEP Home Economist. Their addresses and phone numbers are given below. Sincerely, Janie Burney EFNEP Coordinator 1 19 Morgan Hall PO Box 1071 Knoxville, TN 37901 1-423-974-7402 EFNEP Program Assistant EFNEP Home Economist I have read this consent form and agree to participate. Signature of Participant D3.2. Consent Form for Group C (Back) 1 8 8 APPENDIX E Grant and Contract Budget and Expenditure Report 1 89 -t!!j ..... . = .... f» = Q,, r'} 0 a tl � = = Q,, f» = Q. t!!j = Q,, .... .... = "l:S 0 THE UNIVERSITY OF TENNESSEE OFFICE OF THE TREASURER UPON RECEIPT, ITEMS APPEARINO ON THIS REPORT SHOULD BE VERIFIED FOR ACCURACY. IN REPORTINO OR REOUESTINO INFORMATION, PLEASE REFER TO ACCOUNT NUMBER ANO TRANSACTION REFERENCE NUMBER. GRANT AND CONTRACT BUDGET AND EXPENDITURE REPORT REPORTING PERIOD PRINCIPAL INVEST ANO ADDRESS AWARD AMOUNT DATE DOCUMENT NUMBER ACCT NAME BUOOET ENTITY REPORTINO ORO. ATIRIBUTES AWARD-DIRECT EXPENSE AWARD F&A PURCHASE ORDER NUMBER TRANSACTION TYPE ANO REFERENCE NUMBER DEPARTMENT COLLEOE MAJOR DIV FUNCTION F&A" I DIRECT C/S " DESCRIPTION I BUDGET ACCOUNT NO STATIS RUN SEQUENCE PAOE NO RUN DATE F&A C/S % RELATED BALANCE ORANT NUMBER ORANT PERIOD UNPAID PURCHASE ORDERS OR ENCUMBERENCE OBJECT CODE EXPENDITURES FREE BALANCE APPENDIX F EFNEP Costs by Month 191 Table 18 . Direct and indirect costs ofEFNEP during May, June, and July. May June Expense State County State County State Personnel Salaries Professional 4971.20 14937.91 4929.SO 14598.98 4937.84 Clerical 630.09 S16.42 630.09 S76.42 630.08 Paraprofessional 4914S.87 72819.97 Benefits Longevity Retirement 577.41 S040.01 S72.66 6749.09 S73.1S Social Security 373.64 42SS.36 370.44 601S.37 371 . 18 Unemployment 48.03 S4.10 S.42 Compensation Workers 2.61 202.94 2.61 303.3 1 2.61 Compensation Group Insurance 23S.20 14483.79 23S.20 I 14440.62 23S.20 401K Matching 3.00 332.S4 3.00 330.46 3.00 I Travel Travel - In State 937.04 S182.S4 279.42 12SS2.99 Travel - Out State 291 .SS I Printing, Duf!licating, and Binding Printing 2039.07 583.05 I Publications and Reports I Communications Postage 263.88 73.31 Freight Telephone 189.07 1.47 204.03 Telecommunication 7.00 Fl. Table of EFNEP Costs for May, June, and July 1 92 July I County 1471 1 .32 S76.42 48033.48 4998.96 4141 .41 418.0S I 200.88 14073.66 316.14 j 11 388.04 I 197.81 I I I I Table 1 8. (continued) May June July Expense State County State County State County I E!!!!i:ement .ij I Equipment I 1600.00 I 3500.00 I I I I I I Maintenance and Re:eairs I Maintenance and 366.98 1050.00 305.00 1050.00 305.00 1050.00 Repairs I Professional Senices and Fees I Subscriptions I I I 'I I I I I SUJ!J!lies I Operating 574.22 646.53 1533. 1 1 2054.07 16.66 Computer Software I Contractual and S:eecial Services I Group Food and Lodging I Commercial 4.24 Setvices Univ Services 1249.00 I i I I Seminar and 621.00 Conference Registration TOTAL DIRECT 12574.30 99591.01 1 1400. 1 132041 .56 7080.74 89106. 17 I COSTS I I TOTAL 2391.31 10079.95 2484.08 13851 . 16 1542.89 9347.24 INDIRECT COSTS I TOTAL COSTS 14965.65 109670.96 13884. 18 145892.72 8623.63 98453.41 Fl.2. Table of EFNEP Costs for May, June, and July 1 93 Table 19. Direct and indire ct co sts o f EFNEP during A ugust, Septe mbe r, and October. August September October Expense Total State County State County State County Personnel Salaries Professional 4397.84 l4S73.2S 4937.84 16041.S8 4937.84 17790.66 1217.66 Clerical 630.08 S16.42 630.08 S63.82 127.9S S16.42 6724.29 Paraprofessional 49SS6.S3 47886.61 47886.61 48372.96 31S81 S.42 Benefits Longevity 400.00 37S.OO 77S.OO Retirement 603.19 SOS9.S3 S73.1S S076.38 S20.0S S308.99 3S6S3.80 Social Security 401.78 4262.43 371. 18 42S7.96 332.77 4293.23 29446.80 Unemployment 8.86 434.46 S.42 416.64 1 . 10 420.90 1813.S8 Compensation Workers 4.27 208.73 2.61 200.24 O.S3 202.2S 1333.S9 Compensation Group Insurance 23S.20 14S04.S l 23S.20 14204.24 23S.20 14722.46 81840.SO 40 IK Matching 3.00 326.16 3.00 3S8.02 3.00 401 .36 ! 2082.68 I Travel ! I Travel - In State 247.20 l 8S8.S8 S46.9S 3913.90 470.33 7S09.32 33886.31 Travel - Out State 291.SS I Printing, Du2licating, and Binding I Printing 662.79 238.64 3S08.34 7229.70 Publications and 309.73 39.00 348.73 Reports I Communications I I Postage 374.82 712.01 Freight 179.92 179.92 Telephone 474.96 474.96 l .4S 870.98 Telecommunication 1 1 .SO 19.SO 8.SO 46.SO F2.1. Table of EFNEP Costs for August, September, and October 19 4 Table 1 9. (continued) August September October Total Expense State County State County State County I E9ui2ment I I Equipment I I I I I I I 5 100.00 I I Maintenance and Re2airs I Maintenance and 305.00 1050.00 305.00 1050.00 305.00 1050.00 8191 .98 Repairs I Professional Semces and Fees I I I Subscriptions I I I 9.97 I I I I 9.97 1 I SUJ!J!lies I I Operating 1 520.23 426.74 355.50 576.02 350. 13 217.97 8271 . 18 Computer Software 75.00 75.00 I Contractual and S2ecial Services I I Group Food and 1046.25 3767.00 4813.25 Lodging Commercial 4.24 Services Univ Services 1249.00 Seminar and 621.00 Conference Registration I TOTAL DIRECT 9920.59 93212.34 93 19.89 95020.37 63379.13 100867.97 554605 COSTS TOTAL 2161.70 9777.97 2030.80 9967.64 13810.3 1 10581 .05 88026.1 INDIRECT COSTS TOTAL COSTS 12082.29 102990.31 1 1350.69 104988.01 77189.44 1 1 1449.02 642630.64 F2.2. Table of EFNEP Costs for August, September, and October 1 95 APPENDIX G Glossary of Terms 196 Glossary of Terms benefit-cost ratio - a mathematical comparison of the benefits of an intervention divided by the costs. When the benefit-cost ratio is greater than 1 , benefits exceed costs. contingent valuation studies - the use of surveys to estimate the willingness-to-pay (WTP) values of health outcomes. In the surveys hypothetical market situations are used to elicit consumer valuation of goods and services. cost analysis - the process of estimating the costs of an intervention. cost-benefit analysis (CBA) - a type of economic analysis that compares alternative interventions by assigning dollar values to costs and to outcomes. Results are expressed in a benefit-cost ratio or net present value. cost-effectiveness analysis (CEA) - a type of economic analysis that compares alternative interventions with similar goals by relating costs, expressed in dollars, to outcomes achieved expressed in natural units, such as pounds of weight lost or decreases in blood cholesterol. cost-feasibility analysis (CFA) - a type of economic analysis that estimates only the cost of an alternative in order to determine whether or not it can be considered. cost-of-illness (COi) - a method of estimating direct medical and nonmedical costs associated with an illness and indirect costs associated with lost productivity due to morbidity or premature mortality. cost-utility analysis (CUA) - a type of economic analysis that compares alternative interventions by comparing their costs and the estimated utility or value of their outcomes. A common measure of CUA is the quality-adjusted life year (QUAL Y) where the quality of life is related to the length of life. decision analysis - a systematic approach to decision making under conditions of uncertainty using explicit, quantitative methods. direct costs - the measure of resources spent to implement an intervention or program. discounting - a way to adjust the value of future costs and benefits to an equivalent present value. GI.I. Glossary of Terms 1 97 discount rate - the rate at which future costs and benefits are discounted to account for changes. indirect costs - the measure of resources forgone to either participate in an intervention or as the result of a health condition. intangible costs - costs for which assigning monetary value are difficult, such as the costs of pain and suffering. net present value (NPV) - the summation of the value of discounted costs from the discounted value of benefits of an intervention. sensitivity analysis - mathematical calculations that isolate factors that may influence the outcome of an economic analysis. time horizon - the defined period over which costs and benefits of an intervention are tracked. Time horizon is sometimes called the analytic horizon. willingness-to-pay (WTP) - the method of estimating the maximum dollar amount an individual would pay in a given risk-reducing situation to reduce the risk of death and illness. Gl.2. Glossary of Terms 1 98 VITA Janie L. Burney was born in Beckley, West Virginia on August 22, 1 956. She attended public schools in Titusville, Florida (Brevard County), where she graduated from Astronaut High School in 1974. She majored in Nutrition Science at The University of Tennessee, Knoxville and graduated with a Bachelor of Science in Human Ecology in 1 978. Following graduation she was employed by Jennie Stewart Memorial Hospital in Hopkinsville, Kentucky as a nutritionist. In 1 98 1 , she entered graduate school at The University of Tennessee, Knoxville and received a Master of Science degree in Nutrition Science with a minor in Food Science. Her major professor was Dr. Jean Skinner and the title of her Master' s thesis was, "The Breakfast and Snacking Habits of Eat Tennessee Adolescents." She has been a Registered Dietitian since 1 983 . In 1983 , she was employed as a Nutritionist by the Tennessee Department of Health in the East Tennessee Region. She currently is employed by The University of Tennessee Agricultural Extension Service as a nutrition specialist. She is Coordinator of the Expanded Food and Nutrition Education Program (EFNEP). Her major job responsibilities include management and supervision ofEFNEP. While employed with the Extension Service, she entered the doctoral program with a major in Nutrition at The University of Tennessee, Knoxville. The doctoral degree was received in December, 1 998. 199

SP738 EFNEP Participant Brochure

Abstract

Extracted data

SP738 EFNEP Participant Brochure

Abstract

Extracted data

Related items

Related items