University of Memphis University of Memphis Digital Commons Electronic Theses and Dissertations 2021 IDENTIFYING POTENTIAL FUNDING INEQUITIES FOR ARKANSAS COMMUNITY COLLEGES Christopher Allen Heigle Follow this and additional works at: https://digitalcommons.memphis.edu/etd Recommended Citation Heigle, Christopher Allen, "IDENTIFYING POTENTIAL FUNDING INEQUITIES FOR ARKANSAS COMMUNITY COLLEGES" (2021). Electronic Theses and Dissertations. 2583. https://digitalcommons.memphis.edu/etd/2583 This Dissertation is brought to you for free and open access by University of Memphis Digital Commons. It has been accepted for inclusion in Electronic Theses and Dissertations by an authorized administrator of University of Memphis Digital Commons. For more information, please contact khggerty@memphis.edu. IDENTIFYING POTENTIAL FUNDING INEQUITIES FOR ARKANSAS COMMUNITY COLLEGES by Christopher Allen Heigle A Disseration Submitted in Partial Fullfillment of the Requirements for the Degree of Doctor of Education Major: Higher and Adult Education The University of Memphis May 2021 ii Copyright© Christopher Allen Heigle All rights reserved iii Dedication To my wife, who somehow persists in finding the courage and patience to love me, and to my daughter, who reminds me daily of what remains worthy of hope; both of whom I owe my heart and soul. iv Abstract A 2009 federal call to action seeking to improve postsecondary outcomes was largely underscored by a financial recession that continues to have lasting effects on state funding for public postsecondary education. In an effort to improve efficiency while also growing the number of postsecondary completers, states began adopting performance management policies to help link institutional performance with future funding outcomes. The purpose of this study was to determine if funding outcomes at Arkansas community colleges were related to specific student characteristics of the institution. The researcher hypothesized that there was a significant relationship between an institution’s funding outcome and three student characteristics of interest (low-income, non-traditional age, minority) after controlling for credential growth. The study used four research questions to explore the relationship between funding outcomes, credential growth, and the three student characteristics of interest. The study used a hierarchical linear model building procedure to help reduce unexplained variance in both the within-and-between subjects’ levels. After controlling for credential growth, the study found that of the three student characteristics variables, only the low-income variable acted as a statistically significant predictor for funding outcomes. Although this study is underpowered, these results indicate the need for further research regarding the relationship between low-income students and institutional funding outcomes in states utilizing outcomes-based performance funding. Key words: Performance Funding, Outcomes-Based Performance Funding, Distributional Inequity, Equity Metrics, Hierarchical Linear Model v Table of Contents Chapter Page List of Tables vii List of Figures viii 1. Introduction 1 Background of the Study 3 Statement of the Problem 6 Purpose of the Study 9 Research Questions 11 Hypothesis 10 Significance of the Study 13 Theoretical Framework 14 Assumptions 17 Limitations 17 Definitions of Terms 19 Research Design 23 Study Overview 25 2. Literature Review 26 Review of Postsecondary Student Success Theories 26 Review of Non-Market Failure Theory 29 Review of Public Higher Education Funding 31 Review of Performance Funding in Higher Education 37 Understanding the Effects of Outcomes-Based Performance Funding 40 Chapter Summary 43 3. Methodology 46 Epistemology 46 Research Design 48 Research Questions 50 Population 51 Sample 51 Data Collection 55 Variables 56 Hypothesis 58 Data Analysis 59 Hierarchical Linear Model 63 Assessment of Model Fit 63 Chapter Summary 65 4. Results 66 Preliminary Analysis and Descriptive Statistics 67 vi Table of Contents (continued) Chapter Page 4. (cont.) Mixed Linear Modeling Results 74 Null Model 76 Unconditional Growth Model 77 Conditional Model 1 78 Conditional Model 2a 79 Conditional Model 2b 81 Conditional Model 2c 82 Conditional Model 3a 84 Conditional Model 3b 86 Conditional Model 3c 87 Conditional Model 4 89 Chapter Summary 91 5. Conclusions 93 Summarizing Results by Research Question 95 Summarizing Results by Hypotheses 97 Discussion and Implications 99 Recommendations for Practice 103 Recommendations for Further Research 104 Final Thoughts 105 References 109 vii List of Table Table Page 1 Methodology Used to Produce the Study’s Variables 11 2 Average Institutional Characteristics of Sample in Percentages 54 3 Conceptual Model for Study’s Longitudinal Data 57 4 Descriptive Statistics for the Study’s Continuous Variables 67 5 Model Estimates Table 75 6 Assessing Model Goodness-of-Fit 76 viii List of Figures Figure Page 1 Frequency polygon of the intensity for annual productivity scores (PFINX) 69 2 Institutional Linear Slopes for Performance Funding Scores (PFINX) 70 3 Visual Inspection of the time-varying covariant (CREDS) by the index variable (TIME) 71 4 Frequency polygon of the low-income student variable (AVG_PELL) 72 5 Frequency polygon of the non-traditional age student variable (AVG_AGE) 73 6 Frequency polygon of the minority student variable (AVG_MIN) 74 1 Chapter 1 Introduction Postsecondary education can significantly impact state and national economies (Berger & Fisher, 2013; Snyder, 2019; Skolnik et al., 2005). Individuals with postsecondary education typically experience higher lifetime earnings, have more economic resilience during recessions, and are less likely to require disability payments from social security (Schanzenbach et al., 2017). In 1988, the United States had the world’s highest percentage of adults aged 25-34 having achieved some form of postsecondary education; by 2019 the United States had fallen to 11th place (Organization for Economic Cooperation and Development [OECD], 2020). Former President Barack Obama first raised the alarm to the growing higher education achievement crisis in 2009 by launching the American Graduation Initiative, which set a goal for the United States to have the highest percentage of college graduates for any developed country by the year 2020 (American College Personnel Association [ACPA], 2020). Failing to achieve that goal, the United States now trails ten spots behind and 19.4 percentage points below South Korea, the world’s leader in postsecondary attainment (OECD, 2020). Although the United States has made some progress toward improving postsecondary outcomes, not all student groups have experienced equal improvement (Digest of Education Statistics, 2019). The National Center for Education Statistics’ [NCES] Digest of Education Statistics (2019) reports that from 1996 to 2012, graduation rates for White students grew from 36.3% to 48.3%, while it only grew from 19.5% to 23.8% for Black students. An additional gap in achievement can be identified between students with different socioeconomic backgrounds (Kahlenberg et al., 2018). Although approximately 72% or students with a higher socioeconomic status (SES) complete a postsecondary credential, approximately only 35% of lower SES 2 students ever complete a degree (Postsecondary Attainment, 2015). Students from different racial and economic backgrounds often experience different pathways through postsecondary education (Kahlenberg et al., 2018) White students and students from higher income backgrounds tend to begin postsecondary at a traditional university, while Black students and students with lower income backgrounds typically begin their postsecondary education at a local public community college (Bragg & Durham, 2012; Ma & Baum, 2016). The problem of lagging higher education attainment rates coupled with a persistent achievement gap remains a policy concern for most states (Li, 2018). The economic recession of 2007 caused significant reductions in state-level tax revenues and led many state legislatures to emphasize the importance of improving student outcomes with no-new, and in some cases less, funds (Umbricht et al., 2017). To achieve these policy goals, legislators have begun turning to a form of performance management known as performance funding (Hillman et al., 2015; Umbricht et al., 2017). Performance funding for higher education has transitioned several times since it was first adopted by Tennessee in 1979 (McKeown-Moak, 2013). Over the past fifteen years, a more aggressive form of performance funding, known as outcomes-based performance funding (OBPF) has grown in popularity (Hearn, 2015). OBPF differs from more traditional performance funding models by allowing the potential loss of funding if state-defined goals are not met, or likewise gaining funds if the institutions outperform previous years (Rutherford & Rabovsky, 2014; Umbricht et al., 2017). Despite the growing popularity of OBPF models, there remains a lack of empirical evidence that shows these policies actually produce more credentials for students within states that ultimately ask institutions to do more with less (Hillman et al., 2015; Li & Kennedy, 2018; Tandberg et al., 2014). The lack of evidence for the policy’s effectiveness and the potential for institutions to lose funding year-over-year could pose 3 significant concerns for rural community colleges that already serve higher populations of traditionally underserved students (Kelchen, 2019). Background of the Study In its original form, performance funding sought to link a percentage of new money or bonus funds to an institution’s previous year’s enrollment (Hillman et al., 2014). This original form of performance funding, often referred to as performance funding 1.0, was meant to help support institutions showing growth beyond existing funding levels (Daugherty et al., 2016). Over the past fifteen years, state legislatures have begun adopting a particularly aggressive form of performance funding known as outcomes-based performance funding, or performance funding 2.0 (Daugherty et al., 2016). Several economic and political factors have influenced the way contemporary policymakers have sought to improve postsecondary outcomes. Despite a growing need to improve student completion rates and close the achievement gap, states have largely divested fiscal support for higher education due to several underlying political and economic factors beginning in 1999. Chief among these factors was the repeal of the Glass-Steagall Act of 1932 and its replacement by the Gramm-Leach-Bliley Act of 1999, which was signed into law by then President Bill Clinton (Mahon, 1999). The repeal of Glass-Steagall significantly reduced consumer-protections by allowing commercial banks to purchase privately held mortgages and trade these bundled-securities as investments, leading many banks to incentivize large home-purchases for individuals that might not otherwise qualify (Merle, 2018). When the markets eventually collapsed due to the decline of the “housing bubble” over $9 trillion in personal wealth was lost by these drastic reductions in home values (Merle, 2018). Nine years after the repeal of Glass-Steagall, the average American was significantly less wealthy which led to 4 reductions in consumer confidence and caused a steep decline in the collection of state and federal tax revenue. The economic recession caused by the repeal of Glass-Steagall and the subsequent reduction in tax revenues meant that legislatures would have to begin reducing expenditures, and overwhelmingly these cuts affected state supported institutions of higher education (Katsinas, et al., 2014). State legislatures already concerned with high year-to-year appropriations for public higher education began looking at more aggressive cost-cutting measures for public postsecondary institutions within their state (McKeown-Moak, 2013). Public services such as healthcare, state pensions, K-12 education, prisons, and transportation were prioritized over higher education spending (Katsinas et al., 2014). Overall, state higher education spending dropped approximately 12 percent from 2003 to 2012 (State Funding Trends, December 2014). States now spend approximately 18 percent less per student than they did in 2008 (Mitchell et al., 2019). The partisanship of state legislatures largely determined the degree to which state funding for higher education was affected following significant reductions in tax revenues (Tandberg, 2010). States with Democratic led legislatures were associated with increased state appropriative spending and Republican legislatures were associated with a reduction in state funding for public postsecondary institutions (McLendon et al.,2009; Tandberg, 2010). Following the largescale election wins by republican state lawmakers in 2010, newly elected lawmakers already experiencing revenue declines due to an economic recession, were challenged to grow student credentials while also conserving necessary state funds. This dynamic caused many states to begin adopting more aggressive forms of performance management known as outcomes-based performance funding (Dougherty & Natow, 2015; Li, 2017; Rabovsky, 2014). 5 Although this new form of performance management does produce new funding for some institutions, the potential for funding losses due to lower-performing students could have devastating effects for rural serving community colleges who often-times serve larger populations of student groups experiencing lower overall academic success (Bragg & Durham, 2012; Umbricht et al., 2017). As states adopt aggressive performance funding policies with little acknowledgement of the underlying predictors to student retention and completion, larger institutions will begin to shift their admissions criteria to limit access to traditionally underrepresented students (Umbricht et al., 2017); the situation could be even more dangerous for rural serving community colleges that typically have open-admissions policies (Kelchen, 2019). A report commissioned by the National Postsecondary Education Cooperative focusing on student success emphasized the role that a student’s socioeconomic status (SES) has on postsecondary outcomes, finding that the quality of high school academic preparation and a family’s education background are strongly associated with overall postsecondary success (Kuh et al., 2006). For community colleges, which often serve large populations of academically underprepared students (Bragg & Durham, 2012), the consequences could have increasingly dire effects when student success initiatives are directly impacted by state appropriative support (Kelchen, 2019). The current literature on OBPF suggests that current models of the policy should consider student characteristics to essentially lessen the fiscal liability of allowing open enrollment at institutions that largely serve traditionally marginalized student populations (Kelchen, 2019; Li et al., 2018; Umbricht et al. 2017). The lack of empirical evidence of OBPF effectiveness as a policy tool and the growing popularity of these policies could have the unintended effect of further exacerbating the existing 6 student achievement gap for underrepresented student populations by reducing state appropriations for institutions that serve higher levels of these traditionally underserved and academically underprepared students (Bragg & Durham, 2012; Kelchen, 2019;). Left unchecked, the economic outcomes for states that implement these policies could have lasting and dire fiscal consequences. Ranking 47th in overall state poverty, Arkansas is a prime example of a performance funding state whose outcomes should be better understood (Income and Poverty, 2020). The state’s 22 community colleges primarily serve as economic development partners by providing many rural communities with technical training centers needed to promote regional job growth (Hillman & Orians, 2013). Despite the importance of Arkansas’s community colleges to the state’s economy, Arkansas’s performance funding policy continues to cause more community colleges to lose rather than gain funding (Productivity Outcomes, 2020); and this study sought to determine if those funding losses were related to the enrollment characteristics of the institution. Statement of the Problem Although the number of students pursuing postsecondary education in the United States has grown, student completion remains relatively stagnant (Hillman et al., 2014). On average, only 60 percent of students graduate from a public college in the United States (Graduation Rates, 2019; Rabovsky, 2014; The Condition of Education, 2020). Despite significant attention given to improving overall completion rates and addressing the known attainment gap between higher and lower SES students, as well as between White and Black students, there remains little overall growth in both four-year and six-year cohort graduation rates (Graduation Rates, 2019). Looking at public postsecondary institutions’ six-year graduation rates at public four-year institutions, White students improved an overall 5.6%, from 58.1% in 1996 to 63.7% in 2010; 7 while Black students improved only 0.5 or half of a percent, from 38.9% in 1996 to 39.3% in 2010 (Graduation Rates, 2019). It is clear that what few gains the Unites States has made toward President Obama’s completion agenda have almost entirely benefitted White and higher-SES students (Graduation Rates, 2019). Despite a lack of evidence that OBPF has any effect on student credential attainment, since 2009 over 20 states have adopted some form of the funding policy (Hearn, 2015; Li et al., 2018). Institutions that enroll less academically prepared students will likely struggle to make year-to-year gains in performance metrics, which will reduce subsequent years’ appropriative base funding. This trend which is supported by prior literature on student success (Kuh et al., 2006) suggests that the policy could have increasingly detrimental effects on postsecondary access at public community colleges by allowing for annually compounded losses in state appropriative funding. The policy ultimately forces institutions to continuously improve while having fewer resources to address success initiatives. While OBPF holds institutions accountable for meeting the higher education needs of the state (Fowles, 2014; Hearn, 2015); in practice outcomes-based funding has yet to show substantial student-level outcomes in years following state implementation (Dougherty et al., 2016; Hearn, 2015; Hillman et al., 2015; Li & Kennedy, 2018; McKeown-Moak, 2013; Tandberg et al., 2014), and data could indicate that the policies are failing to adequately account for known predictors to student completion such as low-SES, non-traditional age, and minority student enrollments (Kuh, et al., 2006). Contemporary OBPF models do little to meaningfully correct for known predictors to student completion, potentially creating a funding environment where small, rural, minority-serving community colleges are increasingly required to annually produce more credentials with fewer state funds. Existing forms of OBPF potentially harm rural and minority serving 8 institutions because the policies fundamentally value student completion without adequately understanding the differences in the educational needs of a state’s diverse populations. Research into the achievement gap between different student groups suggest a systemic difference in the K-12 student experience, which is often moderated by familial support, school quality, teacher quality, and economic conditions, suggesting that students immediately and persistently experience grouping practices that “serve to demoralize low-performing students and maintain, perpetuate, and exacerbate differences in achievement” (Yeh, 2020, p. 3). Previous research into OBPF has indicated a need for performance models to include specific equity metrics that target low-income, non-traditional age, and minority student degree completion, gateway course success, developmental course success, and other indicators that could control for academic preparedness by rewarding institutions that help targeted students reach certain milestones (Dougherty & Natow, 2015; Li et al., 2018; Ubmricht et al., 2015). Arkansas’s productivity model is an outcomes-based formula that could be disproportionally affecting underserved student populations by not adequately accounting for the difficulty associated with educating students with poor prior education experience (Kuh et al., 2006). If left unmeasured, legislators may be emboldened to continue to support a policy that inequitably divests from community colleges serving large numbers of marginalized student. Arkansas currently ranks 47th in overall poverty and 40th in college preparedness, but despite these significantly low economic indicators, the state’s productivity model has led 14 of the state’s 22 community colleges to lose funding in two of the model’s three years (Income & Poverty, 2020; Productivity Outcomes, 2020; Rankings, 2020). Arkansas’s OBPF model adheres to the recommendations of previous researchers (Cielinski & Pham, 2017; Dougherty & Natow, 2015; Li et al., 2018; Ubmricht et al., 2015) by including equity metrics that award additional 9 points for targeted student characteristics. Targeted student groups include Pell-receiving, non-traditional ages, and minority students (Productivity Funding, 2020). Arkansas community colleges submit an annual report of all credentials awarded for a given year and are provided additional (1.29) multipliers for Pell receiving, non-traditional age, and minority students earning a credential. The state then provides the college with their productivity index score which either indicates a gain or loss over the previous year’s data (Productivity Funding, 2020). Despite recommendation to include equity metrics, there remains a gap in the literature regarding those metrics’ ability to protect community colleges from experiencing distributional inequity, which could occur due to discreet biases based on enrollment characteristics (Cielinski & Pham, 2017; Kelchen, 2019; Umbricht et al., 2017; Weber, 2020; Wolf, 1978). Arkansas community colleges have a higher percentage of enrollment of academically underprepared students than most other states (Rankings, 2020), and the high cost of developmental education (Ganga et al., 2018), and the potential for funding inequity could significantly impact the state’s ability to improve postsecondary outcomes for targeted student populations. Although the Arkansas productivity model includes equity metrics, prior to this study there existed an empirical gap in the literature regarding the relationship between funding, student outcomes, and student characteristics within Arkansas. Purpose of the Study The purpose of this study was to determine if an institution’s student enrollment characteristics significantly impact productivity index scores. This study proposed to develop a mixed effects multilevel regression model that best-explains the relationship between an Arkansas community college’s productivity index score and the annual difference in credentials awarded, and if that relationship is moderated by the enrollment characteristics of an institution. 10 If the proposed regression model’s explanation of the relationship between productivity index scores and the annual difference in credentials earned is not improved by the inclusion of specific student characteristics of interest, it is likely that the funding model’s included equity metrics are adequately controlling for known barriers to student success attributed to low-income, non-traditional age, and minority students. If, however, the regression model’s explanation is improved by the inclusion of specific student characteristics, and that the relationship is moderated by the student characteristics of interest, it is likely that institutions are experiencing distributional inequity as a consequence of both political and systemic biases (Umbricht et al., 2017; Weber, 2020; Wolf, 1978). This study included all 22 public community colleges in the state of Arkansas and used data from the current performance funding model implemented by Act 148 of 2017 (Productivity Funding, 2020). The model proposed by Act 148 of 2017 has produced four years of productivity index scores as of 2020. Model-year one was designated as a “hold harmless” year, which blocked any institution from receiving a negative productivity index score. Model-years two through four were permitted to reflect negative productivity index scores, and therefore were included in the study. Model year two reflects funding outcomes for first year 2020, model year three reflects funding outcomes for fiscal year 2021, and model year four reflects funding outcomes for fiscal year 2022. This data is available online and represents a total of 22 institutions spanning three years, with a total of 66 repeated-measures observations of the outcome variable. A graphical representation of the years included within the study, as well as the methodology used to produce the variables for this study are found in Table 1. 11 Table 1 Methodology used to produce the study’s variables. Time Productivity Index Scores Credentials Student Characteristics (Pell, Age, Minority) Time-Varying Covariate Outcome Variable Time-Varying Covariate Time-Invariant Covariates Model Year 4 Fiscal Year 22 Difference in average of (19,18,17) and (18,17,16) Average for Years 2015-2018 or 2011-2018 Model Year 3 Fiscal Year 21 Difference in average of (18,17,16) and (17,16,15) Average for Years 2015-2018 or 2011-2018 Model Year 2 Fiscal Year 20 Difference in average of (17,16,15) and (16,15,14) Average for Years 2015-2018 or 2011-2018 Research Questions The following research questions guided this study. 1. What is the relationship between an institution’s productivity index score and the total credentials earned? 2. Is there a relationship between the percent of low-income students and productivity index scores after controlling for credentials, and if so, does the percent of low-income students moderate the relationship between an institution’s productivity index score and the total credentials earned? 3. Is there a relationship between the percent of non-traditional age students and productivity index scores after controlling for credentials, and if so, does the percent of non-traditional age students moderate the relationship between an institution’s productivity index score and the total credentials earned? 4. Is there a relationship between the percent of minority students and productivity index scores after controlling for credentials, and if so, does the percent of minority students 12 moderate the relationship between an institution’s productivity index score and the total credentials earned? Hypothesis The study’s hypothesis is based on the researcher’s assumption that there are distributional inequities within the policy’s performance metrics and that the inclusion of bonus points for underrepresented students does not preclude the model from discreet biases (Bragg & Durham, 2012; Kelchen, 2019; Umbricht et al., 2017). The study’s included hypotheses are based on literature regarding postsecondary student success and the additional supportive learning needs of students with deficient prior learning experiences (Chen, 2016; Kuh et al., 2006). The hypotheses are also informed by literature related to political biases within policy platforms (Weber, 2020), and systemic biases found within efficiency systems; both of which often lead to distributional inequities within marginalized populations (Cielinski & Pham, 2017; Umbricht et al., 2017; Wolf, 1978;). The following hypotheses were used to develop a multilevel regression model that better explains the relationship between funding outcomes and student success (Woltman et al., 2012). 1. The difference in annual credentials is related to an institution’s productivity index score. 2. The percent of Pell receiving students at an institution is related to an institution’s productivity index score, after controlling for the difference in annual credentials. 3. The percent of non-traditional age students at an institution is related to an institution’s productivity index score, after controlling for the difference in annual credentials. 13 4. The percent of minority students at an institution is related to an institution’s productivity index score, after controlling for the difference in annual credentials. 5. The percent of Pell receiving students moderates the relationship between credentials and productivity index scores. 6. The percent of non-traditional students receiving students moderates the relationship between credentials and productivity index scores. 7. The percent of minority students moderates the relationship between credentials and productivity index scores. Significance of the Study Identifying the effect that student enrollment characteristics have toward an institution’s productivity index score provides higher education administrators, state legislators, and members of the state’s executive branch empirical evidence of the policy’s equitability. Arkansas serves a diverse population, and the state’s community college often closely reflect the make-up of the communities they represent. Arkansas is largely a rural state and many communities suffer from high unemployment and systemic poverty (Income & Poverty, 2020). Many of the state’s community colleges serve communities with average median-household incomes as low as $27,000, while other community colleges serve communities with over $50,000 in average median-household income (Income & Poverty, 2020). These significant income differences between communities mean that students throughout the state enter post-secondary education with significantly different prior learning experiences (Chen, 2016; Kuh et al., 2006). The study sought to determine if Arkansas’ Act 148 of 2017 produced a funding model for higher education that equitably distributed funds among the state’s public community colleges. Arkansas’ performance funding model has caused 14 out of 22 community colleges to 14 lose funding in two of the model’s three years (Productivity Outcomes, 2020). If the state’s OBPF policy continues to produce funding outcomes similar to those included within this study, community colleges serving higher percentages of low-income students will likely continue to lose small annual percentages of funding. If left unaddressed, this slight but compounding loss could eventually create large reductions in funding for institutions serving higher populations of the very student’s that could benefit the most from public two-year postsecondary education (Bragg & Durham, 2012; Cielinksi & Pham, 2017; Kuh et al., 2006; Umbricht et al., 2017; Wolf, 1978). The state of Arkansas is focused on creating more degrees and this goal drives much of the purpose for Arkansas’s productivity funding model (Productivity Funding, 2020). Although Arkansas’s productivity model includes equity metrics, it remains unclear if those metrics are adequately controlling for the known achievement gap among some postsecondary students with lower socioeconomic backgrounds (Kuh et al., 2006). Results from this study suggest that an institution’s percentage of low-income students may be related to productivity index scores, which should alert the state’s legislators, higher education administrators, and executive branch leadership of a potential bias within the policy. Theoretical Framework Arkansas’s productivity funding model is a form of performance management which focuses on building accountability, improving efficiency, and reducing spending (Act 148, 2017; Umbricht et al., 2017). Performance funding formulas link state-level goals to institutional-level outcomes through state funding (Umbricht et al., 2017). Institutions must show growth each year in order to retain their current funding distribution, and institutions that lose funding must outperform previous years with less fiscal support. This cyclical pattern may create potential 15 unintended outcomes for schools with large numbers of traditionally underrepresented students. This study is informed by three primary theories which explain the observed and hypothesized effects of Arkansas’s productivity model. Building on previous research into the effectiveness of OBPF, legislatures adopt OBPF to improve student credential outcomes (Hillman et. al., 2015; Hillman, et al., 2017; Tandberg et. al., 2014). Legislatures implementing forms of OBPF rely on a system of incentives, where institutions that achieve greater student completion numbers are rewarded with additional funding. This reward structure is strongly supported by principle-agent theory (Mitnick, 2019). Principle-agent theory, also known as agency theory, was concurrently developed by researchers Ross and Mitnick in the early 1970s (Mitnick, 2019). Both Ross and Mitnick were concerned with understanding the relationship between compensation structures and the behaviors which produce an intended outcome (Mitnick, 2019). In agency theory, “principles” act as the consumer and use compensation or incentive to achieve an outcome (Mitnick, 2019). An “agent” acts as the service-provider and achieves the goals of the “principle” if the compensation or incentive adequately entices the agent’s behavior (Mitnick, 2019). Principle-agent theory places the state legislature as the “principle” and the various state-supported institutions of higher education as “agents” (Hillman et al., 2017). State legislatures adopt the overall goals for postsecondary education and enlist public community colleges to act as “agents” in order to achieve these specific policy goals (Hillman et al., 2017; Mitnick, 2019). State goals for public higher education have increasingly focused on growing the number of postsecondary credential holders (Hillman et al., 2017). The degree to which the institution responds to the potential for increased/decreased funding is supported by resource-dependence theory (RDT) (Fowles, 2014). Resource 16 dependency theory was first introduced in Pfeffer and Salancik’s 1978 publication of The External Control of Organizations: A Research Dependence Perspective. Pfeffer and Salancik were concerned with understanding the importance of social context toward organizational decision making (Pfeffer & Salancik, 2003). RDT suggests that external pressures drive organizational actions more than internal pressures, such as leadership values and beliefs (Pfeffer & Salancik, 2003). RDT views organizations as being embedded in larger networks of interdependencies, such as political or social relationships (Pfeffer & Salancik, 2003). State funding remains the largest source of funding for public community colleges (Datapoints, 2020). In this context, funding pressures drive institutional behavior toward state goals. This assumes that institutions are not fully utilizing resources, and that the potential loss of subsequent funding is all that is necessary to make significant gains to student success (Kelchen, 2019). The third theory used within this study is known as distributional inequity, which originated from political science literature and is strongly associated with policy analysis, public service delivery, and performance management (Umbricht et al., 2017; Wolf, 1978). Wolf’s (1978) framework proposed that similar to private enterprise, government intervention has the potential to create distributional inequities, and that these inequities in public goods are often a result of biases within efficiency systems. Wolf suggests that non-market (i.e., public policy) failures also create disproportional inequities based on divisions of power, income, or other externalities leading to undesired policy outcomes (Fording et al., 2007; Umbricht et al., 2017; Wolf, 1978). States using an outcomes-based form of performance management as a policy instrument, such as Arkansas’s productivity funding model, could experience disproportionate funding losses at institutions with higher percentages of low-income students, resulting in what Wolf described as distributional inequity (Fording et al., 2007; Umbricht et al., 2017; Wolf, 17 1978). In the case of Arkansas’s performance model, distributional inequity could mean restricting access to quality public higher education by withholding critical state funds to institutions that serve higher percentages of academically underprepared students (Cielinski & Pham, 2017; Kelchen, 2019; Kuh et al., 2006; Umbricht et al., 2017). Assumptions This study is guided by four primary assumptions. 1. Arkansas’s performance metrics were designed to equitably distribute funding to community colleges with the intention of aiding colleges with larger enrollment percentages of traditionally underserved students (Productivity Funding, 2020). 2. Students from income or racially underserved populations experience more barriers due to social, psychological, organizational, cultural, and economic structures and must exhibit more agency than more privileged populations to complete the same program of study (Kuh, et al., 2006). 3. Secondary data is accurately reported by institutions using annual surveys to the National Center for Education Statistics’ Integrated Postsecondary Education Data System and to the Arkansas Division of Higher Education. 4. Guided by resource-dependence theory, it is assumed that institutions will take an aggressive stance in closing completion gaps that could lead to additional funding or at least mitigate any immediate funding losses (Pfeffer & Salancik, 2003). Limitations This study was limited by the absence of institutionally specific practices that might help to overlay contextual data on what factors might be related to student success for each institution’s specific enrollment characteristics. For instance, some institutions may be investing 18 more resources toward inclusive policies, or in the use of data to support decision making, which could have a mitigating effect on student outcomes even in communities with higher percentages of underserved students. The study is limited by the available data for the targeted years of Arkansas’s productivity model. The National Science Foundation’s data retrieval website provides access to student level demographic and outcome data from 1997-2018 (NSF Table Builder, 2020). The student characteristics data was collected by the Integrated Postsecondary Data System’s (IPEDS) Enrollment and Completions surveys, and although this data did provide the study with a more complete understanding of the policies effectiveness toward promoting graduation for targeted populations, the aggregated totals do not reflect student level achievement. In the absence of student level data, this study assumed that student achievement is uniform across student groups within the institutions. This study determined if specific student characteristics, in the form of aggregated totals, had a significant relationship with productivity index scores after controlling for the annual difference in awarded credentials, which is to say, how student characteristics affected funding outcomes. Another limitation to measuring the policy’s effectiveness is related to the aggregated nature of the data, and what Bankston (2011) warned as the mass production of credentials. Institutions facing budget cuts based on student outcomes are more incentivized to create higher number of short-term credentials that may not directly improve the employment or educational outcomes of students pursuing such programs of study (Li & Kennedy, 2018); which could be creating an inflated view of overall institutional effectiveness. The data within this study was aggregated into institutional totals and does not analyze student-level outcomes. Student-level data could indicate if institutions are impacting student completion totals for low-income, non- 19 traditional age, and minority students, or if potential increases in credentials awarded are attributed to student groups with traditionally higher completion rates. This study serves as a baseline to indicate potential inequities; however, once student-level data becomes available for academic year 2019, further analysis could indicate the true impact of the policy toward student completion rates for marginalized student populations. Definitions of Terms Appropriation: An appropriation refers to a single fiscal year appropriation bill that funds federal or state postsecondary institutions for a subsequent year only (GAO Glossary of Terms, 2020). For this study, appropriation refers to a single fiscal year funding bill for postsecondary institutions in the state of Arkansas. Community College: A community college is a public or private postsecondary institution that focuses on awarding associate degrees, one-year, and less-than-one-year credentials. (Cohen & Kisker, 2010). For the purpose of this study, community college refers to public two-year postsecondary institutions in the state of Arkansas. Credential: A credential refers to any completed program of study at the postsecondary level and includes certificates of less than one year of study, certificates of at least one but less than two years of study, and certificates of at least two but less than four years of study; the associate degree representing a sub-baccalaureate program of study (College Level Glossary, 2020). Distributional Inequity: Non-market theory holds that government intervention could lead to distributional inequities for marginalized populations (Wolf, 1978). Distributional inequity has been used to describe unintended consequences within performance management 20 policies that negatively affect poor and minority individuals disproportionately than other portions of society (Dollery and Wallis, 1997; Umbricht et al., 2017). Equity Metric: An equity metric is a measurement of specific goals to improve outcomes for marginalized student populations. Equity metrics can provide bonus points or may act as a point multiplier to an existing outcome measure (Cielinski & Pham, 2017). An example of an equity metric for a targeted student population are those used in the Arkansas productivity model (Productivity model, 2020). Arkansas’s model provides an additional 0.29 points for every 1.00 point awarded for individuals with a targeted student characteristic (Productivity Funding, 2020). Intermediate Performance Indicator: An intermediate performance indicator is a metric used in performance-management that focuses on formative measures of success rather than the singular outcome of the policy (Dougherty & Natow, 2015). Intermediate performance indicators include non-credential earning metrics such as student progression along a degree pathway, developmental course success, and the lack of non-contributing coursework (Dougherty & Natow, 2015; Productivity Funding, 2020). Marginalized, Underrepresented, or Underserved student populations: In this study marginalized, underrepresented, or underserved students refer to populations of college students that experience barriers to postsecondary success beyond those typically experienced by white, traditional age, or higher-income students, and is based on the broader literature regarding postsecondary student success (Bragg & Durham, 2012; Cielinksi & Pham, 2017; Kuh et al., 2006). This study is specifically including student populations targeted by the equity metrics within Arkansas’s productivity model, and therefore is a reference to low-income, non-traditional age, and minority students (Productivity Funding, 2020). 21 Minority Student: Kuh et al., (2006) in a report on postsecondary success found that minority students experience more known barriers to academic success than their white counterparts. In addition, often times minority students are over-represented within other lower performing student groups (Kuh et al., 2006). The achievement gap between white and minority students is made worse by the fact that of the gains made by the United States toward growing postsecondary education, almost all of those gains were experienced by white students (Estep, 2016; Graduation and Retention Rates, 2020; Kahlenberg et al., 2018). Cielinski and Pham (2017) suggest that equity metrics targeting specific student characteristics, including minority student success, are necessary to overcome these known barriers causing academic duress. Arkansas’s productivity model provides an additional 0.29 points for “Black or African American” students and for “Hispanic” students (Productivity Model, 2020). Non-traditional student: The term non-traditional student is a broad reference to several characteristics that differentiate a student from the traditional view of a college student that enrolls in postsecondary immediately following the completion of a high school diploma (Definitions and Data, 2020). The term is largely associated with the age of a college student, typically referring to a student 25 years and older (Definitions and Data, 2020). This study uses the definition provided by Arkansas’s productivity model which provides an additional 0.29 points for students over the age of 24 (Productivity Funding, 2020). Outcomes-based performance funding: Outcomes-based performance funding refers to state postsecondary funding policies that link a percentage of state appropriations for an institution to state defined metrics (Hearn, 2015). These policies grew in popularity following the Lumina Foundation’s focus on quality improvement initiatives in several states during the mid-2000’s (Dougherty et al., 2014; Hearn, 2015). Outcomes-based performance funding differs from 22 traditional forms of enrollment-based performance funding by allowing an institution to lose a portion of state funding if state-defined goals are not achieved (Hearn, 2015). Pell-receiving Student: Cielinski and Pham (2017) suggest Pell receiving students as a relevant student population to be targeted by equity metrics seeking to improve outcomes for low-income students. A Pell receiving student refers to an undergraduate student attending a postsecondary institution that is accredited by a U.S. Department of Education approved regional accrediting agency (Pell Grant, 2020; Accreditation in the United States, 2020). The Pell grant is authorized by the Higher Education Act of 1965, as amended, Title IV, Part A, Subpart 1; 20 U.S.C. 1070a (Federal Pell Grants, 2020). Pell awards are made to low-income students and the award amount is based on the financial need of the student as determined by the cost of attendance and the expected family contribution (How Aid is Calculated, 2020). Principle-Agent Theory: Principle-Agent theory refers to the relationship between an entity that desires certain outcomes but lacks the ability to directly affect those outcomes, and therefore enlists an agent to act on its behalf (Poth & Selck, 2009). Productivity Funding Model: Productivity funding model refers to the policy instrument used by the state of Arkansas to link performance-based outcomes to subsequent years funding. Arkansas’s productivity model produces this study’s dependent variable, productivity index scores (Productivity Funding, 2020). Within this study, the productivity funding model provides each institution with a productivity index score that indicates either a positive or negative integer, thus indicating a gain or loss in funding, respectively. Resource Dependency Theory: Resource-Dependence theory refers to the interconnected relationship and inherent power found between various organizations (Hillman et al., 2009). This inherent power difference is exploited when an organization is dependent on the 23 resources granted by another, creating a relationship where the resource granter exhibits power directly onto the dependent organization (Hillman et al., 2009). Student Success: Although student success can be many things, when framed by the United States contemporary focus on degree completion, student success is seen primarily as a student’s completion of a postsecondary credential (Borden et al., 2019). Research Design This study is foundationally postpositivistic in its approach to understanding the relationship between funding outcomes and the student characteristics of an institution’s enrollment. Researchers using the post-positivist worldview typically employ quantitative analysis and hypothesis testing to discreetly analyze outcomes that may influence the identified research problem (Creswell, 2009). Although this study’s methodology was postpositivistic, the author admits that the identification of the research problem and the development of the underlying hypotheses were strongly influenced through the critical perspective (Creswell, 2009). Research founded in critical theory seeks to develop knowledge that can be transformatively applied toward political, social, and economic systems of oppression (Creswell, 2009). This study’s research problem was identified when the researcher observed the majority of Arkansas’s community colleges lose funding in two of the productivity model’s three years (Productivity Outcomes, 2020). As previous literature has demonstrated, community colleges tend to serve higher percentages of traditionally underrepresented students (Kelchen, 2019). Previous literature has likewise indicated that these students are least likely to complete a postsecondary credential (Kuh et al., 2006). The potential for an annually repeated loss of funding to the state’s community colleges due to discreet biases against specific student 24 characteristics could detrimentally impact the quality of postsecondary education received by marginalized student populations. The author used an inferential research approach using longitudinal ex post facto data collected from Arkansas’s community colleges (Kwok et al., 2008). The study used a form of growth curve modeling known as a hierarchical linear model (HLM), also known as a multi-level mixed-effects linear model (Curran et al., 2010). Growth curve modeling often refers to a form of repeated measures analysis which estimates “inter-individual variability in intra-individual patterns of change over time” (Curran et al., 2010, p. 2). The study’s HLM analyzed ex post facto longitudinal data with the purpose of determining the relationship between an institution’s awarded credentials and the subsequent productivity index score, and to what extent the inclusion of additional student characteristic data effect that relationship (Curran et al., 2010; Woltman et al., 2012). The study used a form of non-probability sampling that purposively included all of Arkansas’s twenty-two public community colleges. The institutions included within this study’s sample are all governed by the same statewide executive division of the Governor’s cabinet, and therefore experience similar policy and funding conditions leading to the measurement of the institutions’ performance under Arkansas’s productivity funding model. Each institution reports their percentage of underserved student populations to the IPEDS annual enrollment survey, and each institution reports their total awarded credentials in the annual IPEDS completions survey as well as to the Arkansas Division of Higher Education. Aggregated totals for this data are available online and represent a total of twenty-two (N = 22) participant institutions spanning three years for a total of 66 repeated observations of the outcome variable. Utilizing all available data allowed the researcher to have a more complete understanding of how each institution’s 25 productivity index score is affected by specific student characteristics; and the method of sampling did not add undue burden to the researcher (Laerd, 2020). Study Overview Chapter 1 introduced the economic and political influences that caused many states to adopt more aggressive policies aimed at improving overall student completion. This chapter identified the problem that Arkansas’s higher education funding model may include systemic biases (Cielinski & Pham, 2017; Umbricht et al., 2017). The chapter provided several hypotheses based on several assumptions supported by the study’s theoretical framework and assumes college productivity outcomes are affected by an institution’s total percentage of marginalized students (Kuh et al., 2006; Kelchen, 2019). Chapter 1 addressed the study’s limitation and provided a list of definitions supporting the research purpose. Chapter 2 presented the idea of market failure, the need for public intervention, and the potential inequity that can occur toward marginalized populations. Chapter 2 presents the history of public funding for higher education and provides an in-depth review of the literature supporting the concepts used in this study. Chapter 3 identifies the methodology used to study potential distributional inequity within Arkansas’s funding model, and includes the study’s population, sampling methods, analytical model, variables, and validity standards. Chapter 4 presents the findings of the study. Chapter 5 summarizes the study’s findings, contextualizes the findings for the specific research problems in a discussion, and offers recommendations for future research into performance funding for higher education. 26 Chapter 2 Literature Review The following literature review served as the foundation to pursuing further study regarding OBPF and the potential for distributional inequities as a result of discreet policy biases. The review begins by discussing President Obama’s “completion agenda” (ACPA, 2020), and the widening attainment gap for low-income, non-traditional, and minority students (Bragg & Durham, 2012; Hanushek et al., 2020). The chapter then discusses the theories related to postsecondary student success and identifies the need for equitable policy outcomes (Kuh et al., 2006). The review then discusses Wolf’s (1978) theory of non-market failure, contending that government services like market-goods can create distributional inequities in marginalized populations. The chapter discusses the history of America’s support for public higher education and describes the economic and political factors that influenced many states to adopt more aggressive forms of performance funding. The review provides a background of empirical studies on the effectiveness of OBPF (Li et al., 2018), and describes the position of contemporary scholars regarding the importance of equity metrics (Cielinski & Pham, 2017; Umbricht et al., 2017; Fording et al., 2007). The review concludes by discussing Arkansas as a performance funding state with large populations of marginalized individuals which may be disproportionately harmed by policies that inadequately control for systemic bias. Review of Postsecondary Student Success Theories States began to shift toward improving postsecondary outcomes following national efforts to increase the number of adults holding a postsecondary credential (Blankenberger & Phillips, 2016). In 2009, President Obama introduced the American Graduation Initiative which sought to “substantially increase the number of postsecondary credential holders, to compete 27 internationally, and to satisfy workforce demands” (Blankenberger & Phillips, 2016, p. 885). The policy’s goal was to have the United States once again lead all other developed nations in the percent of working-aged adults holding a postsecondary credential by the year 2020 (ACPA, 2020). As of March 2021, the United States ranked 11th globally for individuals aged 25-34 holding a postsecondary credential (OECD, 2020), falling significantly short of President Obama’s goal. In 2016, the Higher Learning Commission, a postsecondary institutional accreditor, partnered with the Lumina Foundation to describe student success in a response to today’s completion agenda (Borden et al., 2019). The report titled Defining Student Success Data: Recommendations for Changing the Conversation places degree completion as the premier measurement of student success (Borden et al., 2019). Although there remains no singular definition of student success, Ewell and Wellman (2007) state that student success can be best understood as “getting students into and through college to a degree or certificate” (p. 2). The report acknowledges the growing attainment gap between certain student populations and mentions the need to improve overall postsecondary outcomes for specific marginalized student groups (Borden et al., 2019). A 2006 report by the National Postsecondary Educational Cooperative (NPEC) has served as a framework for understanding contemporary student success since its publication (Kinzie & Kuh, 2017). The NPEC report commissioned five authors to produce a comprehensive review of contemporary student success literature (Ewell & Wellman, 2007), determining that there were four primary themes related to student success. The NPEC report indicated that student success is largely influenced by the student’s precollege background, socioeconomic condition, and the influence of their peers (Kuh et al., 2006). Other factors related to student 28 success include the college’s overall behavior, faculty members approach to teaching, the alignment of efforts between K-12 and postsecondary education, and the use of evidence to inform future policies targeting student success (Ewell & Wellman, 2007). The NPEC report’s findings continues to provide much of the theoretical understanding on postsecondary student success (Kinzie & Kuh, 2017). Student success literature continues to indicate that postsecondary completion is influenced by a student’s economic conditions and prior educational experience, which has contributed to overall disparities in low-income, non-traditional, and minority degree completions (Kinzie & Kuh, 2017). A student’s ability to persist within the degree program is paramount to achieving a postsecondary credential (Kinzie & Kuh, 2017; Kuh et al., 2006). Vincent Tinto’s (1975, 1987, 2012) research has for decades served as a foundation to understanding the reasons students stop attending college (Smith, 2017). Tinto’s (1975) model for predicting student departure from college is based on student characteristics, which includes socioeconomic status, parental involvement, parental educational attainment, personality, measured ability, gender, and past education experiences (as cited in Smith, 2017). Ewell and Wellman (2007) contend that precollege experiences are a predominant factor to postsecondary outcomes, and that these experiences are also highly shaped by those same circumstances predicted in Tinto’s model (Ewell & Wellman, 2007; Kuh, et al., 2006). Tinto’s interactionalist theory remains the leading sociological perspective for understanding student drop out and contends that students must first distance themselves from their previous associations and incorporate within the postsecondary institution (Kuh et al., 2006). Tinto and Pusser mention that “each student exists in a particular context that shapes his or her probability of succeeding in higher education” (as cited in Ewell & Wellman, 2007, p. 5). Higher education 29 policy alone cannot alter a student’s precollege experiences, but a greater focus on the individual interventions which effect specific student characteristics could greatly impact student outcomes for marginalized populations (Ewell & Wellman, 2007). High and low-income students can experience the same institutions quite differently; with low-income students often experiencing other structural barriers complicated by poor academic preparation, racial prejudices, familial obligations, or access to financial aid resources (Rowen-Kenyon, 2007; Teasley, 2019). Low-income students tend to enroll in postsecondary institutions at least two years after their higher income peers (Rowen-Kenyon, 2007). Socialization is an important component to Tinto’s student success model, and students that enroll later tend to experience different social groupings than students that enroll immediately following high school (Kuh et al., 2006). Reduced familiar support, lower representation in on-campus engagement, and additional difficulties due to academic preparation mean that low-income students have a significantly greater burden to persist than their higher income peers (Bragg & Durham, 2012). This dynamic leading to poor student persistence rates for low-income students is particularly important for community colleges who typically serve higher proportions of lower-income and traditionally underserved students (Bragg & Durham, 2012; Rowan-Kenyon, 2007). Review of Non-Market Failure Theory The nation’s focus on improving public postsecondary outcomes has led to significant policy interventions for higher education (Li et al., 2018). Policy intervention, and specifically the degree to which policy should engage within the economy, has long been argued and understood by applying economic and political science principles (Fleming et al., 2007; Umbricht et al., 2017). Wolf contends that “the principal justification for public policy intervention lies in the frequent and numerous shortcomings of market outcomes” (Wolf, 1993, 30 p. 17). Although for-profit colleges have grown in popularity over the past two decades, public postsecondary institutions continue to lead in overall nationwide enrollments (Undergraduate Enrollment, 2020). The public’s investment into postsecondary education illustrates a policy intervention that originated from a market failure (Wolf, 1978). This category of market failure is known as distributional inequity and occurs when market economies prioritize efficiency over equity (Wolf, 1993). Applying Wolf’s theory to higher education, traditional private postsecondary education was often too expensive for most Americans; and in the absence of an equitable distribution of postsecondary education, states began adopting policies and systems to allow more individuals access to the service (Wolf, 1978). Like market failures, policy interventions can themselves lead to distributional inequities, which Wolf refers to as “non-market failures” (Wolf, 1978). Wolf (1978) mentions that “failures, whether market or non-market, are evaluated against the same criteria of success: allocative efficiency and distributional equity judged according to some explicit social or ethical norm” (p. 115). A policy intervention that produces outcomes, either intentionally or unintentionally, and acts counter to the public good is seen as a non-market failure (Wolf, 1978). A public policy intervention that creates unintended negative outcomes for marginalized populations is likewise referred to as distributional inequity within the non-market failure framework (Wolf, 1978). Distributional inequity is a major concern in performance management policies which typically prioritize bureaucratic efficiency over potential inequities (Umbricht et al., 2017). Fording et al. (2007) found that state performance management policies led to disproportional inequities in federal Temporary Assistance for Needy Families (TANF); which indicated that Black beneficiaries were 22% to 35% more likely to have experienced a sanction compared to a White beneficiary (Fording et al., 2007). The nation’s significant postsecondary completion gap 31 for both low-income, non-traditional, and minority students and the tendency for performance management policies to produce distributional inequity underscores the importance of better understanding the effect of, and inherent biases within, state funding policies for higher education (Dougherty & Natow, 2015; Fording et al., 2007; Umbricht et al., 2017). Review of Public Higher Education Funding In the absence of a singular federal model, public postsecondary education in the United States has largely been a responsibility of the state (Cohen & Kisker, 2010). Funding for institutions rapidly shifts between sporadic federal investments, tuition and fees, and fluctuations in state funding support (Kahlenberg et al., 2018; Lorenzo, 2018; Richardson & de los Santos, 2001). The relationship between state and federal governments and the institutions charged with achieving their respective postsecondary goals is significant. Beginning with the Morrill Act of 1862, the federal government offered states the ability to sell federally held lands to help subsidize the creation of land-grant institutions seeking to meet the workforce needs of the growing nation (Altbach et al., 2011; Cohen & Kisker, 2010). The strategy failed to create effective institutions capable of targeting critically needed training sectors (Cohen & Kisker, 2010); and a second Morrill Act was passed that standardized fiscal support by providing annual appropriations to each land grant college (Cohen & Kisker, 2010). Following the Morrill Acts, the next major federal investment into higher education occurred at the end of World War II with the creation of the Servicemen Readjustment Act of 1944, more commonly known as the G.I. Bill (Altbach et al., 2011). The G.I. Bill created a tremendous influx of students and increased the demand for a college diploma, which caused institutions to grow facilities, staff, faculty, services, and resources to meet the needs of the returning servicemen (Altbach et al., 2011; Cohen & Kisker, 2010). By the end of World War II, 32 the demand for higher education had dramatically increased, leading to a growing need for smaller community colleges and vocational schools focused on serving rural students and also addressing the growing need for individuals with vocational skills (Altbach et al., 2011; Cohen & Kisker, 2010). The Higher Education Act of 1965’s (HEA) Title IV program significantly transformed access to postsecondary education by introducing direct aid to low-and middle-income students (Pell Grant, 2020). HEA provided students with direct financial aid through several grants and low interest loans (Pell Grant, 2020). Most notable of these programs is the Pell award named for Senator Claiborne Pell who was instrumental in creating HEA’s Basic Educational Opportunity Grants (Pell Grant, 2020). Pell awards are often meaningful contributions to low-income students’ cost of postsecondary attendance. The average Pell award is $4,160 and can be used to cover the cost of tuition, fees, or other expenses supporting the completion of a Title IV approved postsecondary credential (Average Pell Award, 2020). Title IV of HEA significantly improved postsecondary access to low-income students and indirectly helped grown college enrollments nationwide (Altbach et al., 2011). Federal involvement in higher education has led to significant improvements to postsecondary access for low-income students (Altbach et al., 2011). Despite this support, the federal government provides very little direct funding to the public postsecondary institutions themselves (AACC, 2020). The American Association of Community Colleges reports that 33.3% of community college revenue is collected from the state, 27% is collected from tuition and fees, 20.3% is collected from local sources, 11.4% is collected directly from the federal government, and 7.9% is collected from other sources (AACC, 2020). Although state funding represents the largest source of fiscal support for community colleges, that funding also ties colleges to the 33 goals of state policy makers (McLendon et al., 2009; Rabovsky, 2014); and over the past decade policy makers have shifted postsecondary goals toward improving postsecondary completion rates (McLendon et al., 2009; Rabovsky, 2014). Obama’s American Graduation Initiative brought national attention to overall lagging adult postsecondary completion as well as a growing achievement gap just as the country entered one of the worst economic recessions in its history, forcing many states to make significant cuts to public higher education (Leachman & Mail, 2014; Lorenzo, 2018). The economic recession of 2007 drastically impacted state revenues, causing many states to experience budget deficits (Leachman & Mail, 2014; Katsinas, et al., 2014). Reductions in tax revenues meant that states had to choose between raising taxes or making budget cuts to public services (Mitchell et al., 2019). Following the recession, however, only 16% of states used any form of tax or fee increase to cover budget deficits (Mitchell et al., 2019). Mitchell, Leachman, and Masterson (2016) report that from the 2008 to 2012, 45% of budget gaps were in some way corrected by making spending cuts to public postsecondary institutions. The decline in state support for public institutions of higher education is a symptom of the legislative priorities defined through the policy development process (Tandberg, 2010). Unfortunately, policy decisions “do not occur within a vacuum immune to politics or other budgetary forces” (Tandberg, 2010, p. 442). McLendon et al. (2009) researched several variables which were suspected of having a statistically significant effect on state appropriations to higher education. Of the study’s substantial findings, three factors contributed significantly to state appropriation levels for higher education, including: partisan control of legislatures and governorship, professionalism of the legislature, and the size of the state’s lobbying support for higher education (McLendon et al., 2009). McLendon et al. (2009) also found that with regard to 34 state funding for public higher education, party affiliation matters, with a $0.05 decline in appropriate support per $1,000 of personal income for every 1% increase in Republican legislative representation, and a $0.023 decline for Republican governors. In a fixed-effects analysis of state funding trends spanning the years from 1980 to 1990, Democratic control of the state legislature and gubernatorial seats had a positive effect on funding levels, while a separate fixed-effects study covering 1976 to 2000 similarly found that Republican governance had a negative influence for state higher education appropriations (McLendon, 2009). Tandberg (2010) expands on the influencers of state fiscal policy by introducing a framework that includes interest group activity, mass political attributes, governmental institutions, state higher education factors, the previous year’s appropriations amount, economic and demographic factors, and political culture. These factors feed into a three-tiered legislative decision procedure which considers other budgetary demands, attributes of the decision situation, and attributes of the policymakers themselves (Tandberg, 2010). Tandberg (2010) found that the inclusion of political ideology created a more explanatory model than without, and that results indicated that political ideology significantly impacts state expenditures. The analysis found that states with liberal ideology were more permissive toward higher education funding, and states with more conservative ideological underpinnings were less supportive of higher education funding (Tandberg, 2010). The partisan makeup of a state has a real effect on state funding policy (Tandberg, 2010; McLendon et al., 2009). Recent elections have further influenced the partisan make-up of state legislatures; since 2008 more than 800 legislative districts previously belonging to Democrats are now being represented by Republican legislators (Kelderman, 2016). The director of state relations and policy analysis for the State-Colleges Association, Thomas L. Harnisch, contends 35 that legislative priorities in conservative state legislatures will likely consist of tuition freezes and increases in incentive-based appropriations for production of state priorities, which is to say further adoption of performance management models (Kelderman, 2016). Additionally, performance management will likely become a more popular method for legislatures to improve public sector outcomes and to better create organizational learning and improvement, as well as to increase transparency and accountability for improved oversight and political responsiveness (Rabovsky, 2014). Poor economic conditions and significant tax-cuts disproportionately impact higher education (McLendon et al., 2009). The post-recession economy caused state budgets to experience serious competition between detention programs, entitlement programs, and all other functions of the state government (Fowles, 2014). State budgets prioritize expenditures supporting healthcare, state pensions, K-12 education, prisons, and transportation over those going to fund public higher education (Katsinas, et al., 2014); and when states enact budget reductions, public higher education is disproportionately impacted (McLendon et al., 2009; Fowles, 2014). McLendon et al. (2009) found that “over the past quarter-century, higher education suffered disproportionately in state funding” with regards to other state obligations (p.687). 2007’s economic recession caused reductions to postsecondary spending by as much as 20%, and only three states increased spending per-student, leaving 47 states to experience reduction to their spending for higher education (Chokshi, 2015). Kansas for instance, following significant tax-cuts experienced almost $1 billion in revenue deficits over two-years forcing Governor Brownback to cut over $40 million in combined funding from secondary and postsecondary education (Eligon, 2015). On average, states spend 18% less per student than they did in 2008; and only four states, including Montana, North Dakota, Wisconsin, and Wyoming 36 have restored funding to pre-recession levels (Mitchell, et al., 2016). According to the Center on Budget and Policy Priorities, 25 states’ fiscal support for higher education is 20% lower than 2008, nine states have only 30% of their 2008 support, and Arizona and Illinois have lost more than half of their pre-recession state support (Mitchell et al., 2016). Post-recession budget deficits significantly affected public postsecondary spending, which caused many institutions to increase tuition and fees (Fowles, 2014; Mitchell et al., 2016). In 1988, postsecondary institutions received 3.2 times more fiscal support from their respective state governments than they collected through student tuition and fees; today, those institution only receive 1.2 times more fiscal support from state governments than they do from student tuition and fees (Mitchell, et al., 2016). Nationwide, funding support for higher education dropped by approximately 12% from 2003-2012 (Funding Trends, December 2014). In answer, institutions almost entirely offset this reduction in state support by increasing tuition and fees to students (Fowles, 2014). Following the recession, tuition grew by an average of 33% across the United States, and in Arizona the price of tuition grew by 87.8% (Mitchell et al., 2016). The political ideologies of state lawmakers have an even greater impact on the implementation of performance funding policies for higher education, where party affiliation again mattered. Alshehri (2016) contends that “republican legislators appeared to tremendously advocate for performance funding policy compared to their counterpart” (p. 36). As state governments continue to prioritize the improvement of postsecondary outcomes (Hearn, 2015; Lorenzo, 2018), states opposing tax increases have increasingly turned to performance management policies to improve efficiencies within postsecondary institutions (Hearn, 2015; Li et al., 2018). Since 2008, 75% of states have implemented some form of performance funding for higher education (Li et al., 2018; Lorenzo, 2018), underscoring the importance of developing a 37 greater understanding of the impact of performance-based policies on institutions that serve large populations of traditionally underrepresented students (Li et al., 2018; Kelchen, 2019; Cielinski & Pham, 2017; Hearn, 2015; Bragg & Durham, 2012). Review of Performance Funding in Higher Education Performance management has for decades served to hold public institutions of higher education accountable to public policy goals (McKeown-Moak, 2013). The first use of performance management to determine support for higher education institutions can be traced back to post World War II when the federal government began regulating which institutions could accept funds offered through the G.I. Bill (Altbach et al., 2011; Hearn, 2015). Burke (2005) states that “accountability programs for higher education have shifted over time from system efficiency to educational quality, to organizational productivity, and to external responsiveness to public priorities or market demands” (as cited in Alshehri, 2016, p. 34). Performance management in higher education is typically known as performance funding and uses state defined metrics to award institutional funding (Alshehri, 2016). In 1979, Tennessee adopted the first higher education performance-based funding formula which linked enrollment growth to new institutional funds (Hearn, 2015). Early forms of higher education performance funding provided new funds above the existing base funds and is known as base-plus performance funding (Alshehri, 2016). The base-plus approach to performance funding anticipates the institutional needs using a set of criteria and attempts to fund those institutions proportionally to meet the additional need over the previous year’s base-funding (Hearn, 2015). These earlier forms of performance funding allowed institutions to operate from a safe starting-point and contained no potential loss in funding due to unobtained performance measures. By the 38 year 2000, less than 20 states had attempted to implement some form of a performance funding model to help direct state appropriations to public higher education (Hearn, 2015). The momentum of performance-based policies for higher education is evident. By 2019, 38 states had adopted some form of performance-based funding, and over 20 states had adopted, or were planning to adopt the more aggressive OBPF model described above (Li et al., 2018; Hearn, 2015; Wayt & LaCost, 2016). Republican led legislature have overwhelmingly preferred performance-based funding models to help determine state appropriations for higher education (Alshehri, 2016; Rabovsky, 2014). Eager to improve postsecondary outcomes but reluctant to raise taxes, many states have adopted more aggressive forms of performance funding that focus on postsecondary completion growth (Hillman et al., 2014; Mitchell et al., 2016). Outcomes based performance funding links a portion of an institution’s funding to specific performance outcomes (Dougherty et al., 2016; Hillman et al., 2014; Rutherford & Rabovsky, 2014). OBPF differs from traditional forms by allowing potential funding losses in subsequent years when the institution’s outcomes fail to show performance (Dougherty et al., 2016; Productivity Funding, 2020). OBPF, like traditional performance funding, relies on a base-funding level to determine a starting point for an institution’s funding (Alshehri, 2016). OBPF alone offers the potential loss to annual state funds when state-defined goals are not achieved (Alshehri, 2016). When an institution in an OBPF state fails to meet the prescribed performance metrics, the institution loses a proportion of their base-funding. The amount appropriated to the institution becomes the new base-funding level for subsequent fiscal years, and the institutions that lost funding must now achieve state goals with less overall fiscal support from the state or risk further base-funding cuts (Alshehri, 2016). Foundationally, this transition of funding models from enrollment based to completion-based metrics represents a reversal in priority, from 39 meeting the needs of the institution to now meeting the needs of the state, its economy, and the students served by higher education institutions (McKeown-Moak, 2013). Over 20 states have adopted, or are in the process of adopting, a form of OBPF (Li, 2017). Many of these state initiatives had originated at their respective governor’s offices following recommendations made by the National Governors’ Association Complete College America Initiative (McKeown-Moak, 2013). Each state has unique priorities that are guided by their own economic identity; however, a framework of best practices does exist. A list of “Guiding Principles for Developing and Establishing Institutional Performance Indicators” was presented to state lawmakers and institutions to help guide the development of metrics that are successful in creating effective outcomes (McKeown-Moak, 2013). Those best-practices include: the need for credibility; linkage to the mission, strategic plan, and policy goals; simplicity; reliance on valid, consistent, and existing information; recognizing range of error in measurement; adaptable to special situations; minimizing the number of indicators; reflecting industry standards and best practices; incorporating input, process, output, and outcome measurement; and incorporating quantitative and qualitative measures (McKeown-Moak, 2013). The OBPF model represents a drastic paradigm shift from the traditional higher education funding model and creates an opportunity for unintended consequences to poor and traditionally marginalized students (Cielinski & Pham, 2017; Umbricht et al., 2017). Community colleges serve large populations of traditionally underrepresented students (Kelchen, 2019). As Kuh et al. (2006) indicated, prior education experience matters to postsecondary student success, with many traditionally underrepresented students entering community colleges requiring remedial coursework beyond the traditional degree pathway (Li et al., 2018). Although many OBPF policies include goals targeting these student populations, student success initiatives often 40 require additional resources to support improvements to academic pathways, advising, and an increase in the use of disaggregated institutional data (Kelchen, 2019). Community colleges are finding it difficult to improve student outcomes for targeted populations without significant additional funds (Lorenzo, 2018). More concerning is the potential for funding inequities due to inherent systemic biases within the policies, potentially inadvertently rewarding institutions with smaller overall populations of low-income, non-traditional, and minority students (Cielinski & Pham, 2017; Lorenzo, 2018; Umbricht et al., 2017; Wolf, 1978). Understanding the Effects of Outcomes-Based Performance Funding The adoption of OBPF is often influenced by the state’s desire to grow the total number of earned credentials (Li et al., 2018). A significant portion of existing research into the effects of OBPF has utilized principle-agent theory to understand the relationship between state funding, state goals, and institutional behavior toward completing those goals (Hillman et al., 2014; Hillman et al., 2017; Kelchen, 2019; Li & Kennedy, 2018; Rutherford & Rabovsky, 2014). Principle-agent theory was developed by researchers Ross and Mitnick in the early 1970s (Mitnick, 2019). Both Ross and Mitnick sought to understanding the appropriate compensation required for an intended outcome (Mitnick, 2019). Applied to performance funding, the state acts as the “principle” and postsecondary institutions acts as “agents” (Kelchen 2019). Principle-agent theory contends that the principle lacks the ability to make direct action and therefore enlists agents on its behalf, often employing incentives or “ex post sanctions” to control bureaucratic outcomes (Poth & Selck, 2009, p. 139). As state legislatures are understandably incapable of directly increasing student credential outputs, ex post sanctions take the form of reduced appropriative supports to postsecondary institutions when these goals are not met. 41 The degree to-which institutions respond to the ex post sanctions on state funding is supported by resource-dependence theory (Fowles, 2014; Pfeffer & Salancik, 2002). Resource dependency theory is often used within theoretical frameworks of studies focusing on OBPF (Dougherty et al., 2014; Fowles, 2014; Kelchen, 2019; Li & Kennedy, 2018). Introduced in 1978 by Pfeffer and Salancik’s The External Control of Organizations: A Research Dependence Perspective, resource dependency theory seeks to understand the impact that external pressures have toward organizational decision making (Pfeffer & Salancik, 2002). As state funding remains the largest source funding support for public postsecondary education (AACC, 2020); resource dependency theory holds that public colleges will make internal decisions that create the most advantageous funding outcomes (Fowles, 2014; Kelchen, 2019). As states have shifted their priorities toward improving student outcomes, resource dependency theory supports the institution’s simultaneous focus on achieving the state’s goals. Despite the assumptions supported by principle-agent theory and resource dependency theory, research into the effectiveness of performance funding has yet to provide any empirical evidence that these policies induce institutional behaviors that improve overall student completion rates (Blankenberger & Phillips, 2016; Dougherty et al., 2014; Hillman et al., 2014; Hillman et al., 2017; Li & Kennedy, 2018; Rutherford & Rabovsky, 2014). Although several studies have shown small improvements to short-term credentials and marginal effects to 6-year graduation rates (Dougherty et al., 2014; Hillman et al., 2014; Li & Kennedy); there remains growing concern that the policy’s real impacts may be disproportionally affecting low-income, non-traditional, and minority students (Lorenzo, 2018). Many of the existing studies on OBPF suggest the inclusion of equity metrics that specifically target underrepresented student populations, noting the potential that OBPF may 42 have toward marginalized student access if institutions begin to restrict admissions to only the most well-prepared students (Cielinski & Pham, 2017; Gandara, 2019; Horn & Lee, 2017; Kelchen, 2019; Li et al., 2018; Umbricht et al., 2017). Despite the overwhelming support for equity metrics in OBPF, there is little research indicating the policy’s effect on marginalized student populations or the efficacy of equitability metrics themselves. There are only three current quantitative studies related to the policy’s impact on equitability. A study of Indiana’s OBPF model found that the policy led to a decline in overall admission rates at institutions without open-access policies, and an increase in the average standardized test scores for students, indicating at-least some preferential admissions for students with better academic preparedness (Umbricht et al., 2017). Although Umbricht et al. (2017) had indicated a slight impact to admissions, two other studies failed to show any meaningful impact to equitability for community colleges or minority serving institutions (Kelchen, 2019; Li et al., 2018). In fact, only one study exists within the OBPF literature that focuses on the equitability of funding outcomes based on the potential for systemic biases within the policy. A 2018 study of minority-serving institutions in Texas and Washington compared minority and non-minority serving institution funding outcomes, failing to identify any preferences based on the type of institution (Li et al., 2018). Although the OBPF literature has extensively sought, and failed, to identify the impact the policies have on student completions at public postsecondary institutions, researchers continue to indicate the importance of including equity metrics within state OBPF policies (Cielinski & Pham, 2017; Gandara, 2019; Horn & Lee, 2017; Kelchen, 2019; Li et al., 2018; Umbricht et al., 2017). As states continue to implement these aggressive policies, there remains a critical gap in the literature regarding the policy’s potential for funding inequities at community 43 colleges that serve large populations of low-income, non-traditional, and minority students, as well as the overall efficacy of the equity metrics themselves (Cielinski & Pham, 2017). Chapter Summary This chapter provided the economic and political conditions, as well as the underlying theories that influenced over 20 states to implement an outcomes-based performance funding policy for public higher education (Li, 2017). Over the past 30 years, the United States went from having the highest percentage of adults age 25-34 holding a postsecondary degree, to now having only the 11th highest percentage (OECD, 2020). To address this decline, President Obama launched the American Graduation Initiative which sought to improve postsecondary completion totals and began the nation’s focus on what has become known as the completion agenda (ACPA, 2020). During this same time period, the United States experienced an economic collapse that forced many states to reduce spending for higher education (Chokshi, 2015); and over a decade after the financial crisis state spending for higher education continues to lag behind pre-recession totals (Mitchell et al., 2016). Tandberg (2010) determined that partisanship strongly influences a state’s fiscal policy; and McLendon et al., (2009) found that Republican legislatures and governors tend to reduce higher education funding compared to their Democratic counterparts. Following the economic recession, over 800 legislative districts previously belonging to Democrats have flipped to Republicans (Kelderman, 2016). Overwhelmingly, these legislatures have turned to performance-based funding models to make funding decisions for public postsecondary institutions (Alshehri, 2016). States began adopting performance management policies to help inform funding policy in 1979 (Hearn, 2015). As of today, 75 percent of states use some form of performance-based 44 funding (Li et al., 2019; Wayt & LaCost, 2016). The relationship between state legislatures and public postsecondary institutions is best understood through the lens of principle-agent theory, which suggests that states acting as “principles” enlist public colleges to act as “agents” capable of improving postsecondary student outcomes (Mitnick, 2019). The institutions’ response to this relationship is guided by resource dependency theory, which understands organizational behaviors as being guided by external pressures (Fowles, 2014). Using this framework, states prioritize postsecondary completions and hold public colleges accountable to specific outcomes. The postsecondary institutions having experienced external funding pressure would then change the organization’s behavior to best optimize their chances of maintaining or increasing funding. Unfortunately, this approach negates the significant investment that colleges must make in order to address postsecondary student success (Bragg & Durham, 2012; Kinzie & Kuh, 2016; Saxon, 2017). A 2006 report commissioned by the National Postsecondary Education Cooperative (NPEC) introduced a framework of student success which indicated that students precollege academic experience, socioeconomic conditions, and peer influences largely impact postsecondary student success (Kinzie & Kuh, 2017). Overall, the NPEC report contends that low-income students are far less likely to persist in, or graduate from, postsecondary institutions (Kuh et al., 2006). Community colleges must make significant investments into improving the overall student experience for these students with, in some cases, increasingly less funds. Community colleges often serve as the gatekeepers to postsecondary education for low-income, non-traditional age, and minority students (Bragg & Durham, 2012; Hearn, 2015; Kelchen, 2019; Ma & Baum, 2016). States that implement OBPF could be creating funding disparities for community colleges that serve larger than average numbers of low-income, non-traditional age, and minority students. Arkansas’s community colleges have experienced 45 significant funding losses in two of the funding policy’s three years (Productivity Outcomes, 2020). Arkansas ranks 40th nationwide in college readiness (Rankings, 2020); and many of the students attending the state’s public community college require additional developmental education in order to successfully complete a postsecondary credential (Ganga et al., 2018). Arkansas’s community colleges serve students from vastly different academic backgrounds, and the institution’s enrollment characteristics closely reflect the communities they serve (Income & Poverty, 2020; NSF Table Builder, 2020). Wolf’s (1978) non-market failure framework contends that disproportional inequities can arise from public policy interventions. Applying non-market failure theory to the Arkansas productivity funding model, institutions may be experiencing disproportional inequities if funding outcomes are largely explained by the student characteristics of the institution. Arkansas’s productivity model awards the majority of an institution’s annual index-score based on the institution’s average growth in the total number of credentials awarded (Productivity Funding, 2020). Much of the literature regarding OBPF mentions the importance of including equity metrics to improve outcomes for targeted student populations (Cielinski & Pham, 2017; Gandara, 2019; Horn & Lee, 2017; Kelchen, 2019; Li et al., 2018; Umbricht et al., 2017). Although Arkansas’s model includes several equity metrics, awarding an additional 0.29 points for low-income, non-traditional age, and minority students completing a credential (Productivity Funding, 2020), there remains no evidence that those metrics adequately address the nation’s known achievement gap (Graduation Rates, 2019). This study addressed a critical gap in the literature regarding OBPF’s potential to create funding inequities as a result of unintended biases against specific student characteristics (Cielinski & Pham, 2017; Umbricht et al., 2017; Wolf, 1978). 46 Chapter 3 Methodology The purpose of this study was to determine if student enrollment characteristics at Arkansas community colleges impacted the relationship between credentials awarded and the institution’s annual productivity index score. This study’s longitudinal data was analyzed using a form of growth curve modeling known as a hierarchical linear model (Curran et al., 2010), and included a single outcome variable and four independent variables. Results from this analysis were compared to the study’s hypotheses to determine if statistical significance existed based on the assessment of specific model-fit criteria found within this chapter. Epistemology This study is foundationally postpositivistic in its approach to understanding the relationship between funding outcomes and the student characteristics of an institution’s enrollment. Postpositivism arose from the positivist worldview which sought to understand the world through rationalities and the identification of absolute truths (Creswell, 2009). Positivism was developed in the mid-19th century by the French sociologist Auguste Comte, who focused on understanding the world through empirical evidence, observation, and reasoning (Creswell, 2009). The positivist approach sought to understand phenomena based on the assumption that everything is founded in rationalities and that more observation will lead to better understanding, with little consideration given toward the complexities of human behavior (Perumal & Padmanabhan, 1988). Postpositivism moves beyond the positivist approach by understanding that knowledge related to human behavior is imperfect and therefore cannot be absolute (Creswell, 2009). Researchers using the post-positivist worldview typically employ quantitative 47 analysis and hypothesis testing to discreetly analyze outcomes that may influence the identified research problem (Creswell, 2009). Although this study’s methodology is postpositivistic, the author admits that the identification of the research problem and the development of the underlying hypotheses were strongly influenced through the critical perspective (Creswell, 2009). Critical theory can be traced back to the University of Frankfurt’s Institute for Social Research, and is grounded in the philosophies of Kant, Hegel, Nietzsche, and Marx (Bronner, 2011). The critical perspective has traditionally sought to identify political and cultural structures that disenfranchise portions of society (Bronner, 2011); and is often used to highlight oppressive systems that preclude individuals from experiencing freedom, equality, and equity (Marinopoulou, 2017). Research founded in critical theory seeks to develop knowledge that can be transformatively applied toward political, social, and economic systems of oppression (Bronner, 2011; Creswell, 2009). This study’s research problem was identified when the researcher observed the majority of Arkansas’s community colleges lose funding in two of the productivity model’s three years (Productivity Outcomes, 2020). As previous literature has demonstrated, community colleges tend to serve higher percentages of traditionally underrepresented students (Kelchen, 2019). Previous literature has likewise indicated that these students are least likely to complete a postsecondary credential (Kuh et al., 2006). The potential for an annually repeated loss of funding to the state’s community colleges due to discreet biases against specific student characteristics could detrimentally impact the quality of postsecondary education received by marginalized student populations. The author is essentially observing potential systemic biases within the state’s funding policy through the critical perspective and is seeking to bring awareness to these potential inequities using postpositivistic research methodologies, with the 48 specific goal of instigating transformative change to the way Arkansas is addressing student success in marginalized student populations (Weber, 2020). Research Design Researchers in education often experience data that is hierarchically grouped, where observations often represent clusters within time, classrooms, schools, districts, and/or states (Woltman et al., 2012). These clusters often lead to a key failure in traditional regression analyses that assumes the data are independent of one another, and if that interdependence is not adequately accounted for, researchers could inaccurately identify a relationship leading to either type-1 or type-2 error (Kwok et al., 2009). Hierarchical linear models allow researchers to address the interdependence of data by creating a subject-specific slope and intercept for each covariate included within the model (Kwok et al., 2009). Although the hierarchical linear model (HLM) procedure allows more flexibility than traditional regression analyses, researchers conducting HLM must make a series of decisions throughout the analysis. Those decisions often include: producing a model that is informed by contextually appropriate theories regarding the phenomenon being studied; addressing the specific research questions being studied; identifying what variables have fixed and/or random effects, identifying an appropriate covariance structure for the random effects; and determining the most parsimonious model that best reduces the unexplained variation in the dependent variable (Curran et al., 2010; Kwok et al., 2009; Woltman et al., 2012). This complex procedure often leads to the production of several models that are compared using the log-likelihood ratio, or the deviance metric (-2LL), to determine the best fit model, those procedures are discussed further within this chapter (Woltman et al., 2012). 49 The author used an inferential research approach with longitudinal ex post facto data collected from Arkansas’s community colleges (Kothari, 2004). The study used a form of growth curve modeling known as a hierarchical linear model, which can also be stated as a multi-level linear mixed-effects model (Curran et al., 2010). These forms of growth curve models often utilize a repeated measures analysis which estimates “inter-individual variability in intra-individual patterns of change over time” (Curran et al., 2010, p. 2). This study’s HLM procedure was used to determine the correlative relationship between funding increases or decreases over three time-points using four independent variables. The study’s independent variables included a single time-varying covariant representing the annual change in credentials awarded by an institution, and three time-invariant covariates related to student characteristics. The three time-invariant covariates proposed as predictor variables included the average percent of Pell receiving students, the average percent of non-traditional age students, and the average percent of minority students (McCouch & Kaniskan, 2010). The study used the aforementioned student characteristics as independent variables based on previous literature on the relationship between postsecondary completion and specific student demographics (Kuh et al., 2006). The study’s HLM analyzed ex post facto longitudinal data in order to determine the relationship between an institution’s awarded credentials and the subsequent productivity index score, and to what extent the inclusion of additional student characteristic data improved the model’s ability to explain the remaining variance in the outcome variable (Curran et al., 2010; Woltman et al., 2012). This design allowed the researcher to determine if funding losses or gains were attributed to discreet biases against specific student characteristics, or if the model’s 50 included equity metrics were adequately controlling for the known achievement gap between specific student groups (Borden et al., 2019; Kuh et al., 2006). Research Questions Arkansas’s performance funding model has caused 14 out of 22 community colleges to lose funding in two of the model’s three years (Productivity Funding, 2020). Although Arkansas’s productivity model includes equity metrics, prior to this study, it was unknown if those metrics adequately corrected for differences in student degree completion for different income, age, and racial groups. If those metrics are not adequately weighted, some community colleges serving higher percentages of marginalized students may be experiencing distributional inequity (Bragg & Durham, 2012; Cielinksi & Pham, 2017; Umbricht et al., 2017; Wolf, 1978). This study sought to determine if performance-based funding outcomes at Arkansas community colleges are significantly impacted by student enrollment characteristics after controlling for the annual difference in awarded credentials. The following research questions guided this study. 1. What is the relationship between an institution’s productivity index score and the total credentials earned? 2. Is there a relationship between the percent of low-income students and productivity index scores after controlling for credentials, and if so, does the percent of low-income students moderate the relationship between an institution’s productivity index score and the total credentials earned? 3. Is there a relationship between the percent of non-traditional age students and productivity index scores after controlling for credentials, and if so, does the percent 51 of non-traditional age students moderate the relationship between an institution’s productivity index score and the total credentials earned? 4. Is there a relationship between the percent of minority students and productivity index scores after controlling for credentials, and if so, does the percent of minority students moderate the relationship between an institution’s productivity index score and the total credentials earned? Population This study was focused on identifying potential funding inequities at Arkansas’s community colleges; therefore, this study’s population included 22 public two-year postsecondary institutions within the state of Arkansas (Compare Institutions, 2020). Each state has unique priorities that are guided by their own economic identity making policy comparisons difficult (Li, 2017). State-level OBPF policy analysis is no different. Inter-state differences imbedded within the policies are distinctly specific with regards to both punitiveness and potential reward, often referred to as “dosage”, as well as the specific weighting for credentials, student course progression, developmental success, etc. (Li, 2017). Despite the inherent policy differences, there may be similarities within this study’s design that may help other researchers generalize specific elements of this study to other states. The researcher applied a similar but less specific sampling frame and identified 18 additional states other than Arkansas using tenuously similar funding policies: with Montana and Ohio having the most similar policies with regard to credential scoring and equity metrics (Li, 2019). Sample This study proposes a specific sampling frame that includes community colleges funded by an OBPF policy that uses the exact dosage, metrics, and categorical weightings as the 52 Arkansas productivity model. Due to the restrictive sampling frame, this study proposes a form of non-probability sampling known as a total population sample (Kothari, 2004; Laerd, 2020). Non-probability sampling allows the researcher full discretion toward the determination of the study’s participants and does not require a measurement of sampling error (Kothari, 2004). This method of sampling was chosen based on the study’s relatively small population, and few observations, from which to extract meaning using HLM. Growth curve modeling typically requires sample sizes of approximately 100 participants (Curran et al., 2010), and therefore it is important to note that the study results are limited by the small sample size and results may be inconsistent due to the study being underpowered (Curran et al., 2010). No other state utilizes the same funding model (Li, 2019); and therefore, a total population sample that includes all of Arkansas’s 22 community colleges will provide the study with the highest possible number of observations from which to estimate between-subjects variances using three annual fiscal cycles of within-subject changes (Curran et al., 2010). Another factor which led the researcher to use a total population sample is the study’s epistemological goal to develop knowledge for the purpose of instigating political transformation within Arkansas through the application of postpositivistic quantitative research methods. The researcher’s purpose is to inform Arkansas legislators, higher education professionals, and other interested persons within Arkansas of potential discreet biases against specific student characteristics within the state’s higher education funding model (Kuh et al., 2006; Weber, 2020; Wolf, 1978). Building upon the research purpose, the most cogent use of data to influence the decisions of lawmakers is to draw inferences from observations within their very state. For these reasons, this study’s sample included Arkansas’s 22 public community colleges and includes funding outcomes for three fiscal years from which to measure individual 53 productivity index scores. The institutions included within the study’s sample each experience the same conditions leading to measurement of the independent variables. Each institution reports their percentage of underserved student populations to IPEDS annually; and each institution reports their total awarded credentials to the Arkansas Division of Higher Education annually. This data is available online and represents a total of 22 participants spanning three years, with a total of 66 repeated-measures observations of the outcome variable. Utilizing all available data provided the researcher with a more complete understanding of how each institution’s productivity index score was affected by specific student characteristics. This method of sampling did not add undue burden to the researcher (Laerd, 2020). Table 2 describes the student characteristics for each of the twenty-two community colleges included in the study and reflects years 2015-2018 for the institution’s average percent of Pell receiving students and average percent of minority students. The average percent of non-traditional age students reflects years 2011-2018. 54 Table 2 Average Institutional Characteristics of Sample in Percentages Institution Average Percent Low-Income Average Percent Non-Traditional Age Average Percent Minority 1 25.98 38.28 33.19 2 30.80 31.76 57.29 3 25.97 29.90 18.30 4 21.96 31.69 11.44 5 40.82 41.81 3.03 6 19.45 32.20 18.98 7 42.02 38.62 4.55 8 24.61 26.81 32.25 9 27.70 41.39 36.01 10 50.27 39.83 17.32 11 44.59 34.08 6.32 12 21.72 31.61 18.05 13 42.82 42.84 2.73 14 26.81 18.73 49.04 15 38.67 39.77 39.33 16 46.89 50.07 61.12 17 15.52 25.74 33.72 18 27.68 38.43 4.40 19 40.74 34.14 8.33 20 40.77 34.18 15.86 21 42.00 53.64 48.44 22 41.06 33.72 41.52 55 Data Collection Data collection for this study was reviewed by the University of Memphis’ Institutional Review Board and was determined to be an exempt study. Data for this study was collected from publicly available sources using front-facing websites. The two primary data sources for this study were the National Center for Education Statistics’ Integrated Postsecondary Education Data System (IPEDS), and the Arkansas Division of Higher Education’s Annual Comprehensive Report (ADHE). IPEDS data includes the percent of Pell receiving students for each institution, and the percent of minority enrolled students for academic years 2015, 2016, 2017, and 2018; as well as the percentage of students over the age of 24, which includes averages for years 2011 through 2018. The percentages related to the study’s targeted student characteristics were derived using enrollment totals provided through IPEDS and reflect years 2011 through 2018. Each institution’s performance-index score was obtained from ADHE for fiscal years 2020, 2021, and 2022. The study’s variables were saved using the univariate format where “each row represents a specific time point rather than a participant” (Kwok et al., 2009, p. 8). The Excel sheet included a total of seven columns: Column A reflected the institution’s abbreviation and served as a participant ID; Column B was labeled TIME and was coded as either a zero, one, or two for model years two through four, respectively; Column C, labeled PFINX, was the study’s proposed outcome variable and reflected each institution’s annual productivity index score; Column D, labeled CREDS, represented the annual difference-in-averages of an institution’s awarded credentials and reflected the same methodology used within the productivity model to award points for a respective model year; Column E, labeled AVG_PELL, reflected the average percentage of Pell receiving students; Column F, labeled AVG_AGE, reflected the average 56 percentage of non-traditional age students; and Column G, labeled AVG_MIN, reflected the average percentage of minority enrolled students. Variables The data for this study was collected into a single Excel spreadsheet where the researcher reported the single outcome variable and the four included predictor variables, with the addition of the time-varying covariate for time. The data included a single longitudinal outcome variable that reflects each annual productivity index score for each institution over the three included observations of the dependent variable. The time-varying covariate for credential growth reflects the same academic years used in Arkansas’s productivity model for a respective model year; and is computed using a difference-in-averages. Arkansas’s productivity model year 2 reflects the difference in the average credentials earned for academic years 2017, 2016, and 2015, and the average credentials earned for academic years 2016, 2015, and 2014. Each model year adds an academic year to the leading average and drops an academic year from the trailing average; therefore, model year 3 reflects the difference in the average credentials earned for academic years 2018, 2017, and 2016, and the average credentials earned for academic years 2017, 2016, and 2015. The formula used to determine credential growth for the ith institution in the funding model’s third year can be expressed as: CREDSi2 = (average credentials for 2018, 2017, 2016) – (average credentials for 2017, 2016, 2015) The three-remaining time-invariant covariates of interest were computed using averages of the years described in Table 3, below. The first time-invariant covariant of percent of Pell receiving students was computed into an institutional average for academic years 2014-2018 by dividing the total number of students receiving Pell by the institution’s total 12-month 57 enrollment and was saved as AVG_PELL. A second time-invariant covariant relating to the percent of minority enrolled students includes only those racial categories identified within the Arkansas productivity model. The study’s minority variable included Black or African American and Hispanic students for years 2014-2018 divided by the total 12-month enrollment, and was saved as AVG_MIN. A third time-invariant covariant relating to the percentage of non-traditional age students was computed using available data from IPEDS; and unlike Pell and minority student designations, is not an annually required data metric for the NCES data collection surveys. At the time of this study, data is only available from all 22 colleges for years 2011, 2013, 2015, and 2017; although some institutions chose to report that data. The non-traditional age independent variable was created by dividing the total number of students over the age of 24 by the institution’s total 12-month enrollment using all available data for years 2011 through 2018 and was saved as AVG_AGE. Table 3 below illustrates the conceptual design for the data used within the study’s proposal. Table 3 Conceptual Model for Study’s Longitudinal Data Time Productivity Index Scores Credentials Student Characteristics (Pell, Age, Minority) Time-Varying Covariate Outcome Variable Time-Varying Covariate Time-Invariant Covariates Model Year 4 Fiscal Year 22 Difference in average of (19,18,17) and (18,17,16) Average for Years 2015-2018 or 2011-2018 Model Year 3 Fiscal Year 21 Difference in average of (18,17,16) and (17,16,15) Average for Years 2015-2018 or 2011-2018 Model Year 2 Fiscal Year 20 Difference in average of (17,16,15) and (16,15,14) Average for Years 2015-2018 or 2011-2018 58 The data format presented in Table 3 follows the logic model associated with each respective year of the performance funding model. The model’s year one was excluded due to the “hold harmless” clause that blocked any institution from earning a negative productivity index, which were scored as zero in the model’s funding release by ADHE. Hypothesis The study’s hypotheses were based on the researcher’s assumption that there were distributional inequities within the policy’s performance metrics and that the inclusion of bonus points for underrepresented students did not preclude the model from those discreet biases (Bragg & Durham, 2012; Kelchen, 2019; Umbricht et al., 2017). The study’s included hypotheses were based on literature regarding postsecondary student success and the additional supportive learning needs of students with deficient prior learning experiences (Chen, 2016; Kuh et al., 2006). The hypotheses were also informed by literature related to political biases within policy platforms (Weber, 2020), and systemic biases found within efficiency systems; both of which often lead to distributional inequities within marginalized populations (Cielinski & Pham, 2017; Umbricht et al., 2017; Wolf, 1978). The following hypotheses and conditions were used to develop a model that better explained the variance in the outcome variable caused by the predictor variables (Woltman et al., 2012). Hypotheses: 1. The difference in annual credentials is related to an institution’s productivity index score. 2. The percent of Pell receiving students at an institution is related to an institution’s productivity index score, after controlling for the difference in annual credentials. 59 3. The percent of non-traditional age students at an institution is related to an institution’s productivity index score, after controlling for the difference in annual credentials. 4. The percent of minority students at an institution is related to an institution’s productivity index score, after controlling for the difference in annual credentials. 5. The percent of Pell receiving students moderates the relationship between credentials and productivity index scores. 6. The percent of non-traditional students receiving students moderates the relationship between credentials and productivity index scores. 7. The percent of minority students moderates the relationship between credentials and productivity index scores. Conditions: a) There is systemic within-and-between group variance in productivity index scores. b) There is significant variance at the level-1 intercept. c) There is significant variance in the level-1 slope. d) The variance in the level-1 intercept is predicted by the three-included independent variables related to student characteristics. e) The variance in the level-1 slope is predicted by the three-included independent variables related to student characteristics. Data Analysis This researcher used quantitative methods consistent with the postpositivist worldview to analyze the study’s data. As a postpositivist study, the researcher tested data to identify the truthfulness of the study’s hypotheses seeking to identify the relationship between an 60 institution’s productivity index score and the difference-in-averages for awarded credentials; and if that relationship is moderated by the included student characteristics of interest. The goal of this study was to indicate that specific student characteristics impacted the aforementioned relationship, and that funding loses at Arkansas community colleges may be due to discreet biases within the OBPF policy. This study used a statistical framework known as a hierarchical linear model to observe the effect of student characteristics on the relationship between funding outcomes and credentials awarded (Kwok et al., 2009). The study’s HLM produced several models that were compared to determine a best-fit model for explaining variances in productivity index scores. HLM has six assumptions, three of which address the error structure, and the remaining three address the predictor variables (Anderson, 2012). The first assumption regarding the error structure is that the residuals from level 1 must have a mean of zero, a common variance, and must be independent and normally distributed; the second assumption is that the random effects of higher levels within the model must have a mean of zero, a common variance, and must be independent and multivariate normally distributed; the third assumption is that there is no covariance between residuals at different levels of the model (Anderson, 2012). The fourth assumption is related to the predictor variables and assumes that level 1 residuals are independent of level 1 predictors; the fifth assumption is that the predictors in higher levels are independent from the residuals within that level of the model; and the six assumption is that predictors are independent from random effects in lower levels (Anderson, 2012). The outcome variable for this study was observed over three annual time-points and was hypothesized to be controlled primarily by a time-varying covariate, and therefore the study required a regression analysis that could address longitudinal data. HLM as a form of growth 61 curve modeling allows researchers to “estimate between-person differences in within-person change”, by controlling for time in a lower level of the analysis (Anderson, 2012; Curran et al., 2010, p. 2). The data for this study is nested among 22 community colleges and grouped into three time-points. A simple linear regression would not account for this grouping and could create potential heterogeneity issues that may ignore the severity of a variables impact on the hypothesized relationship, or over inflate the significance of the model by attributing all variance in the outcome to the between-subjects term (Anderson, 2012). Essentially, the simple linear regression would not account for the data’s inherent groupings and could lead to type-1 or type-2 error if not properly fit to the regression model. The study’s HLM accounted for this grouping by entering data into two-levels, with level one measuring the within-subjects’ effects and level two measuring the between-subjects’ effects (Anderson, 2012; Curran, et al., 2010; Kwok et al., 2009). This study’s HLM procedure produced three sets of parameters, including, the fixed effects, the random level-1 coefficients, and the variance-covariance components (Woltman et al., 2012). (1) Fixed Effects: The model’s fixed effects do not vary across institutions and provide an overall mean intercept adjusted for each level 2 predictor, as well as the regression coefficient associated with the level 2 predictor relative to level 1 intercepts and slopes (Woltman et al., 2012). Fixed effects for this study include TIME, CREDS, AVG_PELL, AVG_AGE, and AVG_MIN. (2) Random Level 1 coefficients: The model’s random level 1 coefficients, unlike the model’s fixed effects, are allowed to vary across institutions. HLM produces two estimates for the model’s random coefficients, a level 1 regression for institutions and the predicted values for the intercept and slope for ith level 2 unit (Woltman et al., 2012). 62 (3) Variance-Covariance Components: The variance-covariance components for this model consist of three parts, including the variance for level 1 error terms, the variance in level 2 error terms, and the covariance of level 2 errors. The study used IBM’s Statistical Package for the Social Sciences (SPSS) v26 linear mixed model command (MIXED) and began by producing a fully null model to determine the intraclass correlation coefficient (ICC). In linear mixed models, the ICC represents the percentage of variance that remains unexplained by the data’s grouping structure at level 1 (Kwok et al., 2009). The ICC score ranges from 0 to 1, and a very low score could indicate that it may be possible to proceed with other statistical methods such as the analysis of variance or the ordinal least squares regression. A large ICC (>.30) indicates more dependence in the data, thus requiring a multi-level mixed effect model described in this study’s proposed hierarchical model (Woltman et al., 2012). Once a null model had been produced, output from subsequent conditional models were compared to determine if the additional predictor variables attributed more or less unexplained variance to productivity funding index scores (Anderson, 2012). The HLM procedure continued by producing an unconditional growth model which included a time-varying covariant for TIME. The TIME variable, coded as (0), (1), or (2) for fiscal years 20, 21, or 22 respectively, helped to indicate the linear slope for productivity index scores when the mean value for TIME was zero (McCoach & Kaniskan, 2010). This approach allowed the output to be interpreted based on linear growth from year one through year three. The unconditional model served as a baseline for subsequent conditional models that included additional predictor variables, each of which sought to reduce the amount of unexplained variance in the outcome variable (Kwok et al., 2009). 63 The study’s first conditional model included a time-varying covariant to control for the annual difference in credentials awarded, which is theorized to act as the primary predictor to productivity index scores. Subsequent models will sequentially add predictor variables in order to reduce deviance scores and further reduce unexplained variances on the dependent variable attributed to the model’s level 2. Hierarchical Linear Model The study began with the following unconditional growth model, which indicated a baseline for additional explanatory variables. PROINDXit = π0i + π1i (TIME)it + eit π0i = β00 + r0i π1i = β10 After determining the fit for the unconditional growth model, the study sequentially added the following additional variables into the model’s level 2: PROINDXit = π0i + π1i (TIME)it + π2i (CREDS)+ eit π0i = β00 + β01 (AVG_PELL) + β02 (AVG_AGE) + β03 (AVG_MIN) + r0i π1i = β10 + β11 (AVG_PELL) + β12 (AVG_AGE) + β13 (AVG_MIN) π2i = β20 + β21 (AVG_PELL) + β22 (AVG_AGE) + β23 (AVG_MIN) Assessment of Model Fit The study’s HLM sought to reduce the unexplained variance in the outcome variable by adding a control variable for credentials and several student characteristic predictor variables, which were hypothesized to reduce the previous model’s deviance scores (McCoach & Kaniskan, 2010; Kwok et al., 2009). The hypotheses for this study were tested using a chi-square difference test to compare each conditional model to the unconditional model in order to determine if the random coefficients from level 1, as well as if the model’s fixed effects and 64 variances, are significantly different from zero (McCoach & Kaniskan, 2010; Kwok et al., 2009). Each subsequent conditional model’s predictive power was measured using an R-squared type statistic known as a pseudo R2. It is common for researchers to report the R2 of a regression model to address the goodness-of-fit for models with only fixed effects (Nakagawa & Schielzeth, 2013). In models with both fixed and random effects, such as this study’s HLM, R2 is more difficult to interpret and more prone to errors when interpreting estimates of effects. Researchers using mixed-effects models often employ an alternative method to produce a pseudo-R2 (Nakagawa & Schielzeth, 2013); but often vary on which specific formula they use to produce a pseudo R2 (e.g., Bryk & Raudenbush, Snijders & Bosker, Wu, etc.) (as cited in Nakagawa & Schielzeth, 2013). Nakagawa and Schielzeth systematically address the limitations to using several of the most cited methods for producing pseudo R2 (2013). In their work, Nakagawa and Schielzeth suggest that the Snijders and Bosker’s (1994) method is often used due to its ability to explain variance by both level 1 and 2 of the mixed-effects analyses; but that the method should limited to only linear mixed-effects models using maximum likelihood (ML) estimates and having no more than two-levels (2013). This study’s HLM abides by the two conditions mentioned by Nakagawa and Schielzeth, and the research questions would be better informed by the ability to explain additional variance at both level 1 and level 2 of the model. Therefore, this study utilized the Snijder and Bosker (1994) formula for partitioned variances found below, to determine the goodness-of-fit for all models having predictor variables. Level 1 Pseudo-R2 = 1 − ?̂?2(𝑓𝑢𝑙𝑙)+?̂?02(𝑓𝑢𝑙𝑙)?̂?2 (𝑛𝑢𝑙𝑙)+?̂?02(𝑛𝑢𝑙𝑙) Level 2 Pseudo-R2 = 1 − (?̂?2(𝑓𝑢𝑙𝑙)/𝑛)+?̂?02(𝑓𝑢𝑙𝑙)(?̂?2 (𝑛𝑢𝑙𝑙)/𝑛)+?̂?02(𝑛𝑢𝑙𝑙) 65 Like most other linear model building procedures, this study observed the deviance metric (-2LL), Akaike information criterion (AIC), the corrected Akaike information criterion (AICC) for smaller sample size, and the Bayesian information criterion (BIC) which is often cited as a more restrictive criteria for assessing model-fit (McCoach & Kaniskan, 2010). The researcher theorized that these information criteria scores would drop as a result of improved unexplained variance in the outcome variable due to the addition of subsequent predictor variables. Chapter Summary Chapter 3 explained the use of quantitative research methodologies to explore the relationship between funding, credentials, and student characteristics at Arkansas community colleges. This chapter justified the use of quantitative methodologies using the postpositivistic worldview. The researcher commented on the use of critical theory when identifying the research problem, and the potential for political transformation if the analysis indicates the presence of distributional inequity. This chapter explained the use of a hierarchical linear model to analyze longitudinal outcome data that included both time-varying and time-invariant covariates. Chapter 3 described the study’s data collection, as well as the methodology used to create the study’s proposed variables. The researcher provided an overview of the population and sample and indicated why this study’s specific sampling frame was used. Finally, the chapter explained the methods used for determining the model’s fit and predictive power. 66 Chapter 4 Results The purpose of this study was to determine if specific student enrollment characteristics of an institution are related to productivity index scores after controlling for the growth in awarded credentials. This study also sought to determine if those specific student characteristics of the institution moderated the relationship between productivity index scores and the institution’s growth in awarded credentials (Curran et al., 2010; Woltman et al., 2012). This study analyzed productivity outcome data, the difference in annual awarded credentials, and student characteristic data for all 22 community colleges in the state of Arkansas over a three-year period using statistical methodologies supported by the postpositivist worldview. The study’s linear mixed model procedure included two levels, the within-subjects level one with two time-varying covariates and the between-subjects level two that included the time-invariant predictor variables. This study’s linear mixed model sought to answer four research questions: 1) What is the relationship between an institution’s productivity index score and the total credentials earned; 2) Is there a relationship between the percent of low-income students and productivity index scores after controlling for credentials, and if so, does the percent of low-income students moderate the relationship between an institution’s productivity index score and the total credentials earned; 3) Is there a relationship between the percent of non-traditional age students and productivity index scores after controlling for credentials, and if so, does the percent of non-traditional age students moderate the relationship between an institution’s productivity index score and the total credentials earned; 4) Is there a relationship between the percent of minority students and productivity index scores after controlling for credentials, and if so, does the percent of minority 67 students moderate the relationship between an institution’s productivity index score and the total credentials earned? Preliminary Analyses and Descriptive Statistics The analysis for this study began by examining the data using SPSS v26 Descriptive Statistics command. The descriptive analysis of the data included each variable’s sample size, range, minimum value, maximum value, mean, standard error, standard deviation, variance, as well as their skewness and kurtosis values. As discussed previously in Chapter 3, data for this study is longitudinal, and therefore descriptive analysis should account for this grouping for institutions and the variation in scores over time. To better understand the way the model’s level one data is grouped, spaghetti plots, scatterplots, and frequency histograms were produced that display data points for each institution at each specific time-point. Descriptive statistics were produced for each year’s observation of the outcome variable (PFINDX) and the time-varying covariant for credentials (CREDS). Frequency polygons and spaghetti plots of the study’s data are found in Figures 1-6, below. Table 4 provides descriptive statistics for each of the continuous variables found within this study. Table 4 Descriptive Statistics for the Study’s Continuous Variables Skewness Kurtosis N Range Min. Max Mean SE SD Var. Stat SE Stat SE PFINX_Y3 22 19 -8 11 2.17 1.12 5.28 27.85 -0.38 0.49 -0.65 0.95 PFINX_Y2 22 24 -14 10 -1.25 1.15 5.38 28.96 -0.11 0.49 0.57 0.95 PFINX_Y1 22 17 -9 9 -1.27 0.95 4.46 19.86 0.61 0.49 -0.002 0.95 68 Table 4 (Continued) Skewness Kurtosis N Range Min. Max Mean SE SD Var. Stat SE Stat SE CREDS_Y3 22 619 -400 219 14.6 23.8 112 1252 -2.33 0.49 9.27 0.95 CREDS_Y2 22 582 -458 124 -13.1 23.3 109 1198 -3.32 0.49 14.11 0.95 CREDS_Y1 22 278 -188 90 -8.77 12.5 58.8 3454 -1.30 0.49 3.36 0.95 AVG_PELL 22 34.7 15.5 50.3 33.5 1.23 9.99 99.8 -0.11 0.29 -1.34 0.58 AVG_AGE 22 34.9 18.7 53.6 35.8 0.945 7.67 58.9 0.21 0.29 0.52 0.58 AVG_MIN 22 58.4 2.7 61.1 25.5 2.24 18.2 330.8 0.41 0.29 -1.05 0.58 PFINX variable. This study’s outcome variable PFINX was observed over three years with evenly spaced annual measurements. Year 1 (PFINX_Y1) has a mean score of -1.27 with a standard deviation of 4.457, and scores ranged from 9 to -9, for a total of 17 points. Year 2 (PFINX_Y2) has a mean score of -1.25 with a standard deviation of 5.381, and scores ranged from 10 to -14, for a total of 24 points. Year 3 (PFINX_Y3) has a mean score of -2.17 with a standard deviation of 5.278, and scores ranged from 11 to -8, for a total of 19 points. Year 1 productivity index scores had a skewness score of .61 (SE - .49) and a kurtosis score of -.002 (SE = .95), indicating that scores for year 1 tended to be slightly below the mean and were slightly to the left of center. Year 2 had a skewness score of -.11 (SE = .49) and a kurtosis score of .57 (SE = .95), suggesting that scores for that year were slightly above the mean, and scores tended to be slightly to the right of center. Year 3 had a skewness score of -.38 (SE - .49) and a kurtosis score of -.65 (SE = .95), indicating that year 3 scores tended to be higher than the mean, which was likewise higher than previous years (year 3 = 2.17, year 2 = - 69 1.25, year 3 = -1.27). Year three scores also tended to shift more to the right of center. A visual inspection confirms that the data is within a normal distribution, as indicated by Figure 1. Figure 1. Frequency polygon of the intensity for annual productivity scores (PROINX) visually representing the range, skewness, and kurtosis of the outcome variable. Figure 1 also indicates a slight annual improvement for the overall group mean for institutions. However, as has been mentioned previously, the study’s repeated measure causes grouping for time with institutions nested into each repeated measure; and therefore, each institution will receive their own intercept observed over the three time-points. Figure 2 suggests that the few institutions with an overall positive trend had made significant gains in the funding model, while most other institutions trended slightly-to-significantly lower. These preliminary results indicate that participant grouping at level 1 should be addressed using a random intercept for each institution (INST_ID) accounting for the individual (subject) variance. In addition, time may also be influencing the regression and will likewise be examined within the model using a time-varying covariant, coded 0, 1, and 2. Graphically, the individual institutional slopes for productivity index scores by year are represented by the spaghetti plot in Figure 2. 70 Figure 2. Institutional Linear Slopes for Performance Funding Scores. CREDS variable. CREDS for year 1 (CREDS_Y1) has a mean score of -8.77 with a standard deviation of 58.8, and scores ranged from 90 to -188, for a total of 278 points. Year 2 (CREDS_Y2) has a mean score of -13 with a standard deviation of 109.4, and scores ranged from 124 to -458, for a total of 582 points. Year 3 (CREDS_Y3) has a mean score of 14.6 with a standard deviation of 111.9, and scores ranged from 219 to -400, for a total of 619 points. CREDS for year 1 had a skewness score of -1.307 (SE - .49) and a kurtosis score of 3.36 (SE = .95). CREDS for year 2 had a skewness score of -3.317 (SE = .49) and a kurtosis score of 14.11 (SE = .95). CREDS for year 3 had a skewness score of -2.328 (SE - .49) and a kurtosis score of 9.27 (SE = .95). These values indicate that credentials for year 1 tended to be slightly above the mean and were more frequently on right of center. Year 2 scores likewise tended to be slightly above the mean, and scores were slightly more to the right of center than in year 1. Year 3 scores also tended to have a higher frequency above the mean, which was likewise 71 considerably higher than previous years (year 3 = 14.6, year 2 = -13.1, year 3 = -8.77). The distribution of CREDS by TIME is represented in a frequency polygon in Figure 3. Figure 3. Visual Inspection of the time-varying covariant (CREDS) by the index variable (TIME). AVG_PELL variable. The time invariant covariant (AVG_PELL) has a mean score of 33.5 with a standard deviation of 9.99, and scores ranged from 50.3 to 15.5, for a total of 34.7 points. AVG_PELL has a skewness score of -0.11 (SE - .29) and a kurtosis score of -1.34 (SE = .58). These scores indicate that scores tend to be slightly greater than the mean and have a higher frequency to the left of center. Graphically, the distribution of low-income values for institutions within this study are represented with a frequency polygon in Figure 4. 72 Figure 4. Frequency polygon for low-income students (AVG_PELL). AVG_AGE variable. The time invariant covariant (AVG_AGE) has a mean score of 35.8 with a standard deviation of 7.67, and scores ranged from 53.6 to 18.7, for a total of 34.9 points. AVG_AGE has a skewness score of .21 (SE - .29) and a kurtosis score of 0.52 (SE = .58). These statistics indicate that the percent of non-traditional age students tends to be slightly positive skew, indicating more values above the mean of 35.8, and a slightly higher frequency to the right of center. Graphically, the distribution of non-traditional age values for institutions within this study are represented with a frequency polygon in Figure 5. 73 Figure 5. Frequency polygon of the non-traditional age variable (AVG_AGE). AVG_MIN variable. The time invariant covariant, AVG_MIN, represents the average percentage of minority student enrolled at an institution. Consistent with the previous two time-invariant covariates, AVG_MIN is a static number within the dataset and is one of three variables of interest for predicting productivity scores. AVG_MIN has a mean score of 25.5 with a standard deviation of 18.2, and scores ranged from 61.1 to 2.7, for a total of 58.4 points. AVG_MIN has a skewness score of .41 (SE - .29) and a kurtosis score of -1.05 (SE = .58). These statistics indicate that the percentage of minority enrolled students tends to be slightly greater than the mean, with a higher frequency to the left of center. Graphically, the distribution for the percentage of minority students for institutions included within this study are represented with a frequency polygon in Figure 6. 74 Figure 6. Frequency polygon for the minority student’s variable (AVG_MIN). Mixed Linear Modeling Results Results from the model building procedure were analyzed based on the conditions prescribed in Chapter 3, which include: condition (a), that there be systemic within-& between-group variance in the outcome variable, which was explored using the intraclass correlation coefficient produced by the initial null model (model-ICC); condition (b), that there be significant variance at the level-1 intercept, was confirmed by including TIME in an unconditional growth model (model-UNCG); condition (c), that there be significant variance at the level-1 slope, was addressed using results from first conditional model controlling for credentials (model-1); and conditions (d) and (e), that the variance in the level-1 intercept and slope are predicted by the three primary predictor variables of interest relating to student characteristics (models 2-4). Results for the model’s estimates are found in Table 5 and model output used to assess goodness-of-fit and variance estimates are found in Table 6. 75 Table 5 Model Estimates Table ICC UNCG 1 2a 2b 2c 3a 3b 3c 4 Intercept Estimate -0.12 -1.83 -1.39 3.05 2.25 -2.21 2.21 2.94 2.27 2.75 Std. Error 0.88 1.02 0.80 2.18 3.35 1.20 3.30 3.17 2.42 2.20 T-Stat -0.13 -1.79 -1.73 1.40 0.67 -1.84 0.67 0.93 0.94 1.25 Sig. 0.90 0.08 0.09 0.17 0.51 0.08 0.51 0.36 0.36 0.22 Time Estimate 1.72 1.35 1.39 1.39 1.34 1.38 1.39 1.38 1.34 Std. Error 0.52 0.48 0.48 0.48 0.48 0.48 0.48 0.48 0.49 T-Stat 3.30 2.82 2.90 2.89 2.79 2.87 2.89 2.88 2.77 Sig. 0.00 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 CREDS Estimate 0.03 0.03 0.03 0.03 0.03 0.03 0.03 0.04 Std. Error 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.03 T-Stat 5.30 4.84 4.34 5.44 4.59 4.51 4.91 1.73 Sig. 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.09 AVG_PELL Estimate -0.13 -0.13 -0.14 -0.13 -0.13 Std. Error 0.06 0.07 0.07 0.06 0.06 T-Stat -2.18 -1.73 -1.82 -2.08 -2.06 Sig. 0.04 0.10 0.08 0.05 0.05 AVG_AGE Estimate -0.10 0.00 0.00 Std. Error 0.09 0.10 0.10 T-Stat -1.12 0.03 0.05 Sig. 0.27 0.98 0.96 AVG_MIN Estimate 0.03 0.02 0.02 Std. Error 0.04 0.03 0.03 T-Stat 0.91 0.71 0.71 Sig. 0.37 0.49 0.49 (CREDS by AVG_PELL) Estimate 0.00 Std. Error 0.00 T-Stat -0.63 Sig. 0.53 Random residual (𝜎2) 14.87 11.90 9.91 9.90 9.90 9.92 9.89 9.90 9.89 9.96 Wald 4.69 4.69 4.69 4.69 4.69 4.69 4.68 4.69 4.69 4.67 Sig. .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 Random intercept (𝑟00) 12.13 13.12 5.84 4.24 5.37 5.48 4.08 4.24 4.08 3.99 Wald 2.31 2.51 2.05 1.78 1.98 1.99 1.75 1.78 1.75 1.71 Sig. .021 .012 .04 .075 .047 .046 .081 .075 .081 .088 76 Table 6 Assessing Model Goodness-of-Fit Model ICC UNCG 1 2a 2b 2c 3a 3b 3c 4 -2LL 392.7 383.0 361.1 356.8 359.8 360.3 356.3 356.8 356.3 356.4 AIC 398.7 391.0 371.1 368.8 371.8 372.3 372.3 370.8 370.3 370.4 AICC 399.1 391.6 372.1 370.2 373.3 373.7 374.8 372.7 372.2 372.3 BIC 405.3 399.7 382.0 381.9 385.0 385.4 389.8 386.1 385.6 385.7 X2 Sig. - 0.00 0.00 0.04 0.27 0.37 0.78 0.96 0.48 0.10 t-stat - 9.74 21.90 4.29 1.22 0.81 4.79 4.30 4.79 4.68 df - 1 1 1 1 1 3 2 2 2 ICC 0.45 0.52 0.37 0.30 0.35 0.36 0.29 0.30 0.29 0.29 R2 Level 1 - 0.07 0.37 0.43 0.39 0.38 0.44 0.43 0.44 0.44 Level 2 - -0.07 0.54 0.66 0.57 0.57 0.67 0.66 0.67 0.67 Null Model: The study’s null model (model-ICC) contains only the outcome variable along with the random intercept for individuals and was produced using SPSS’ v26 Linear Mixed Models. The initial null model can be expressed as: PROINDXij = β0j + eij β0j = γ00 + u0j Estimates for this and subsequent models are found within Table 5. The null model estimates the mean productivity index score to be -0.12 (β00 = -0.12, SE = .88, p = .9). Model-ICC confirms that there is significant variance at both the within-groups (𝜎2 = 14.87, Wald = 4.69, p < .000) and between-group (𝑟00 = 12.13, Wald = 2.3, p = .02) levels. Hierarchical liner models calculate the ICC using between-groups 𝑟00 and within-group 𝜎2 variances generated by the model’s output (Woltman et al., 2012). The first model’s ICC equation can be expressed as: 77 ICC = 𝑟00𝑟00+𝜎2 or ( 12.1312.13+14.87 ) = .45 The null model’s ICC of (.45) suggesting that 45% of variation in productivity index scores is left unexplained by the level 1 intercept. The large ICC score along with the statistical significance found for both the within-and-between group variances affirm condition a, and thus the appropriateness of using hierarchical linear modeling in subsequent analyses. Unconditional Growth Model: The next step in the HLM procedure was to produce an unconditional growth model that accounts for TIME as a within-subjects predictor at level 1. The unconditional growth model (model-UNCG) is hypothesized to improve additional unaccounted-for variance in the outcome variable attributed to the model’s level 1. Model-UNCG can be expressed as: PROINDXit = π0i + π1i (TIME)it + eit π0i = β00 + r0i π1i = β10 Results for model-UNCG found in Table 5 estimate the mean productivity index score for all institutions at year 1 to be -.183 (β00 = -1.83, SE = 1.02, p = .08). The unconditional growth model suggests that productivity index scores increase an average of 1.7 points a year (β10 = 1.72, SE = .52, p = .002). Their remains a significant variance at both the within-groups (𝜎2 = 11.90, Wald = 4.69, p < .000) and between-groups (𝑟00 = 13.12, Wald = 2.59, p = .01) levels. Model-UNCG produced the following ICC: ICC = 13.1213.12 + 11.90 = .52 The unconditional growth model’s ICC of (.52) suggests that the inclusion of the time-varying covariant for TIME attributed an additional 7 percentage points of variance to the within-subjects level. If left unaccounted for could mislead interpretations of the model’s results. Model-UNCG’s addition of the TIME variable will allow for a more accurate assessment of the 78 level 2 predictor’s effect on the outcome variable. Subsequent models will attempt to reduce the ICC score by including a control and several predictor variables. The deviance scores and degrees of freedom from Table 6 compared the unconditional model with those in the null model using an approximate chi-square analysis which confirmed that that adding TIME as a fixed factor significantly improved the model’s fit (χ2 = 9.74, df = 1, p <.000). Conditional Model 1: The first conditional model addressed this study’s first hypothesis which proposed that the difference in annual credentials is related to an institution’s productivity index score. Conditional model-1 adds the time-varying covariant CREDS as a level 1 predictor. It was hypothesized that the CREDS variable would act as the most statistically significant predictor variable due to the explicit language used within the authorizing policy that directs the Arkansas Division of Higher Education to annually produce an outcome-based performance funding score for each public postsecondary institution within the state. The procedure’s first conditional model, also known as model-1, addressed the study’s first research question which sought to confirm that there is a statistically significant relationship between an institution’s productivity index score and the total credentials earned. Model-1 can be expressed as: PROINDXit = π0i + π1i (TIME)it + π2i (CREDS)+ eit π0i = β00 + r0i π1i = β10 π2i = β20 Results for model-1 found in Table 5 estimates a mean productivity index score of -.139 (β00 = -1.39, SE = 0.80, p = .09) when both TIME and CREDS are equal to zero. The first conditional model suggests that productivity index scores increase an average of 1.3 points per year (β10 = 1.35, SE = .48, p = .001). Model-1 also identifies a statistically significant positive relationship between CREDS and PFINX, estimating that for every one-unit increase in credentials, productivity index scores grew by 0.03 points (β20 = 0.03, SE = .01, p < .000). 79 Model-1 also contained significant variance at both the within-groups (𝜎2 = 9.91, Wald = 4.68, p < .000) and between-groups (𝑟00 = 5.84, Wald = 2.84, p = .04) levels. Using the same ICC equation as in previous model, the ICC for model-1 can be expressed as: ICC = 5.845.84 + 9.91 = .37 Results addressing the model’s goodness-of-fit and variance estimates are found in Table 6 and suggest that adding a control variable for CREDS significantly improved the second conditional model’s ICC of 37%, a 15-percentage point decrease in unexplained variance at the between-groups level. Output from model-1 was used to produce a Pseudo-R2 using Snijders and Bosker’s (1994) partitioned variance formula, expressed by: Level 1 Pseudo-R2 = 1 − (9.91+5.8411.9 +13.12) = .37 Level 2 Pseudo-R2 = 1 − ((9.91/22)+5.84(11.9/22)+13.12) = .54 These scores indicate the remaining unexplained variance in the outcome variable is partitioned between the within-groups variance of 37% in the model’s level 1 and the between-groups variance of 54% at the model’s level 2. Lastly, model-1 was assessed using a likelihood ratio test with maximum likelihood estimates, which found the model with CREDS to be a significantly improved model (χ2 = 21.9, df = 1, p <.000). These findings confirm that the control variable CREDS significantly improved the previous model containing only TIME and the random intercept for participants. Conditional Model 2a: Model-2a addresses the study’s second hypothesis which proposes that the percent of Pell receiving students at an institution is related to an institution’s productivity index score after controlling for the difference in annual credentials. The model including AVG_PELL (Model-2a) builds upon previous models by adding the first student 80 characteristic predictor variable and is theorized to further reduce the unexplained variance for productivity index scores attributed to the model’s level 2. Model-2a can be expressed as: PROINDXit = π0i + π1i (TIME)it + π2i (CREDS)+ eit π0i = β00 + β01 (AVG_PELL) + r0i π1i = β10 + β11 (AVG_PELL) π2i = β20 + β21 (AVG_PELL) Results for model-2a are found in Table 5 and estimate the mean productivity index score is 3.05 (β00 = 3.05, SE = 2.18, p = .17). Model-2a estimates that productivity index scores improve an average of 1.4 points per year (β10 = 1.39, SE = .48, p = .01). Model-2a identified a positive relationship between CREDS and PFINX, estimating that for every one-unit increase in CREDS, productivity index scores improved by an average of 0.03 points (β20 = 0.03, SE = .01, p < .000). Model-2a identified a statistically significant negative relationship between AVG_PELL and PFINX, estimating that a one-unit increase in AVG_PELL was associated with an average 0.13-point decline in productivity index scores (β01 = -0.13, SE = .06, p = .04). Although this model continues to suggest significant within-groups variance (𝜎2 = 9.89, Wald = 4.69, p < .000), model-2a no longer identifies the significant between-groups variance (𝑟00 = 4.23, Wald = 1.72, p = .075) present in previous models. Using the same ICC equation as in previous models, the ICC for model-2a can be expressed as: ICC = 4.234.23 + 9.89 = .30 Model results comparing goodness-of-fit and variance estimates are found in Table 6 and indicate that the predictor variable, AVG_PELL, improved the between-group unexplained variance by 7 percentage points, from 37% in model-1 to 30% in model-2a. Output from model-2a was used to produce a Pseudo-R2 using Snijders and Bosker’s (1994) partitioned variance formula, expressed by: 81 Level 1 Pseudo-R2 = 1 − (9.89+4.2311.9 +13.12) = .43 Level 2 Pseudo-R2 = 1 − ((9.89/22)+4.23(11.9/22)+13.12) = .66 These scores indicate the remaining unexplained variance in the outcome variable is partitioned 43% within-groups and 66% between-groups. Lastly, Model-2a was assessed using a likelihood ratio test with maximum likelihood estimates, which found the model with AVG_PELL to be a significantly improvement (χ2 = 4.29, df = 1, p <.04) from model-1. These findings confirm that the predictor variable AVG_PELL significantly improved the previous model containing only TIME and the random intercept for participants. Conditional Model 2b: Model-2b addressed the study’s third hypothesis which proposed that the percent of non-traditional age students at an institution is related to an institution’s productivity index score after controlling for the difference in annual credentials. Model-2b includes the second predictor variable of interest relating to the percentage of non-traditional age students at an institution. This model includes a time-varying covariant for both TIME and CREDS, the time-invariant covariant AVG_AGE, and the random participant intercept representing institutions in the model’s level 1. Model-2b is theorized to similarly reduce the variance in productivity index scores attributed to the model’s level 2 as was demonstrated by model-2a in the results presented in Table 5. Model-2b can be expressed as: PROINDXit = π0i + π1i (TIME)it + π2i (CREDS)+ eit π0i = β00 + β01 (AVG_AGE) + r0i π1i = β10 + β11 (AVG_AGE) π2i = β20 + β21 (AVG_AGE) Results in Table 5 show that model-2b estimates that when all predictor variables are equal to zero, the mean productivity index score across all institutions is 2.25 (β00 = 2.25, SE = 3.35, p = .51). Model-2b also estimates that productivity index scores improve an average of 1.4 82 points per year (β10 = 1.38, SE = .48, p = .006). Model-2b continues to identify a statistically significant positive relationship between CREDS and PFINX, estimating that a one-unit increase in CREDS was associated with an average 0.03-point improvement in productivity index scores (β20 = 0.03, SE = .01, p < .000). Model-2b identified a non-significant negative relationship between AVG_AGE and PFINX, estimating that a one-unit increase in the percentage of non-traditional age students was associated with a 0.10-point decline in productivity index scores (β01 = -0.10, SE = .09, p = .273). This model continues to show significant within-groups variance (𝜎2 = 9.89, Wald = 4.69, p < .000), and once again identifies a significant between-groups variance (𝑟00 = 5.37, Wald = 1.94, p = .047) that had been present in all but one previous model. Model-2b’s ICC is expressed by: ICC = 5.375.37 + 9.89 = .35 The predictor variable AVE_AGE improved the between-group unexplained variance by 2 percentage points, from 37% in model-1 to 35% in model-2b. Output from model-2b was used to produce a Pseudo-R2 using Snijders and Bosker’s (1994) partitioned variance formula, expressed by: Level 1 Pseudo-R2 = 1 − (9.89+5.3711.9 +13.12) = .39 Level 2 Pseudo-R2 = 1 − ((9.89/22)+5.37(11.9/22)+13.12) = .57 These scores indicate the remaining unexplained variance in the outcome variable is partitioned 39% within-groups and 57% between-groups. Lastly, Model-2b was assessed using a likelihood ratio test with maximum likelihood estimates, which found the model with AVG_AGE did not significantly improve model-1 (χ2 = 1.22, df = 1, p = .27). Conditional Model 2c: Model-2c addresses the study’s fourth hypothesis which proposes that the percent of minority students at an institution is related to an institution’s 83 productivity index score after controlling for the difference in annual credentials. Like models 2a and 2b, model-2c includes the index variable TIME, the blocking variable CREDS, a random intercept for institutions, but replaces the previous two models’ level 2 time-invariant predictor variable with the percentage of minority enrolled students. Like the two immediately preceding models, model-2c was theorized to reduce the percentage of unexplained variance in productivity index scores attributed to the model’s level 2. Model-2c can be expressed as: PROINDXit = π0i + π1i (TIME)it + π2i (CREDS)+ eit π0i = β00 + β01 (AVG_MIN) + r0i π1i = β10 + β11 (AVG_MIN) π2i = β20 + β21 (AVG_MIN) Results from Table 5 indicate that model-2c estimates that when all predictor variables are equal to zero, the mean productivity index score is -2.21 (β00 = -2.21, SE = 1.20, p = .08). Model-2c also estimates that productivity index scores improved an average of 1.34 points per year (β10 = 1.34, SE = .48, p = .01). Model-2c continues to identify a statistically significant positive relationship between CREDS and PFINX, estimating that a one-unit increase in credentials was associated with a 0.03-point increase in productivity index scores (β20 = 0.03, SE = .01, p < .000). Model-2c identified a non-significant positive relationship between AVG_MIN and PFINX, where a one-unit increase in the percentage of minority students enrolled at an institution was associated with a an average 0.03-point increase in productivity index scores (β01 = 0.03, SE = .04, p = .037) in. This model suggests significant variance at both the within-groups (𝜎2 = 9.92, Wald = 4.68, p < .000), and between-groups (𝑟00 = 5.47, Wald = 1.92, p = .046) levels. Using the same ICC equation as in previous models, the ICC for model-2c can be expressed by: ICC = 5.475.47 + 9.92 = .36 84 Adding the predictor variable representing the percentage of minority students enrolled at an institution made a 1-point improvement to the ICC of the first conditional model. Output from model-2c was used to produce a Pseudo-R2 using Snijders and Bosker’s (1994) partitioned variance formula, expressed by: Level 1 Pseudo-R2 = 1 − (9.92+5.4711.9 +13.12) = .38 Level 2 Pseudo-R2 = 1 − ((9.92/22)+5.47(11.9/22)+13.12) = .57 These scores indicate the remaining unexplained variance in the outcome variable is partitioned 38% within-groups and 57% between-groups. Lastly, Model-2c was assessed using a likelihood ratio test with maximum likelihood estimates, which found that including AVG_MIN did not significantly improve the first-conditional model (χ2 = 0.81, df = 1, p = .269). Conditional Model 3a: Having tested each time-invariant predictor variable within their own specific model (2a, 2b, 2c), the research questions guided the procedure to test if model-1 is significantly improved by the inclusion of more than one student characteristic with the model’s level 2. Like model-1, model-3a includes the index variable TIME, the control variable CREDS, as well as the random intercept for institutions, but simultaneously adds all three time-invariant predictor variables (AVG_PELL, AVG_AGE, AVG_MIN). Similar to model’s 2a-c, model-3a was theorized to reduce the unexplained variance in productivity index scores attributed to the model’s level 2, and can be expressed as: PROINDXit = π0i + π1i (TIME)it + π2i (CREDS)+ eit π0i = β00 + β01 (AVG_PELL) + β02 (AVG_AGE) + β03 (AVG_MIN) + r0i π1i = β10 + β11 (AVG_PELL) + β12 (AVG_AGE) + β13 (AVG_MIN) π2i = β20 + β21 (AVG_PELL) + β22 (AVG_AGE) + β23 (AVG_MIN) Results for model-3a are found in Table 5 and estimate that when all predictor variables are equal to zero, the mean productivity index score is 2.21 (β00 = 2.21, SE = 3.30, p = .051) 85 Model-3a also estimates that productivity index scores improved an average of 1.38 points per year (β10 = 1.38, SE = .48, p = .01). Model-3a continues to identify a positive relationship between CREDS and PFINX, estimating that a one-unit increase in credentials was associated with an average 0.03-point increase in productivity index scores (β20 = 0.03, SE = .01, p < .000). Model-3a identifies a non-significant negative relationship between AVG_PELL and PFINX, where a one-unit increase in the percentage of low-income students was associated with an average 0.13-point decline in productivity index scores (β01 = -0.13, SE = .07, p = .097). Model-3a identifies a non-significant positive relationship between AVG_AGE and PFINX, where a one-unit increase in the percentage of non-traditional age students was associated with an average 0.003-point increase in productivity index scores (β02 = 0.003, SE = .10, p = .979). Model-3a identifies a non-significant positive relationship between AVG_MIN and PFINX, where a one-unit increase in the percentage of minority students was associated with a 0.02-point increase to productivity index scores (β03 = 0.02, SE = .03, p = .486). This model suggests significant within-groups variance (𝜎2 = 9.89, Wald = 4.69, p < .000), but no longer identifies a significant between-groups variance (𝑟00 = 4.07, Wald = 1.74, p = .081) that had been present in all but one previous model. Model-3a’s ICC can be expressed as: ICC = 4.074.07 + 9.89 = .29 Results addressing the model’s goodness-of-fit and variance estimates are found in Table 6. Model-3a’s ICC of 29% improved the between-group variance by 8 percentage points over model-1’s 37%. Output from model-3a was used to produce a Pseudo-R2 using Snijders and Bosker’s (1994) partitioned variance formula, expressed by: Level 1 Pseudo-R2 = 1 − (9.89+4.0711.9 +13.12) = .44 86 Level 2 Pseudo-R2 = 1 − ((9.89/22)+4.07(11.9/22)+13.12) = .67 These scores indicate the remaining unexplained variance in the outcome variable is partitioned 44% within-groups and 67% between-groups. Lastly, Model-3a was assessed using a likelihood ratio test with maximum likelihood estimates, which found the model containing all three student characteristic variables did not significantly improve model-1 (χ2 = 4.79, df = 3, p = .18). These findings confirm that the model containing all three student characteristic predictor variables did significantly improve the first conditional model. Conditional Model 3b: Like model-3a, model-3b includes the index variable TIME, the blocking variable CREDS, and the random intercept for institutions, but removes the variable AVG_MIN, leaving AVG_PELL and AVG_AGE. Model-3b is theorized to reduce the unexplained variance in productivity index scores attributed to the model’s level 2, and can be expressed as: PROINDXit = π0i + π1i (TIME)it + π2i (CREDS)+ eit π0i = β00 + β01 (AVG_PELL) + β02 (AVG_AGE) + r0i π1i = β10 + β11 (AVG_PELL) + β12 (AVG_AGE) π2i = β20 + β21 (AVG_PELL) + β22 (AVG_AGE) Results found in Table 5 indicate that model-3b estimates a mean productivity index score of 2.94 (β00 = 2.94, SE = 3.16, p = .361). Model-3b also estimates that productivity index scores improve by an average of 1.38 points per year (β10 = 1.38, SE = .48, p = .006). Model-3b continued to identify a statistically significant positive relationship between CREDS and PFINX, estimating that a one-unit increase in credentials was associated with an average 0.03-point increase in productivity index scores (β20 = 0.03, SE = .01, p < .000). Model-3b identified a non-significant negative relationship between AVG_PELL and PFINX, where a one-unit increase in the percentage of low-income students was associated with a 0.135-point decline in productivity index scores (β01 = -.135, SE = .075, p = .083) in the percentage of low-income students enrolled 87 at an institution. Model-3b identifies a non-significant positive relationship between AVG_AGE and PFINX, where a one-unit increase in the percentage of non-traditional age students was associated with an average 0.004-point increase in productivity index scores (β02 = .004, SE = .104, p = .962). This model suggests significant variance at the within-groups level (𝜎2 = 9.92, Wald = 4.68, p < .000), but does not indicate significant variance at the between-groups level (𝑟00 = 4.24, Wald = 1.78, p = .075). Using the same ICC equation as in previous models, the ICC for model-3b can be expressed by: ICC = 4.244.24 + 9.92 = .36 Results addressing the model’s goodness-of-fit and variance estimates are found in Table 6. Model-3b’s ICC of 36% made a 1-percentage point improvement to the first conditional model’s ICC of 37%. Output from model-3b was used to produce a Pseudo-R2 using Snijders and Bosker’s (1994) partitioned variance formula, expressed by: Level 1 Pseudo-R2 = 1 − (9.92+4.2411.9 +13.12) = .43 Level 2 Pseudo-R2 = 1 − ((9.92/22)+4.24(11.9/22)+13.12) = .66 These scores indicate the remaining unexplained variance in the outcome variable is partitioned 44% within-groups and 66% between-groups. Lastly, Model-3b was assessed using a likelihood ratio test with maximum likelihood estimates, which found that including only the predictors AVG_PELL and AVG_MIN did not significantly improve the first-conditional model (χ2 = 4.30, df = 2, p = .117). Conditional Model 3c: Like models 3a and 3b, model-3c includes the index variable TIME, the blocking variable CREDS, and the random intercept for institutions, but removes the variable AVG_AGE, leaving AVG_PELL and AVG_MIN as level 2 predictors. Likewise, 88 similar to all previous models, model-3c is theorized to reduce the unexplained variance in productivity index scores attributed to the model’s level 2, and can be expressed as: PROINDXit = π0i + π1i (TIME)it + π2i (CREDS)+ eit π0i = β00 + β01 (AVG_PELL) + β02 (AVG_MIN) + r0i π1i = β10 + β11 (AVG_PELL) + β12 (AVG_MIN) π2i = β20 + β21 (AVG_PELL) + β22 (AVG_MIN) Results for model-3c are found in Table 5 and indicate that when all predictor variables are equal to zero, the mean productivity index score is 2.27 (β00 = 2.27, SE = 2.42, p = .357). Model-3c also estimates that productivity index scores improve by an average of 1.34 points per year (β10 = 1.34, SE = .48, p = .006). Model-3c continues to identify a statistically significant positive relationship between CREDS and PFINX, estimating that a one-unit increase credentials was associated with an average 0.03-point increase in productivity index scores (β20 = 0.03, SE = .005, p < .000). Model-3c identified a statistically significant negative relationship between AVG_PELL and PFINX, where a one-unit increase in the percentage of low-income students was associated with an average 0.13-point decline in productivity index scores (β01 = -.127, SE = .06, p = .048). Model-3c identified a non-significant positive relationship between AVG_MIN and PFINX, where a one-unit increase in the percentage of minority students was associated with an average 0.02-point increase in productivity index scores (β02 = .023, SE = .03, p = .485). This model suggests significant variance at the within-groups level (𝜎2 = 9.89, Wald = 4.69, p < .000), but does not show significant variance at the between-groups level (𝑟00 = 4.07, Wald = 1.72, p = .08). Using the same ICC equation as in previous models, the ICC for model-3c can be expressed by: ICC = 4.074.07 + 9.89 = .29 Results addressing the model’s goodness-of-fit and variance estimates are found in Table 6, which suggest that adding the predictor variable representing the percentage of minority students 89 enrolled at an institution improved the ICC by only a single percentage point from that of the first conditional model. Output from model-3c was used to produce a Pseudo-R2 using Snijders and Bosker’s (1994) partitioned variance formula, expressed by: Level 1 Pseudo-R2 = 1 − (9.89+4.0711.9 +13.12) = .44 Level 2 Pseudo-R2 = 1 − ((9.89/22)+4.07(11.9/22)+13.12) = .67 These scores indicate the remaining unexplained variance in the outcome variable is partitioned 44% within-groups and 67% between-groups. Lastly, Model-3c was assessed using a likelihood ratio test with maximum likelihood estimates, which found that including both AVG_PELL and AVG_MIN did not significantly improve the first-conditional model (χ2 = 4.79, df = 2, p = .09). Conditional Model 4: The final conditional model sought to address the research question regarding the degree to which the student characteristic variable moderates the relationship between CREDS and PFINDX. Model-4 was used to identify if there was a statistically significant interaction between the statistically significant predictor variables. Previous models indicated that AVG_PELL was the only statistically significant predictor variable within the three hypothesized student characteristic variables of interest. Therefore, the final model will determine if AVG_PELL moderates the relationship between CREDS and PFINDX, and can be expressed as: PROINDXit = π0i + π1i (TIME)it + π2i (CREDS)+ eit π0i = β00 + β01 (AVG_PELL) + β02 (CREDS*AVG_PELL) + r0i π1i = β10 + β11 (AVG_PELL) + β12 (CREDS*AVG_PELL) π2i = β20 + β21 (AVG_PELL) + β22 (CREDS*AVG_PELL) Results for model-4 are found in Table 5 and indicate that when all predictor variables are equal to zero, the mean productivity index score is 2.75 (β00 = 2.75, SE = 2.19, p = .221). Model-4 also estimates that productivity index scores improved by an average of 1.34 points per 90 year (β10 = 1.34, SE = .48, p = .008). Model-4 continues to identify a non-significant positive relationship between CREDS and PFINX, estimating that a one-unit increase in credentials was associated with an average 0.04-point increase in productivity index scores (β20 = 0.04, SE = .025, p = .089). Model-4 identified a statistically significant negative relationship between AVG_PELL and PFINX, where a one-unit increase in the percentage of low-income students was associated with an average 0.13-point decline in productivity index scores (β01 = -.127, SE = .06, p = .05). Model-4 did not identify a statistically significant interaction between CREDS and AVG_PELL (β02 = -.0004, SE = .0006, p = .530). This model suggests significant variance at the within-groups level (𝜎2 = 9.96, Wald = 4.67, p < .000), but does not show significant variance at the between-groups level (𝑟00 = 3.99, Wald = 1.708, p = .088). Using the same ICC equation as in previous models, the ICC for model-4 can be expressed by: ICC = 3.993.99 + 9.96 = .29 Results found in Table 6 indicate that adding the interaction variable for CREDS-by-AVG_PELL improved the ICC of the first conditional model by 8 percentage points; however, this model includes all variables found in model-2a (.30), with the addition of the interaction variable, and only improved the ICC of model-2a by a single percentage point. Results from model-4 were used to produce a Pseudo-R2 using Snijders and Bosker’s (1994) partitioned variance formula, expressed by: Level 1 Pseudo-R2 = 1 − (9.96+3.9911.9 +13.12) = .44 Level 2 Pseudo-R2 = 1 − ((9.96/22)+3.99(11.9/22)+13.12) = .67 These scores indicate the remaining unexplained variance in the outcome variable is partitioned 44% within-groups and 67% between-groups. Lastly, model-4 was assessed using a 91 likelihood ratio test with maximum likelihood estimates, which found that including both AVG_PELL and the interaction term did not significantly improve the first-conditional model (χ2 = 4.68, df = 2, p = .096). Chapter Summary This chapter provided the results from the hierarchical linear modeling procedure which sought to produce a best-fitting model for explaining variance in productivity index scores. Overall, the results confirmed that of the included variables, credential growth was the strongest predictor to productivity index scores. Of the three hypothesized student characteristic variables, only the low-income variable was identified as a significant predictor. Results presented in Tables 5 and 6 illustrate that model-2a produced a best-fitting model while also retaining statistical significance in the predictor variable. The model containing the average percentage of Pell-receiving students produced the lowest overall AIC, AICC, and BIC score, the second lowest ICC score, and a statistically significant improvement to overall model-fit based on the interpretation of the model’s X2. Model-2a produced a higher pseudo-R2 for both level 1 and level 2 of the model, indicating the true proportion of unexplained variance is approximately 43% at level 1 and 66% at level 2. The study’s findings are limited by the small sample size and relatively few observations of the outcome and control variable. The study’s results indicate that productivity index scores tended to increase by an average of 1.34 points per year, and that credentials have a positive relationship with productivity index scores, with productivity index scores experiencing an average 0.03-point increase per one-unit increase in credentials. Results also indicate that an institution’s percentage of low-income students has a statistically significant negative relationship with productivity index scores, estimating that productivity index scores declined by an average 0.13-points for 92 every one-unit increase in the percentage of low-income students enrolled. These findings are supported by existing language in the Arkansas Division of Higher Education (Productivity Funding, 2020), as well as previous literature pertaining to student success (Kuh et al., 2006). Finally, the analysis rejected any moderating effects that the percentage of low-income students might have on the preidentified relationship between credential growth and productivity index scores. These findings could be amplified or mitigated by the study’s limited sample size, made worse by the only-three waves of the time-varying outcome and control variables. This and other implications for this study are discussed in further length in Chapter 5. 93 Chapter 5 Conclusions Postsecondary education will likely remain one of the nation’s strongest contributors to economic development (Berger & Fisher, 2013; Skolnik et al., 2005; Snyder, 2019). Despite the nation’s collective efforts toward improving postsecondary student outcomes (Blankenberger & Phillips, 2016), the United States continues to trail behind other developed nations with regard to the percentage of adults having obtained some level of postsecondary education (OECD, 2020). Contributing to this decline has been the presence of an achievement gap between students from different academic backgrounds (Hanushek et al., 2020; Postsecondary Attainment, 2015). While the United States has made some progress toward improving postsecondary graduation rates, not all student groups have experienced an equal share of that improvement (Graduation Rates, 2019). Higher-SES students have a postsecondary completion rate of 72%, compared to 35% for students with lower-SES (Postsecondary Attainment, 2015). The economic recession of 2007 further complicated postsecondary success efforts by causing many states to implement drastic budget reductions while also prioritizing credential growth (Umbricht et al., 2017). To achieve these policy goals, legislators began adopting outcomes-base performance policies which sought to improve postsecondary completion by offering funding increases or funding losses based on the successful completion of state outcomes (Hillman et al., 2015; Umbricht et al., 2017). Previous research into OBPF has indicated a need for performance models to include specific equity metrics that target underserved students (Dougherty & Natow, 2015; Li et al., 2018; Ubmricht et al., 2015). Contemporary OBPF models have included these equity metrics in different ways, but despite their agreed-upon importance, little is known of the metrics’ ability to protect against potential unintended biases within the policy. 94 Despite Arkansas’s productivity funding model’s inclusion of additional point-multipliers for traditionally underrepresented students, a majority of Arkansas community colleges have experienced funding losses for two out of the policy’s three years. With a majority of institution’s having lost funding in more than one year, most community colleges in Arkansas are now faced with the problem of producing more credentials with increasingly less state funding. The purpose of this study was to determine if an institution’s student enrollment characteristics impacted productivity index scores and was guided by the following research questions. 1. What is the relationship between an institution’s productivity index score and the total credentials earned? 2. Is there a relationship between the percent of low-income students and productivity index scores after controlling for credentials, and if so, does the percent of low-income students moderate the relationship between an institution’s productivity index score and the total credentials earned? 3. Is there a relationship between the percent of non-traditional age students and productivity index scores after controlling for credentials, and if so, does the percent of non-traditional age students moderate the relationship between an institution’s productivity index score and the total credentials earned? 4. Is there a relationship between the percent of minority students and productivity index scores after controlling for credentials, and if so, does the percent of minority students moderate the relationship between an institution’s productivity index score and the total credentials earned? 95 Summarizing Results by Research Question Research Question 1 The first research question addressed the relationship between an institution’s productivity index score and the total credentials earned. Results for the study’s first conditional model indicate that credentials are a statistically significant predictor to productivity index scores and that the positive relationship between CREDS and PFINX can be interpreted as an average 0.03-point increase in productivity index scores for every one-unit increase in credentials (β20 = 0.03, SE = .01, p < .000). This finding suggests that the state’s funding model is largely driven by credential growth. Research Question 2 The second research question addressed the relationship between the percent of low-income students and productivity index scores after controlling for credentials. Research question two also sought to determine if the percent of low-income students moderated the relationship between an institution’s productivity index score and the total credentials earned. Results from model-2a indicate that after controlling for credentials, the average percent of low-income students was a statistically significant predictor for productivity index scores (β01 = -0.13, SE = .06, p = .04), and that AVG_PELL and PFINX have a statistically significant negative relationship that can be interpreted as an average 0.13-point decline in productivity index scores for every one-unit increase in the percentage of low-income students. This finding suggests that institutional funding is negatively impacted by number of low-income students enrolled at an institution. Results from model-4 explored the moderating effect of AVG_PELL on CREDS and PFINX, which indicated that while AVG_PELL remained a significant predictor to productivity 96 index scores (β01 = -.127, SE = .06, p = .05), AVG_PELL did not moderate the relationship between credentials and productivity index scores (β02 = -.0004, SE = .0006, p = .530), as indicated by the non-statistical significance found in the interaction term. Existing research indicates that low-income students are less likely to graduate (Kuh et al., 2006); and therefore, the researcher had hypothesized that the percentage of low-income students enrolled at an institution would moderate the relationship between credential growth and productivity index scores. This finding suggests that the percentage of low-income students does not impact the relationship between credential growth and productivity index scores, likely indicating that the low-income variable is impacting institutional outcomes in ways other than credential growth. Research Question 3 The study’s third research question sought to determine the relationship between the percent of non-traditional age students and productivity index scores after controlling for credentials. Research question three also sought to identify if the percent of non-traditional age students moderated the relationship between an institution’s productivity index score and the total credentials earned. Results from model-2b indicated that the percentage of non-traditional age students was not a statistically significant predictor to productivity index scores (β01 = -0.10, SE = .09, p = .273). Additionally, model-3b suggested that the presence of both AVG_MIN and AVG_PELL likewise did not produce significance within the outcome variable (β03 = 0.02, SE = .03, p = .486). The study did not proceed to analyze the potential moderating effect on credentials due to the variable’s non-significance. These findings suggest that funding outcomes are not impacted by the number of non-traditional age students enrolled at an institution. 97 Research Question 4 The study’s fourth research question sought to identify a relationship between the percent of minority students and productivity index scores after controlling for credentials. Research question four also sought to determine if the percentage of minority students enrolled moderated the relationship between an institution’s productivity index score and the total credentials earned. Results from model-2c indicated that there is a non-significant positive relationship between the percent of minority students and productivity index scores (β01 = 0.02, SE = .04, p = .037); however, these findings suggest that AVG_MIN is not a statistically significant predictor for productivity index scores. The study did not proceed to analyze the potential moderating effect on credentials due to the variable’s non-significance. These findings suggest that funding outcomes are not impacted by the number of minority students enrolled at an institution. Summarizing Results by Hypotheses The first hypothesis proposed that the largest predictor to productivity index scores was the annual difference in credentials awarded, also known as credential growth. This hypothesis was based on the language and metrics described by the authorizing language (Productivity Funding, 2020), and was theorized to significantly control for variance in the outcome variable. The hypothesis regarding the relationship between credentials and productivity index scores was supported by the research findings. This variable acted as a control variable for all subsequent models based on the variable’s statistical significance. The second through fourth hypotheses proposed that specific student characteristics acted as statistically significant predictors for productivity index scores after controlling for credentials. These hypotheses were founded upon student success literature that identifies low-income, non-traditional age, and minority students as experiencing a lower overall graduation 98 rate than their cohort counterparts (Kuh et al., 2006). Hypothesis two identified low-income students as a significant predictor and was supported by the research findings. Hypothesis three identified non-traditional age students and was not supported by the research findings. Hypothesis four identified minority students and was likewise-not supported by the research findings. The final three hypotheses proposed that each of the three student characteristic predictor variables individually moderated the relationship between productivity index scores and credential growth. Of the study’s three predictor variables relating to student characteristics, only AVG_PELL produced statistical significance, and therefore was the only student characteristic predictor modeled as an interaction term for credentials and productivity index scores. The research findings did not support the hypothesis that low-income students moderated the relationship between PFINX and CREDS. The model building procedure resulted in the production of a best-fit-model, and also allowed the researcher to remove variables that did not significantly impact productivity index scores. Although results from the best-fit model indicate statistical significance within the predictor variables for time, credentials, and low-income students, additional research is likely necessary to completely rule-out a potential relationship between the percentage of minority students or non-traditional age students and productivity index scores. In addition, the study’s restrictive sampling technique targeting only Arkansas community colleges over a specific three-year period led to the study being underpowered (Curran et al., 2010; Kwok et al., 2009). Although results from this study are not generalizable to other states, these findings provide researchers interested in Arkansas, higher education, or performance management, with specific 99 insight into how Arkansas community colleges are impacted by, and are responding to, the state’s OBPF policy. Discussion and Implications Federal, state, and not-for-profit agencies continue to prioritize the production of credentials as a result of the now omnipresent completion agenda (Borden et al., 2019; Kuh et al., 2006; OECD, 2020). Leaning heavily on resource-dependency theory, legislators and agency directors have increasingly adopted outcomes-based funding policies to try and drive degree completion (Fowles, 2014; Hillman et al., 2009; Pfeffer & Salancik, 2002). This completion at all costs has caused many in higher education to warn against the “mass-production of credentials” that may have little or no labor-market value (Bankston, 2011; Li, & Kennedy, 2018). Despite these warnings, short-term credentials are likely to continue to grow at institutions funded by an OBPF policy, a fact that is supported by this study’s research findings. In fact, since the policy’s adoption, Arkansas’s community colleges have experienced an overall increase in awarded credentials, leading to an improved mean productivity index score of almost two-points. Most of that increase, however, can be attributed to the third and most recent year of the funding model, possibly suggesting that institutions are beginning to see the fruits of their prior year’s labors, or alternatively, are learning to optimize their performance under the model by making tremendous gains in areas that require the least effort. In application, this might present itself as a college requiring an imbedded credential within a larger degree, often times awarded without the prior knowledge or consent of the student. Regardless of the presence of short-term wins, the policy has, in at-least one sense, failed to equitably distribute funding to institutions by allowing for some bias against institutions with higher percentages of low-income students. Results from this study indicate that as the 100 percentage of low-income students increases, productivity index scores are estimated to decline, a finding that is supported by prior literature on postsecondary student success (Ewell & Wellman, 2007; Kinzie & Kuh, 2017; Kuh et al., 2006). Existing literature on student success identifies low-income, non-traditional age, and minority students as experiencing the lowest overall graduation rates (Kuh et al., 2006). Although the study failed to produce statistical significance in either the minority or non-traditional age student variables, the results estimate a statistically significant negative relationship between the average percentage of low-income students and productivity index scores, an assuredly troubling finding. The study’s findings suggest that including the low-income variable provided a better fitting model and indicated a reduction in unexplained variance in the outcome variable. This study’s findings support existing student success research that suggests that low-income students typically experience more barriers to postsecondary success than their high-income peers, and as a result experience lower overall rates of postsecondary success (Ewell & Wellman, 2007; Kinzie & Kuh, 2017; Kuh et al., 2006). Results from this study also support prior research into outcomes-based performance funding, specifically with regard to the warnings made by other researchers about the policy’s potential to produce bias for-or-against an institution based on the enrollment characteristics of its students (Cielinski & Pham, 2017; Gandara, 2019; Horn & Lee, 2017; Kelchen, 2019; Li et al., 2018; Umbricht et al., 2017). This study’s findings also support existing literature regarding non-market theory and the potential for distributional inequities in public goods due to efficiency measures (Umbricht et al., 2017; Wolf, 1978). Although the study was underpowered and additional research should be conducted in order to identify distributional inequity within Arkansas’ productivity funding 101 model, this study’s findings do indicate some bias against schools with higher percentages of Pell-receiving students. This study had hypothesized that the inequity was due to differences in credential attainment among student with higher-and lower-SES (Kuh et al., 2006). The study did not find that the percentage of Pell receiving students significantly moderated the relationship between credentials and productivity index scores, and therefore the state’s funding model is likely negatively impacted by the percentage of Pell-receiving students in other, yet unidentified ways. Overall, institutions tended to do better with time, with the majority of successes having been experienced in the third and final observed year of the model. This finding supports existing literature regarding outcomes-based performance funding and resource-dependence theory, which suggests that institutions will adjust their behaviors in order to optimize state funding returns (Pfeffer & Salancik, 2003). This finding also supports existing literature on agency theory, which contends that state legislatures are able to achieve goals through the enlistment of postsecondary institutions acting as agents (Mitnick, 2019). Although, the policy did lead to a slight increase in the number of short-term credentials awarded within Arkansas, not all institutions experienced the same linear growth following the policy’s implementation and at-least some portion of that variance among productivity index scores can be attributed to the socioeconomics of the institution’s students. This study’s findings estimate that a one-point increase to the percentage of low-income students enrolled at an institution corresponded to an average 0.13-point decline in productivity index scores. Although this study finds this impact to be small, these implications certainly indicate that additional research is necessary to determine to what extent institutions are impacted by the loss of funds, and the potential socioeconomic inequity that may be impacting Arkansas community colleges. 102 Results from this study indicate that state funding for Arkansas community colleges is to-some-extent negatively affected by the percentage of low-income students. Although the performance model did lead to funding gains to some community colleges, for two of the model’s three years, the majority of institutions lost funding. Some institutions having previously experienced several years of losses in the funding model must now produce more student credentials with increasingly less funding. Without additional resources, any efforts requiring resources to improve student success must rely on funding that contends with existing operational resources. This study’s findings suggest the need for additional research into the relationship between student enrollment characteristics and funding within Arkansas community colleges. Although troubling, these results alone are insufficient to confirm the presence of distributional inequity within the state’s funding policy. This study’s results are made more troubling however, by the fact that Arkansas’s policy specifically includes specific equity metrics designed to protect against these very outcomes (Cielinski & Pham, 2017; Productivity Model, 2020). It is clear, however, that for at-least the low-income student measure, the funding policy has allowed some bias against institutions with higher-percentages of low-income students. Further analysis is required to completely rule-out discreet bias in the form of distributional inequity (Weber, 2020; Wolf, 1978). This study’s insight into a mere three-years of the policy’s implementation provides only a glimpse of the true impact that student characteristics might have toward an institution’s state funding. Despite the study being underpowered, these findings should serve as the starting point for additional analysis on the relationship between low-income students and productivity index scores in Arkansas. 103 Recommendations for Practice The individuals that would benefit the most from applying and further analyzing these findings are higher education administrators responsible for student success or performance funding outcomes, and higher education policy makers within Arkansas and other states interested in outcomes-based performance funding’s relationship with marginalized student populations. As indicated by this study, there is likely a need to readdress the efficacy of the funding policy’s included equity metrics. In Arkansas, these metrics continue to allow the presence of funding discrepancies between institutions with lower-and-higher socioeconomic demographics, suggesting that metrics should potentially be more impactful. These findings likely also suggest the need for targeted funding for equity programs at institutions with higher percentages of low-income students, specifically targeting retention and completion, a known indicator for degree completion (Cielinski & Pham, 2017; Kuh et al., 2006). It is unlikely that institution’s serving larger percentages of low-income students can maintain academic quality with funding losses in perpetuity. These small, mostly rural institutions will inevitably experience a breaking point. For many students, these small community colleges act as the gatekeeper to a greater journey into higher education, and for others, these institutions serve as a starting point to enter the regional labor force. Students typically attend a community college to either learn a technical trade in order to join the workforce, or to complete a traditional academic program in the hopes of transferring to a four-year university. Although communities with higher populations of low-income individuals stand to gain the most from the presence of a community college, this study’s findings indicate that schools that tended to lose funding had the highest percentages of low-income students. Those 104 individuals able to steer policy should closely consider the benefits of reinvesting into rural community colleges having experienced funding losses. It is likely that institutions with higher percentages of low-income students require more programmatic intervention to change the outcomes of an already marginalized study body. It is, ironically, these very students attending the institutions having experienced multiple funding losses that stand to benefit the most from postsecondary education. Recommendations for Further Research As mentioned throughout this study, the purposive total population sampling restricted the sample size to only twenty-two community colleges in Arkansas. This study was entirely interested in the relationship between student characteristics and the Arkansas productivity funding model, and therefore findings were not designed to be generalized to other states. This study’s research findings support existing research into postsecondary student success regarding low-income student completion rates, however, these findings only begin to reveal the true relationship between low-income students and Arkansas productivity funding outcomes. This study relied on aggregated institutional data to provide insight into general trends within the state. Future research, however, could include student-level outcome data to provide a more complete understanding of how the policy impacts specific student demographics, and to build adequate statistical power within the analysis. At the time of this study, the Arkansas productivity funding model had only three years of fiscal outcomes, and disaggregated student credential data for the final year of the study would not be available from the National Center for Educational Statistics until the following year. Once student-level data becomes available, future research could use a multi-level model that allows for the prediction of both student and institutional level outcomes, which would provide 105 researchers with better insights into the relationship between student socioeconomic status and institutional funding outcomes. Institutions that have experienced losses in several of the funding years should be examined for additional shared behaviors or characteristics that might help identify additional variables to further reduce the unexplained variance in productivity index scores. Qualitative research into institutions with higher percentages of low-income students may yield rich contextual data that could help researchers better understanding the relationship between low-income students and institutional funding losses. Future qualitative research should explicitly focus on the intersection between the student’s experience and the administrative perspective leading to credential growth. Finally, this study should act as a springboard for additional research between outcomes-based performance funding and rural community college funding trends. It is likely that community colleges in other OBPF states are experiencing similar funding losses unique to their own respective funding model. The researcher contends that it is highly unlikely that states using OBPF will adopt a single nationwide framework; yet despite this lack of uniformity, states adopting OBPF policies should carefully observe the policy’s funding outcomes with regards to specific student characteristics. Final Thoughts Despite significant focus on improving higher education graduation rates, the United States continues to experience less postsecondary growth than other industrialized nations (OECD, 2020). The gains toward student completion that the United States has experienced have almost entirely benefitted white, higher-income students (Graduation Rates, 2019). The 2007 economic recession caused states to divest in public higher education, and over a decade later, 106 state higher education appropriations continue to fall below pre-recession dollars (Fowles, 2014; McKeown-Moak, 2013; Mitchell et al., 2016; AACC, 2020). As states divested from higher education, legislatures simultaneously began adopting funding policies that incentivized credential growth and reduced funding for credential declines (Alshehri, 2016). Many critics of outcome-based funding policies warned about the potential for bias due to the known achievement gap between students from varying backgrounds (Cielinski & Pham, 2017; Lorenzo, 2018; Umbricht et al., 2017). In light of these potential unintended outcomes, states began adopting outcomes-based policies that included specific equity metrics to help control for systemic bias. In at-least one OBPF state however, the policy’s targeted equity metrics have continued to allow some colleges to experience annual funding losses that are at-least partially explained by the percentage of low-income students enrolled at the institution (Productivity Funding, 2020). Although this study was underpowered (Curran, 2010), these finding suggest that funding losses are in some-part explained by the percentage of low-income students enrolled at an institution. These students often require additional resources to overcome academic barriers not experienced by their higher-SES counterparts, but as institutions serving more low-income students continue to experience funding losses, the availability of additional programmatic resources necessary to impact low-income student success will increasingly become fewer. Although this study’s findings suggest only short-term and minor funding losses, over time the impact could be devastating to rural community colleges and the low-income communities they serve. The study’s author has been employed within public higher education for almost a decade and typically serves in roles that promote student success through the implementation of data 107 driven digital initiatives. This study was conducted during what has now infamously become known as 2020, and as a result of the COVID-19 global pandemic the author has witnessed higher education rapidly shift a significantly greater portion of its academic product online. Following the implementation of statewide restrictions, the author observed many colleges within Arkansas begin undertaking significant efforts to provide students with the required technologies for online learning; specifically, hotspots or laptops, and oftentimes both. COVID-19 very quickly required many colleges to invest in additional devices and solutions to improve, expand, or give access to online learning. Colleges quickly became focused on retaining the students who remained interested in pursuing a postsecondary education during the global pandemic. Despite significant efforts to promote online learning, as of March 2021 the National Student Clearinghouse Research Center reports that nationwide postsecondary enrollment is down by 2.5% with community colleges experiencing the greatest decline at 13% (Amour, 2020). The enrollment decline alone is troubling, but the significantly higher decline at community colleges is likely signaling an even greater problem related to the tenuous relationship between online learning and low-income students; likely a side effect of the nation’s digital divide between those with and those without access to reliable technology (Lee, 2020). While there will inevitably remain a segment of the population that are steadfast against online learning, following the experiences of 2020 it is safe to assume that a great number of individuals have become far more comfortable with meaningful online engagement. As the nation, and world, continue to experience substantive change as a result of a global pandemic, now more than ever institutions should begin investing in transformative initiatives that target low-income student success (i.e., retention and completion); helping to ensure that existing gaps 108 in achievement are not further exacerbated by an exclusionary digital divide. Some immediate examples might include providing students with required technology, improving the digital architecture of the learning management system, making online student services more intuitive, or reducing the amount of text clutter within the student portal. While improving low-income student success is admittedly no small task; as educators, it is important to remember that although initiatives begun today may take years to produce outcomes, there remains no better time to plant a tree than today. 109 References AACC. (2020). DataPoints: Community college revenues. American Association of Community Colleges. Retrieved from https://www-aacc-nche-edu.ezproxy.memphis.edu/2020/07/06/datapoints-community-college-revenues/ Accreditation in the United States. Institutional accrediting agencies. United States Department of Education. Retrieved from https://www2.ed.gov/admins/finaid/accred/accreditation_pg6.html#RegionalInstitutional ACPA. (2020). The completion agenda: Opportunities for college student educators: ACPA. Retrieved from https://www.myacpa.org/blogs/presidents-desk/completion-agenda-opportunities-college-student-educators ACT 148. (2017). An act to adopt a productivity-based funding model for state supported institutions of higher education. Arkansas State Legislature. Retrieved from https://www.arkleg.state.ar.us/Acts/Document?type=pdf&act=148&ddBienniumSession=2017%2F2017R Alshehri, Y. M. (2016). Performance-based funding: History, origins, outcomes, and obstacles. Journal of Higher Education Theory and Practice, 16(4), 33. Retrieved from https://search-proquest-com.ezproxy.memphis.edu/docview/1860981967 Altbach, P. G., Gumport, P. J., & Berdahl, R. O. (2011). American higher education in the twenty-first century (3rd ed.). Baltimore: Johns Hopkins University Press. Retrieved from https://ebookcentral-proquest-com.ezproxy.memphis.edu/lib/ [SITE_ID]/detail.action?docID=3318779 Amour, M. S. (2020, Dec 17). Few positives in final fall enrollment numbers. Inside Higher Education, Retrieved from https://www.insidehighered.com/news/2020/12/17/final-fall-enrollment-numbers-show-pandemics-full-impact#:~:text=Higher%20education%20lost%20about%20400%2C000,decline%20in%20the%20last%20report. Anderson, D. (2012). Hierarchical linear modeling (HLM): An introduction to key concepts within cross-sectional and growth modeling frameworks. technical report #1308. Behavioral Research and Teaching. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=eric&AN=ED545279&site=eds-live&scope=site Average Pell Award. (2020). Trends in college pricing and student aid. The College Board, Retrieved from https://research.collegeboard.org/pdf/trends-college-pricing-student-aid-2020.pdf Bankston, I., Carl L. (2011). The mass production of credentials: Subsidies and the rise of the higher education industry. Independent Review, 15(3), 325-349. Retrieved 110 from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=bth&AN=57309148&site=eds-live&scope=site Berger, N., & Fisher, P. (2013, Aug 23,). A well-educated workforce is key to state prosperity. Economic Policy Institute Retrieved from https://www.epi.org/publication/states-education-productivity-growth-foundations/ Blankenberger, B., & Phillips, A. (2016). Performance funding in illinois higher education: The roles of politics, budget environment, and individual actors in the process. Educational Policy (Los Altos, Calif.), 30(6), 884-915. doi:10.1177/0895904814556748 Borden, V., Cartney, M., English, S., Jankowski, N., Johnstone, S., Jose, B., & Warmack, D. (2019). Defining student success data recommendations: A compendium of papers from HLC's defining student success data initiative. Chicago, Il: Higher Learning Commission. Retrieved from http://download.hlcommission.org/initiatives/StudentSuccessConversation.pdf Bragg, D. D., & Durham, B. (2012). Perspectives on access and equity in the era of (community) college completion. Community College Review, 40(2), 106. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=edb&AN=75009982&site=eds-live&scope=site Bronner, S. E. (2011). Critical theory: A very short introduction. Oxford University Press. 263. Retrieved from https://books.google.com/books?id=Z7oVDAAAQBAJ&newbks=0&hl=en&source=newbks_fb Chen, X. (2016). Remedial course taking at U.S public 2- and 4-year institutions: Scope, experiences, and outcomes. National Center for Education Statistics. Retrieved from https://nces.ed.gov/pubs2016/2016405.pdf Chokshi, N. (2015, May 13,). The economy’s bouncing back. but higher education funding isn’t. The Washington Post Retrieved from https://www.washingtonpost.com/blogs/govbeat/wp/2015/05/13/the-economys-bouncing-back-higher-education-funding-isnt/ Cielinski, A., & Pham, D. (2017). Equity measures in state outcomes-based funding: Incentives for public colleges to support low-income and underprepared students. Center for Postsecondary and Economic Success. Retrieved from https://www.clasp.org/sites/default/files/public/resources-and-publications/publication-1/Equity-Measures-in-State-Outcomes-Based-Funding.pdf Cohen, A. M., & Kisker, C. B. (2010). The shaping of American higher education: Emergence and growth of the contemporary system (2nd ed. ed.) Jossey-Bass. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=cat06917a&AN=uml.b2113577&site=eds-live&scope=site 111 College Level Glossary. (2020). Postsecondary/college level glossary. National Center for Education Statistics. Retrieved from https://nces.ed.gov/surveys/ctes/tables/glossary_college.asp Compare Institutions. (2020). Integrated postsecondary education data system. Retrieved from https://nces.ed.gov/ipeds/datacenter/InstitutionByName.aspx?goToReportId=1 Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches. Los Angeles [u.a.]: Sage. Retrieved from http://www.fachportal-paedagogik.de/fis_bildung/suche/fis_set.html?FId=1106768 Curran, P., Obeidat, K., & Losardo, D. (2010). Twelve frequently asked questions about growth curve modeling. Journal of Cognition & Development, 11(2), 121-136. doi:10.1080/15248371003699969 Datapoints. (2020). Datapoints: Community college revenues. Retrieved from https://www-aacc-nche-edu.ezproxy.memphis.edu/2020/07/06/datapoints-community-college-revenues/ Definitions and Data. (2020). Nontraditional undergraduates / definitions and data. National Center for Education Statistics, Retrieved from https://nces.ed.gov/pubs/web/97578e.asp Dollery, B., & Wallis, J. (2001). The theory of market failure and equity-based policy making in contemporary local government. Local Government Studies, 27(4), 59-70. doi:10.1080/714004119 Dougherty, K. J., & Natow, R. S. (2019). Performance-based funding for higher education: How well does neoliberal theory capture neoliberal practice? Higher Education, doi:10.1007/s10734-019-00491-4 Dougherty, K., Jones, S., Lahr H., Natow, R., Pheatt L., & Reddy V. (2016). Looking inside the black box of performance funding for higher education: Policy instruments, organizational obstacles, and intended and unintended impacts. RSF: The Russell Sage Foundation Journal of the Social Sciences, (1), 147. doi:10.7758/rsf.2016.2.1.07; 10.7758/rsf.2016.2.1.07 Dougherty, K. J., & Reddy, V., (2011). The impacts of state performance funding systems on higher education institutions: Research literature review and policy recommendations. CCRC working paper no. 37. Community College Research Center, Columbia University. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=eric&AN=ED527751&site=eds-live&scope=site Dougherty, K. J., & Natow, R. S. (2015). The politics of performance funding for higher education: Origins, discontinuations, and transformations. Baltimore: Johns Hopkins University Press. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=e000xna&AN=882681&site=eds-live&scope=site 112 Eligon, J. (2015, Jan 17,). Pressed by budget squeeze, Kansas governor pulls back on tax cuts. The New York Times Estep, T. M. (2016, October). The graduation gap and socioeconomic status: Using stereotype threat to explain graduation rates. American Psychological Associations. Retrieved from https://www.apa.org/pi/ses/resources/indicator/2016/10/graduation-gap Ewell, P., & Wellman, J. (2007). Enhancing student success in education: Summary report of the NPEC initiative and national symposium on postsecondary student success. National Postsecondary Education Cooperative. Retrieved from https://nces.ed.gov/npec/pdf/Ewell_Report.pdf Federal Pell Grants (2020, December 26). 20 USC 1070a. Retrieved from https://uscode.house.gov/view.xhtml?req=(title:20%20section:1070a%20edition:prelim) Fleming, E., Heinecke, D., & Dollery, B. (2007). An analytical framework for assessing the impact of internalities on policy and projects using examples from Papua New Guinea. Impact Assessment and Project Appraisal, 25(3), 199-207. doi:10.3152/146155107X233462 Fording, R. C., Soss, J., & Schram, S. F. (2007). Devolution, discretion, and the effect of local political values on TANF sanctioning. Social Service Review, 81(2), 285. doi:10.1086/517974 Fowles, J. (2014). Funding and focus: Resource dependence in public higher education. Research in Higher Education, 55(3), 272-287. doi:10.1007/s11162-013-9311-x Funding Trends. (2014). Higher education: State funding trends and policies on affordability. Washington D.C.: Retrieved from https://www.gao.gov/assets/670/667557.pdf Gándara, D. (2019). Does evidence matter? an analysis of evidence use in performance-funding policy design. Review of Higher Education, 42(3), 991-1022. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=eric&AN=EJ1212781&site=eds-live&scope=site https://doi-org.ezproxy.memphis.edu/10.1353/rhe.2019.0027 Ganga, E., Mazzariello, A., & Edgecombe, N. (2018). Developmental education: An introduction for policymakers. Education Commission of the States. Retrieved from https://ccrc.tc.columbia.edu/media/k2/attachments/developmental-education-introduction-policymakers.pdf GAO Glossary of Terms. (2005). GAO: A glossary of terms used in the federal budget process. United States Government Accountability Office. Retrieved from https://www.gao.gov/new.items/d05734sp.pdf 113 Graduation and Retention Rates. (2020). The condition of education: Undergraduate retention and graduation rates. National Center for Education Statistics. Retrieved from https://nces.ed.gov/programs/coe/indicator_ctr.asp Graduation Rates. (2019). Digest of education statistics: Graduation rate 1996-2012. National Center for Education Statistics. Retrieved from https://nces.ed.gov/programs/digest/d19/tables/dt19_326.10.asp Hanushek, E. A., Peterson, P. E., Talpey, L. M., & Woessmann, L. (2020). Long-run trends in the U.S. SES-achievement gap. Bonn: Institute of Labor Economics (IZA). Hearn, J. C. (2015). Outcomes-based funding in historical and comparative context. Lumina Foundation for Education. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=eric&AN=ED561322&site=eds-live&scope=site Hillman, A. J., Withers, M. C., & Collins, B. J. (2009). Resource dependence theory: A review. Journal of Management, 35(6), 1404-1427. doi:10.1177/0149206309343469 Hillman, N. W., Tandberg, D. A., & Gross, J. P. K. (2014). Performance funding in higher education: Do financial incentives impact college completions? Journal of Higher Education, 85(6), 826-857. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=swh&AN=87492&site=eds-live&scope=site Hillman, N. W., Fryar, A. H., & Crespín-Trujillo, V. (2017). Evaluating the impact of performance funding in Ohio and Tennessee. American Educational Research Journal. 0002831217732951. doi:10.3102/0002831217732951 Hillman, N. W., Tandberg, D. A., & Fryar, A. H. (2015). Evaluating the impacts of 'new' performance funding in higher education. Educational Evaluation and Policy Analysis, 37(4), 501-519. doi:10.3102/0162373714560224 Hillman, N., & Orians, E. (2013). Community colleges and labor market conditions: How does enrollment demand change relative to local unemployment rates? Research in Higher Education, 54(7), 765-780. doi:10.1007/s11162-013-9294-7 Horn, A. S., & Lee, G. (2019). Evaluating the accuracy of productivity indicators in performance funding models. Educational Policy, 33(5), 702-733. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=eric&AN=EJ1217044&site=eds-live&scope=site http://dx.doi.org.ezproxy.memphis.edu/10.1177/0895904817719521 How Aid is Calculated. (2020). Federal student aid. United States Department of Education. Retrieved from https://studentaid.gov/complete-aid-process/how-calculated 114 Income and Poverty. (2020). United states census: Income and poverty in the united states. United States Census Bureau. Retrieved from https://www.census.gov/data/tables/2020/demo/income-poverty/p60-270.html Kahlenberg, R. D., Shireman, R., Quick, K., & Habash, T. (2018, OCTOBER 25,). Policy strategies for pursuing adequate funding of community colleges. Retrieved from https://tcf.org/content/report/policy-strategies-pursuing-adequate-funding-community-colleges/ Katsinas, S. G., D'Amico, M. M., & Friedel, J. N. (2014). Public higher education funding, budget drivers, and related issues: The state community college director perspective. Educational Considerations, 41(2), 20-26. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=eft&AN=99642623&site=eds-live&scope=site Kelchen, R. (2019). Exploring the relationship between performance-based funding design and underrepresented student enrollment at community colleges. Community College Review, 47(4), 382-405. doi:10.1177/0091552119865611 Kelderman, E. (2016). Will higher education be 'trumped' by state lawmakers? The Chronicle of Higher Education, 63(17), A33. Retrieved from https://search-proquest-com.ezproxy.memphis.edu/docview/1865635817 Kinzie, J., & Kuh, G. (2017, May 4,). Reframing student success in college: Advancing know-what and know-how. Change (New Rochelle, N.Y.), 49, 19-27. doi:10.1080/00091383.2017.1321429 Retrieved from http://www.tandfonline.com.ezproxy.memphis.edu/doi/abs/10.1080/00091383.2017.1321429 Kothari, C. R. (2004). Research methodology: Methods and techniques. Daryaganj: New Age International Ltd. Retrieved from https://ebookcentral-proquest-com.ezproxy.memphis.edu/lib/[SITE_ID]/detail.action?docID=431524 Kuh, G. D., Kinzie, J., Buckley, J. A., Bridges, B. K., & Hayek, J. C. (2006). What matters to student success: A review of the literature. National Postsecondary Education Cooperative. Retrieved from https://nces.ed.gov/npec/pdf/Kuh_Team_Report.pdf Kwok, O., Underhill, A. T., Berry, J. W., Luo, W., Elliott, T. R., & Yoon, M. (2008). Analyzing longitudinal data with multilevel models: An example with individuals living with lower extremity intra-articular fractures. Rehabilitation Psychology, 53(3), 370-386. doi:10.1037/a0012765 Laird. (2020). Non-probability sampling. Laerd Statistics. Retrieved from http://dissertation.laerd.com/non-probability-sampling.php 115 Leachman, M., & Mail, C. (2014). Most states still funding schools less than before the recession. Center on Budget and Policy Priorities. Retrieved from http://gateway.proquest.com.ezproxy.memphis.edu/openurl?url_ver=Z39.88-2004&res_dat=xri:policyfile&rft_dat=xri:policyfile:article:00175629 Lee, N. T. (2020, March 17). What the coronavirus reveals about the digital divide between schools and communities. The Brookings Institute. Retrieved from https://www.brookings.edu/blog/techtank/2020/03/17/what-the-coronavirus-reveals-about-the-digital-divide-between-schools-and-communities/. Li, A. (2017). Covet thy neighbor or 'reverse policy diffusion'? state adoption of performance funding 2.0. Research in Higher Education, 58(7), 746-771. doi:10.1007/s11162-016-9444-9 Li, A. (2018). Lessons learned: A case study of performance funding in higher education. The Third Way. Retrieved from https://www.thirdway.org/report/lessons-learned-a-case-study-of-performance-funding-in-higher-education Li, A., Gandara, D., & Assalone, A. (2018). Equity or disparity: Do performance funding policies disadvantage 2-year minority-serving institutions? Community College Review, 46(3), 288-315. doi:10.1177/0091552118778776 Li, A., & Kennedy, A. I. (2018). Performance funding policy effects on community college outcomes: Are short-term certificates on the rise? Community College Review, 46(1), 3-39. doi:10.1177/0091552117743790 Lorenzo, G. (2018). Facing the future: State funding of community colleges. Retrieved from https://www.edpath.com/index_htm_files/state%20funding.pdf Ma, J., & Baum, S. (2016). Trends in community colleges: Enrollment, prices, student debt, and completion. The College Board. Retrieved from https://research.collegeboard.org/pdf/trends-community-colleges-research-brief.pdf Mahon, J. (1999). Financial services modernization act of 1999. Retrieved from https://www.federalreservehistory.org/essays/gramm_leach_bliley_act Marinopoulou, A. (2017). Critical theory and epistemology. Manchester: Manchester University Press. doi:10.2307/j.ctv18b5gkj Retrieved from http://portal.igpublish.com.ezproxy.memphis.edu/iglibrary/search/MUPB0000576.html McCoach, B. D., & Kaniskan, B. (2010). Using time-varying covariates in multilevel growth models. Frontiers in Psychology, 1 doi:10.3389/fpsyg.2010.00017 McKeown-Moak, M. (2013). The "new" performance funding in higher education. Educational Considerations, 40(2), 3-12. Retrieved 116 from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=eft&AN=87590712&site=eds-live&scope=site McLendon, M. K., Hearn, J. C., & Mokher, C. G. (2009). Partisans, professionals, and power: The role of political factors in state higher education funding. The Journal of Higher Education, 80(6), 686. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=edsjsr&AN=edsjsr.27750757&site=eds-live&scope=site Merle, R. (2018, September 10). A guide to the financial crisis: 10 years later. The Washington Post. Retrieved from https://www.washingtonpost.com/business/economy/a-guide-to-the-financial-crisis--10-years-later/2018/09/10/114b76ba-af10-11e8-a20b-5f4f84429666_story.html Mitchell, M., Leachman, M., & Masterson, K. (2016). Funding down, tuition up state cuts to higher education threaten quality and affordability at public colleges. Center on Budget and Policy Priorities, Retrieved from https://www.cbpp.org/research/state-budget-and-tax/funding-down-tuition-up Mitchell, M., Leachman, M., & Saenz, M. (2019, October 24). State higher education funding cuts have pushed costs to students, worsened inequality. Retrieved from https://www.cbpp.org/research/state-budget-and-tax/state-higher-education-funding-cuts-have-pushed-costs-to-students Mitnick, B. M. (2019). Origin of the theory of agency: An account by one of the theory's originators. Social Sciences Research Network, Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1020378 Nakagawa, S., & Schielzeth, H. (2012). A general and simple method for obtaining R2 from generalized linear mixed-effects models. British Ecological Society, 4(2), 133-142. Retrieved from https://besjournals.onlinelibrary.wiley.com/doi/10.1111/j.2041-210x.2012.00261.x NSF Table Builder. (2020). NSF table builder. National Science Foundation. Retrieved from https://ncsesdata.nsf.gov/builder/ipeds_e OECD. (2019). Population with tertiary education: OECD data. Organization for Economic Cooperation and Development. Retrieved from https://data.oecd.org/eduatt/population-with-tertiary-education.htm Pell Grant. (2020). What is a pell grant? Federal program offers a source of financing for students. The College Board. Retrieved from https://bigfuture.collegeboard.org/pay-for-college/grants-and-scholarships/what-is-a-pell-grant 117 Perumal, C. A., & Padmanabhan, V. K. (1988). Post-empiricist methodology. The Indian Journal of Political Science, 49(2), 285-296. Retrieved from http://www.jstor.org.ezproxy.memphis.edu/stable/41855371 Pfeffer, J., & Salancik, G. R. (2003). The external control of organizations: A resource dependence perspective. Stanford Business Books. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=cat06917a&AN=uml.b3323391&site=eds-live&scope=site Postsecondary Attainment. (2015). The condition of education: Differences by socioeconomic status. National Center for Education Statistics. Retrieved from https://nces.ed.gov/programs/coe/indicator_tva.asp Poth, S., & Selck, T. J. (2009). Principal agent theory and artificial information asymmetry. Politics, 29(2), 137-144. doi:10.1111/j.1467-9256.2009.01349.x Productivity Funding. (2020). Arkansas productivity funding model. Arkansas Division of Higher Education. Retrieved from https://www.adhe.edu/institutions/productivity-funding/ Productivity Outcomes. (2020). General revenue data. Retrieved from https://www.adhe.edu/institutions/institutional-finance/higher-education-funding/general-revenue-data Rabovsky, T. (2014). Support for performance-based funding: The role of political ideology, performance, and dysfunctional information environments. Public Administration Review, 74(6), 761-774. doi:10.1111/puar.12274 Rankings. (2020). Pre-K - 12 rankings measuring how well states are preparing students for college. U.S. News and World Report. Retrieved from https://www.usnews.com/news/best-states/rankings/education/prek-12 Rowan-Kenyon, H. (2007). Predictors of delayed college enrollment and the impact of socioeconomic status. Journal of Higher Education, 78(2), 188-214. doi:10.1353/jhe.2007.0012 Rutherford, A., & Rabovsky, T. (2014). Evaluating impacts of performance funding policies on student outcomes in higher education. The Annals of the American Academy of Political and Social Science, 655, 185. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=edsjsr&AN=edsjsr.24541756&site=eds-live&scope=site Saxon, D. P. (2017). Developmental education: The cost literature and what we can learn from it. Community College Journal of Research and Practice, 41(8), 494-506. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=eric&AN=EJ1144485&site=eds-live&scope=site http://dx.doi.org.ezproxy.memphis.edu/10.1080/10668926.2016.1202875 118 Schanzenbach, D. W., Bauer, L., & Breitwieser, A. (2017, August 26). Eight economic facts on higher education. The Hamilton Project. Retrieved from https://www.hamiltonproject.org/papers/eight_economic_facts_on_higher_education Skolnik, M. L. (2016). How do quality assurance systems accommodate the differences between academic and applied higher education? Higher Education, (3), 361. doi:10.1007/s10734-015-9908-4 Smith, B. E. (2019). Retention of degree-seeking community college students based on age, enrollment intensity, and academic progress. [Doctoral dissertation, The University of Memphis]. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=cat06917a&AN=uml.b3425159&site=eds-live&scope=site https://umwa.memphis.edu/etd/index.php/view/home/4869 Snyder, T. (2019, September 10,). Education at a glance 2019: Putting U.S. data in a global context. Retrieved from https://nces.ed.gov/blogs/nces/post/education-at-a-glance-2019-putting-u-s-data-in-a-global-context Tandberg, D. A. (2010). Politics, interest groups and state funding of public higher education. Research in Higher Education, (5), 416. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=edsjsr&AN=edsjsr.40785167&site=eds-live&scope=site Tandberg, D. A., Hillman, N., & Barakat, M. (2014). State higher education performance funding for community colleges: Diverse effects and policy implications. Teachers College Record, 116(12), 1-18. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=psyh&AN=2014-44493-004&site=eds-live&scope=site Teasley, M. L. (2019). Education policy and outcomes within the African American population. Social Work in Public Health, 34(1), 61-74. doi:10.1080/19371918.2018.1562405 The Condition of Education. (2020). Institute of education sciences. Retrieved from https://nces.ed.gov/pubs2020/2020144.pdf Umbricht, M. R., Fernandez, F., & Ortagus, J. C. (2017). An examination of the (un)intended consequences of performance funding in higher education. Educational Policy, 31(5), 643-673. doi:10.1177/0895904815614398 Undergraduate Enrollment. (2020). The condition of education. National Center for Educational Statistics. Retrieved from https://nces.ed.gov/programs/coe/indicator_cha.asp#:~:text=Between%20fall%202000%20and%20fall,10.5%20million%20to%2013.0%20million 119 Wayt, L. K., & LaCost, B. Y. (2016). Transitioning to performance-based state funding: Concerns, commitment, and cautious optimism. Educational Considerations, 43(2), 1-6. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=eft&AN=117652079&site=eds-live&scope=site Weber, T. (2020). Discreet inequality: How party agendas embrace privileged interests. Comparative Political Studies, 53(10), 1767-1797. doi:10.1177/0010414020912286 Wolf, C., Jr. (1993). Markets or governments: Choosing between imperfect alternatives. Cambridge, Mass: The MIT Press. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=nlebk&AN=1369&site=eds-live&scope=site Wolf, C., Jr. (1978). A theory of 'non-market failure': Framework for implementation analysis. Santa Monica, California: The Rand Corporation. Retrieved from https://www.rand.org/content/dam/rand/pubs/papers/2006/P6034.pdf Woltman, H., Feldstain, A., MacKay, C. J., & Rocchi, M. (2012). An introduction to hierarchical linear modeling. Tutorials in Quantitative Methods for Psychology, 8(1), 52-69. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=edsdoj&AN=edsdoj.81637f881ae74650a6b9f8299b32c690&site=eds-live&scope=site Yeh, S. (2020). Educational accountability, value-added modeling, and the origin of the achievement gap. Education and Urban Society, 52(8), 1181-1203. Retrieved from http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com.ezproxy.memphis.edu/login.aspx?direct=true&db=eric&AN=EJ1264868&site=eds-live&scope=site http://dx.doi.org.ezproxy.memphis.edu/10.1177/0013124519896823

2016-12-02 Performance Funding Report

Abstract

Extracted data

2016-12-02 Performance Funding Report

Abstract

Extracted data

Related items

Related items