ۥ- Z{{{{{{{ 24&vmo What are Performance Indicators and Can they Be Used to Manage Colleges? A Comparative Study of Similar Colleges in the US and the UK by John R. Bolt, Infomation Systems Manager, Handsworth College, Soho Road, Birmingham B21 9DP, UK (Tel. +44 121 551 6031) and Keith J. Roberts, Director of Planning Research and Evaluation, Milwaukee Area Technical College, 700 West State St., Milwaukee, WI 53233-1443 (Tel. 414-297-6816) Presented at The Association for Institutional Research Annual Forum Lake Buena Vista, Florida May 19, 1997 What are Performance Indicators and Can they Be Used to Manage Colleges? A Comparative Study of similar Colleges in the US and the UK Abstract Performance indicators, student outcomes assessment, quality indicators, institutional effectiveness indicators are all attempts to identify how well colleges do what they say they do. In an attempt to think outside of our parochial boxes we have chosen to examine our two similar institutions (in different locations) regarding indicators used to manage our colleges. The institutions in this study are both Technical Community Colleges in industrialized cities in the US and the UK with significant minority enrollment. Both Wisconsin and England have been identified by Olaf C. McDaniel as having decentralized governance (Paradigms of Governance in Higher Education, Vol. 9, June 1996). We have addressed the following issues; how we measure ourselves, how we compare to each other, and what we can learn from each other. We are including in our discussion the concept of "Value Added to the Community" in response to the Community College research agenda set forth by the AACC (American Association of Community Colleges). What are Performance Indicators and Can they Be Used to Manage Colleges? A Comparative Study of similar Colleges in the US and the UK Introduction Not everything that can be counted counts; and not everything that counts can be counted. - Albert Einstein This paper will consider performance indicators that measure how effectively two inner city Community Colleges in different countries are providing Value Added to their local regional and national communities. Value Added to the Community is one of the specified areas within the American Association of Community Colleges Research Agenda for 1996/97 and opportunely fits within existing research areas that both colleges have been pursuing for some time. Each college sees performance indicators within the context of the greater environment. In the US example, Milwaukee Area Technical College, the influencing factors are the Institutional Effectiveness Measures developed for the Wisconsin Technical College System and the Student Outcomes Measures expected by the North Central Association accrediting agency. The UK example, Handsworth College, is influenced by the expectations of the Further Education Funding Council. In several states in the United States the idea of performance indicators has been the result of legislative action and tied somewhat to funding. South Carolina, for example, is currently in the process to tying all its funding to performance indicators. This has not been the case in Wisconsin. In late 1992 the Wisconsin Technical College Board, along with the State Technical College Presidents Association, proposed the development of a model for institutional effectiveness for the Wisconsin technical colleges. The Community College Consortium at the University of Michigan coordinated the efforts. The main objective was to develop a comprehensive effectiveness model for adaptation within the context of the individual colleges. A task force was assembled with representatives of the member colleges of the Wisconsin Technical College System working with many constituent groups to develop the core indicators and a definition of effectiveness. Effectiveness was defined as: An effective college is one that provides efficient educational programs and services that anticipate and respond to external and internal customer needs and result in outcomes that meet or exceed customer goals and expectations The task force developed seventeen core indicators of effectiveness. They are: Indicators of Employer Satisfaction WI 1 Employer satisfaction with graduates work performance. Indicators of Student Achievement and Satisfaction WI 2 Identification of student needs, goals and interests. WI 3 Identification of student functional skills at entry. WI 4 Course completion. WI 5 Student grades. WI 6 Student satisfaction with courses, programs and services. WI 7 Student retention rates. WI 8 Student completion and graduation rates. WI 9 Student achievement of educational goal(s). WI 10 Student knowledge and skills at exit. WI 11 Pass rates/scores on licensure exams. WI 12 Placement rates/employment rates. Indicators of Organizational Quality, Harmony and Efficiency WI 13 Achievement of institutional goals and standards. WI 14 Organizational climate. Indicators of Public Satisfaction WI 15 Articulation and linkages with external organizations. WI 16 Identification of customer needs and expectations. WI 17 Public satisfaction. Additional performance indicators used in Wisconsin relate to the Carl Perkins funds for vocational education. These moneys are targeted to special populations of people who are disabled, academically disadvantaged, economically disadvantaged, limited English speaking, plus programs to eliminate sex bias and persons in correctional institutions. These standards have to do with the performance of the special populations relative to the total student population. Therefore, statewide standards have been established. These measures and standards are as follows; Measure 1: Program Graduates Standard 1: 40% Measure 2: Employment and Availability for Work Standard 2: 85% Measure 3: Number and Percent of Population Served Who had a Least One Training Outcome. Standard 3: 50% Measure 4: Number and Percent of Population Served Standard 4: Set locally Measure 5: Successfully Completed Courses Standard 5: Set locally In the UK, on the other hand, all the funding is tied to performance indicators. The Further Education Funding Council (FEFC) is an arm of government which provides the large majority of the funding for colleges such as Handsworth. The FEFC has identified a range of performance indicators and management statistics which it calculates from data provided in statutory returns by the colleges. This information is published annually for each college and is commonly known as The League Tables where the performances of different colleges can readily be compared with each other. The performance indicators defined in this way are: PI 1 Achievement of the volume of education on which FEFC income is based PI 2 Student Number trends - an indicator of college responsiveness PI 3 Student continuation rates (retention) PI 4 Student achievement rates PI 5 Numbers of students achieving qualifications towards National Training and Education Targets (NTETs) PI 6 Out-turn level of funding (value for money) The management indicators are: MS 1 Students by age group and sex MS 2 Students by qualification (Analysis of what the students are studying) MS 3 Fee Remission (Analysis of reasons why students have free tuition fees) MS 4 Student Retention MS 5 Staff numbers by age and sex MS 6 Staff FTEs by type of work and contract MS 7 Current assets to current liabilities MS 8 Liquid assets to current liabilities MS 9 Cash days in hand MS 10 Annual surplus as percentage of income MS 11 General reserves as a percentage of income MS 12 Long-term liabilities as a percentage of reserves MS 13 Cash generated from operations as a percentage of income MS 14 Pay expenditure as a percentage of income MS 15 New building costs MS 16 Space efficiency MS 17 Value of college estates MS 18 Total expenditure per student FTE MS 19 Pay expenditure per student FTE MS 20 Pay expenditure on teaching and leaning per student FTE MS 21 Council Funding per achieving student FTE MS 22 Total expenditure on teaching support services per student FTE MS 23 Student FTEs per staff FTE spent on providing or supporting teaching and learning FE colleges are inspected officially every three to four years by inspectors appointed by the FEFC. For each college, gradings together with a lengthy commentary are published nationally. These gradings are: IN 1 Responsiveness and range of provision IN 2 Governance and management IN 3 Students recruitment, guidance and support IN 4 Quality Assurance IN 5 Resources for each of staff, equipment/learning resources and accommodation IN 6 The quality of the teaching in each of the curriculum areas In addition to these the college regularly calculates other indicators, such as; CL 1 Value added - comparison of students abilities and previous education compared with qualifications achieved at the end of the students chosen program. CL 2 Regular comparison of numbers of students studying and volume of education delivered against college plan. CL 3 Students progression rates to further and higher education, and employment. Profile of Our Communities Both communities are located in large cities, one in the United States and the other in the United Kingdom. Both cities have traditional manufacturing industrial bases which have experienced decline in the last twenty years or so. The populations of both cities have substantial numbers of minority group members, many of whom are non-English speakers. Milwaukee The city of Milwaukee, population 623,531, is the 17th largest city in the United States. Milwaukee Countys population is nearly one million and the four-county metropolitan area has over 1.4 million people. There are about 38,000 metro Milwaukee businesses which employ over 802,600 people. This includes 13,700 service firms, 8,200 retailers, 3,200 manufacturers, 3,300 wholesalers and 3,500 construction firms. Twenty-two percent of Milwaukees workforce is in manufacturing. Contrary to popular perceptions, less than 1% of Milwaukees workforce is employed in brewing. (Though many more are engaged in product testing, particularly when the Packers win the Superbowl!) Milwaukee area manufacturers are leaders nationally in the production of X-ray apparatus & tubes, small gasoline engines, malt beverages, iron & steel forgings, mining machinery, speed changers and drives, leather goods and lithographic commercial printing. Milwaukee is a leading producer of medical diagnostic instruments, computers, industrial robots, electronic controls & systems for factory automation, printed circuit boards, CAD/CAM processing, avionics and advanced ceramics. The African American population represents the largest minority group in metro Milwaukee. They numbered 197,183 in the 1990 census (13.8% of the population) Other minority groups were as follows; Hispanic at 51,306 (3.6% of the population), Asian at 1.3% and Native American at 0.6%. Of the European Americans in metro Milwaukee, 40% are of German heritage. The rate of unemployment in January 1997 in Milwaukee County is 4.0%. This compares with 5.5% in the City of Milwaukee, 4.3% in the State of Wisconsin and 5.9% in the US as a whole. As of February of 1997, the unemployment rate in Wisconsin had dropped to 3.5%, the lowest in twenty years. Birmingham Birmingham with a population of 961,041 is the second largest city in England. It is located in the conurbation of the West Midlands which has a total population of 2.5 million. The part of Birmingham in which Handsworth College is located is a mainly a poor working-class area where the population is predominately non-white. The 1991 census shows that largest group in this area is Indian/Pakistani/Bangladeshi at 33.9%, followed by the Black ethnic group at 23.6% and others (including Chinese) at 5.1%. Within the White ethnic group, 5.0% were born in the Irish republic. In the local area, in October 1996, 23.4% of the economically active adults aged over 16 were unemployed. This compares with 12.8% in the city of Birmingham and 7.9% for the UK as a whole. Of the employed adults the biggest group (38.9%) are employed in services (including banking and finance), 24.9% are employed in manufacturing and 19.0% in distribution and catering. In levels of education achieved, 0.9% of the adult population has a higher degree and another 9.3% has a Bachelors degree or higher education diploma. Profile of Our Colleges Both colleges are located in the inner areas of industrial cities and have large numbers of students studying subjects across a wide range of curriculum areas. Both colleges have substantial numbers of students learning English as a second language. In addition, much of the teaching takes place in the community in addition to the colleges campuses. Milwaukee Area Technical College has been identified in a recent survey as being a major contributor to the quality of life in the area. Handsworth College has pioneered schemes to facilitate groups to set up and deliver education activities for themselves. The college provides mentoring and administrative support and has obtained national funding to support these activities. Milwaukee Area Technical College The Milwaukee Area Technical College (MATC) is a four campus urban community-technical college located in the metropolitan area including and immediately surrounding the City of Milwaukee. MATC is one of sixteen districts that comprise the Wisconsin Technical College System and accounts for between 20 and 25% of the total state Technical College enrollments. Founded in 1912 as the Central Continuation School and modeled on the fortbildungschule in Munich, MATC has a long history of vocational and technical education. In addition to the regular post-secondary offerings, MATC operates an Adult High School that has been accredited since the 1920s. It is also a principal provider of adult basic education (ABE) in the Milwaukee Area. Over 16,000 students enroll in ABE and English as a Second Language (ESL) in a typical year through a system of on-campus instruction, in-house training in industry and instruction at community based organizations. The college operates the two public Television stations in Milwaukee and is also the principal investor in two small business incubators (the Milwaukee Enterprise Centers). Graduates from the many associate degree and diploma programs have a placement rate consistently over 90%. During the 1995-96 school year 31,113 students were enrolled in post-secondary programs, 19,462 were enrolled in the Adult High School and ABE/ESL courses and an additional 20,645 were enrolled in continuing education. MATC has approximately 600 full-time faculty and another 1,200 part-time faculty. MATCs North Campus, among other things, offers associate degrees in automotive technology under contract arrangements with both Ford and General Motors. Although MATC is well known internationally, as evidenced by the large number of foreign educators and business people who visit the college and also by a recent three part series shown on Korean Television showcasing MATC, the college itself has not been active in delivering education internationally. The District is unique, as a public institution, in the large number of citizens who are involved in planning and reviewing its educational programs. These citizens advisory committees are composed of business, industry, and labor representatives as well as current and past students. Currently MATC has 94 separate advisory committees with 1,054 citizen representatives. The Board of Trustees is also representative of the community. The nine members of the Board of Trustees are appointed by the elected presidents of the k-12 school boards within MATCs service area. The Board not only represents business, industry and labor but is also constructed to represent the ethnic and gender makeup of the district. This kind of involvement over the years has created a situation in which the college is able to respond to market needs. Handsworth College Handsworth College is one of eight further education colleges within the city of Birmingham. It is a six- campus urban further education college located near the center of the city. It was founded in 1893 as Handsworth Technical College and originally provided technical and commercial training to many of the small manufacturing companies in the local area. In recent years, the college has cultivated links with many minority groups in the metropolitan area and beyond and has enabled groups to provide education to students who, due to cultural influences, were unlikely to enter mainstream post-school education. In particular, Handsworth has become a center of excellence in ESL provision, and now operates such programs nationwide. By means of this type of activity Handsworth College has experienced substantial growth in the mid 1990s where the student enrollment numbers have trebled in the past four years. Handsworth College has maintained its strong links with the motor vehicle industry and operates the Birmingham Automotive Training Centre in which students receive instruction in all aspects of the motor industry, in particular in collaboration with local companies such as Land Rover and Jaguar. Handsworth College has developed links with colleges in other countries. It has pioneered an associate degree program with the City Colleges of Chicago, and has developed ESL programs with a network of colleges in USA, Canada, Netherlands and South Africa. During the 1995/96 school year 22,372 students were enrolled at Handsworth College, who amounted to 6,492 full time equivalents. These students were in studying in curriculum areas as shown in table 1 below. Table 1 Handsworth College - Student Population 1995/96 Curriculum Area % of FTEs Ethnicity % of FTEs Science 5.9% Black 17.1% Humanities 19.9% Chinese/Asian 2.3% Engineering 13.2% Indian/Pakistani 36.2% Business Studies 12.7% White 44.4% Hotel and Catering 6.1% Health & Community Care 19.7% Art and Design 9.2% Basic Education 14.1% The College Corporation (Board of Trustees) comprises 17 members of whom 11 represent local business and industry and the remainder represent the local community. Performance Indicators Over the last several years there has been a wide variety of methods, terms and concepts used to address accountability. We have discussed Institutional Effectiveness, Student Outcomes Assessment, Program Viability, Faculty Productivity and various other topics to represent measures of accountability. The term that we feel is most appropriate for our discussion is Performance Indicator because we feel that all of these terms and concepts are really ways of indicating how the institution is performing. We also understand that performance indicators are just a part of a broader context in which post secondary education is operating both in the United States and Europe. Martin Trow states that ...the underlying nature of accountability is one of three fundamental ways in which colleges and universities are linked to their surrounding and supporting societies: the others are trust and the market. We find it appropriate to define the place in post secondary education where each of our institutions are situated. In the UK, a clear distinction is made between Higher Education and Further Education and Handsworth college is clearly a further education institution. In the US no such clear distinction is made, and, in fact, community colleges and other two year colleges are usually thought of, discussed and accredited as institutions of Higher Education. But when comparing the two institutions, MATC is clearly the kind of institution that would be considered as a further education college in the UK. Therefore when we reference further education in this paper we will be referring to both institutions. In order to address Trows three fundamental linking activities in the proper context, the placement of our institutions within the mosaic of post secondary education in Europe and in the US was necessary. It is our position that older established universities have value in their communities because of the factor of trust. Further education institutions on the other hand have historically existed because of their apparent response to market needs. In both the US and the UK it is easy to develop a continuum of post secondary education with Old Prestigious Universities on one side (Oxford, Harvard), and further education colleges on the other (Handsworth College, Milwaukee Area Technical College). Although all post secondary education must attend to all of the three linking activities, it is clear to us that further education colleges can depend less upon trust and more upon accountability and markets. The American Association of Community Colleges identification of Value Added to the Community as a research need reinforces the interrelationship of accountability and markets for these colleges. For the purposes of our discussion, accountability will be measured by performance indicators. The inherent problem with accountability is one which is highlighted by Trow when he states Education is a process pretending to have a measurable outcome. This is especially true of the institutions depending largely upon trust. Those depending largely upon markets have a history of thinking that they have measurable outcomes and therefore are more likely to be involved in the performance indicator discussions. Aspects of Accountability The United States and the United Kingdom differ in the circumstances of their accountability to their respective communities. In the US regional accrediting bodies dictate the accountability measures whereas in the UK the accountability measures are administered by the government and linked closely to funding. In either case, there are certain aspects of accountability that have been discussed by Trow. These are first, distinction between external and internal accountability on one axis and a distinction between legal and financial accountability and academic accountability on the other axis. This can be illustrated by the 2 X 2 matrix below: Table 2 Accountability Matrix Accountability Legal/financial Academics External Cell 1 Cell 2 Internal Cell 3 Cell 4 Each performance indicator can be placed in one or more cells in the above matrix. The nature of the placement of the indicators will give the institution insights into what sources are driving the need for accountability in their institutions. The cell placement of the indicators identified for the Wisconsin Technical College System and for Handsworth college are shown in tables 3 and 4 below: Table 3: WISCONSIN ACCOUNTABILITY MATRIX    LEGAL/FINANCIAL  ACADEMIC   EXTERNAL  WI 7 Student retention rates WI 8 Student completion and graduation rates WI 15 Articulation and linkages with external organizations 3 items  WI 1 Employer satisfaction with graduates work performance WI 4 Course completion WI 6 Student satisfaction with course, programs, and services WI 9 Student achievement of educational goal(s) WI 10 Student knowledge and skills at exit WI 11 Pass rates/scores on licensure exams WI 12 Placement rates/employment rates WI 15 Articulation and linkages with external organizations WI 16 Identification of customer needs and expectations WI 17 Public satisfaction 10 Items   INTERNAL  WI 2 Identification of student needs, goals, and interests WI 3 Identification of student functional skills at entry WI 5 Student grades WI 13 Achievement of institutional goals and standards WI 14 Organizational climate 5 items  WI 9 Student achievement of educational goal(s) WI 10 Student knowledge and skills at exit WI 13 Achievement of institutional goals and standards WI 14 Organizational climate 4 items   Table 4: HANDSWORTH COLLEGE ACCOUNTABILITY MATRIX    LEGAL/FINANCIAL  ACADEMIC   EXTERNAL  PI 1 Achievement of volume of education on which FEFC income is based PI 2 Student Number trends - an indicator of college responsiveness PI 6 Out-turn level of funding (value for money) MS 1 Students by age group and sex MS 3 Fee Remission (Analysis of reasons why students have free tuition fees) MS 5 Staff numbers by age and sex MS 6 Staff FTEs by type of work and contract MS 7- 17 other financial indicators MS 23 26 items  PI 3 Student continuation rates (retention) PI 4 Student achievement rates PI 5 Numbers of students achieving NTETs MS 2 Students by qualification (Analysis of what the students are studying) MS 4 Student Retention IN 1 Responsiveness IN 3 Students recruitment, guidance and support IN 4 Quality Assurance 8 items   INTERNAL  PI 2 Student number trends - an indicator of college responsiveness CL 2 Regular comparison of numbers of students studying and volume of education delivered against college plan 8 items  CL 1 Value added - comparison of students abilities/previous education compared with qualifications achieved at the end of the students chosen program CL 3 Students progression rates to further programs and employment 2 items   Each indicator is placed in at most two cells by the authors based upon their perceptions of the motivations the indicators together with count of the number of indicators in each cell. The tables above illustrate that in both instances the driving forces of accountability are external. In the case of MATC, the majority of the external forces are pressures that are forcing academic accountability. In Handsworths case the majority of the external forces are legal or financial. This is consistent with the historical source of the measures. In the case of the Wisconsin indicators, they were developed by representatives of the member colleges in response to what looked like political pressure from outside of the colleges. In England, on the other hand, the indicators were bureaucratically imposed for purposes of equitable funding. Relationship to Funding The accountability measures in the UK are tied closely to funding while the accountability measures in the US have little or no relationship to funding. We will start with the Wisconsin example. The State Technical College Funding formula accounts for approximately 22% of the budget, local property tax accounts for 53%, tuition and fees account for 14%, federal 5% and other 6%. None of these funds are in response to any performance indicators that are usually defined in the literature and used in college planning and accreditation. The only indicator that has any relationship to funding is number enrolled, or, more precisely, FTE. Let us examine the technical college state aid formula in Wisconsin. It is essentially cost driven. That means that the amount that a local technical college district spends determines how much it receives in state aid. Each local district has an annual cost of operating the district with certain inclusions and exclusions by state statute. This cost is the aidable cost for that district. This factor is then modified by an equalization index (EI) which is defined by the formula below: EI = (Statewide Property Value)(District Aidable FTE)/(Statewide Aidable FTE)(District Property Value) The equalization index puts all the technical college districts in competition with each other for the fixed State funds available. Examining the EI formula shows that not only are a colleges district property assessed valuation and FTEs important but they are important relative to the property values and FTEs in the entire state. Trying to map this funding formula back to any kind of performance indicator is fruitless. In the case of Handsworth College, the FEFC provides 86% of the budget, the European Union 3%, tuition fees 1% and 10% comes from a range of sources such as other government departments. The method of funding further education in the UK was revolutionized in 1994, when the FEFC was established and introduced a formula based funding methodology to apply to all colleges in the sector. The aims of the method were to be fair and equitable to all, to be easy to understand, to maintain funding stability and to produce efficiency gains in a growing sector. In practice, the methodology has become increasingly complex as new features have been added to the formula to endeavor to maintain fairness to colleges and at the same time to encourage compliance to government initiatives. The method basically operates as follows. Some four months prior to the start of the school year, the volume of education (known as Funding Units) that each college is to be contracted to deliver, is determined by the FEFC based on college bids and other criteria that the FEFC applies. In practice, every college is allocated a number of units based on the previous year, together with a level of growth. The units are multiplied by the Average Level of Funding which is a calculated amount of money for each unit. This gives a budget figure on which the college is funded. For instance, in 1996/97 Handsworth College has been funded to deliver 912,000 funding units @ 14.32 per unit, giving a budget income of 13.1m. The Average Level of Funding (ALF) is based on the colleges previous value and a convergence factor. Currently, Handsworth College has the lowest budgeted ALF in England where its value is about half that of the highest in general further education. The FEFC is seeking to ensure full convergence will be achieved within the next three to five years whereafter the ALF will be the same for all colleges. This is likely to cause financial problems for the high ALF colleges as their budgets are reduced. At three census dates during the school year, the colleges progress towards its target number of funding units is calculated and money is clawed back if it is not close to target. Until recently, there were funds available for colleges who exceeded the target, but this has now been stopped. The funding units for each student are based on a complex formula dependent on the nature of the students program, the students retention and the students achievement of the qualification. The biggest factor is student retention as units are deducted proportionally based on the dates students withdraw. The whole process is subject to intense audit scrutiny. Teams of professional auditors visit the college several times a year to check the records and report on their accuracy. A whole new industry has now been created in running training events on how to optimize the methodology and then afterwards survive the audit. The two funding formulae are both examples of bureaucratic attempts to be equitable in the distribution of funds for the respective colleges. The Wisconsin formula attempts to dispense funds in such a way that the districts that have fewer potential resources (property tax base) are compensated by the State so that those districts can offer programs at an equal quality level to the more affluent districts. The relative performance of the individual districts is accounted for in the Wisconsin formula by the FTE factors. In England, on the other hand, equitable distribution means funding allocation relative to a unit of production. Funds are allocated according to performance measures that describe that production. Value Added to the Community The American Association of Community Colleges has placed Value added to the community on its research agenda for 1996-97. This fits our discussion well because it addresses the two sectors in which our colleges interact with the community: markets and accountability. By nature, further education colleges and community colleges are intended to be responsive to the community and therefore adding value to that community. The Milwaukee Area Technical College has been fortunate in that a local survey research firm, Management Decisions, Incorporated, has been conduction a Quality of Life Survey and has been issuing the MDI Metro Milwaukee Quality of Life Report in 1994, 1995 and 1996. The following information is directly from the 1996 MATC Student Needs Report prepared by Management Decisions, Incorporated for MATC in September of 1996. The image of MATC and of 49 other metro organizations as quality of life contributors were evaluated in two ways on the 1996 MDI Metro Quality of Life Report: * Their average score on a 11 point scale among only those who are aware of and feel they can rate an organization and its contribution; and * the percentage who give a positive or high rating among all adults, both those who are aware and those unaware of an organization. These two evaluation scores are highly interrelated. Overall, ratings for Metro Milwaukee organizations declined between 1994 and 1995, only to increase again over the past year. In 1994, the average rating scores for the 44 metro organizations were 7.04 and 43%, respectively. In 1995, the average rating scores for 51 metro organizations were 6.63 and 33%. By 1996, the average rating scores of 50 metro organizations are 6.90 and 38%. Rating scores are likely to reflect both the perceived quality (what is contributed) and the quantity (how much is contributed) to the area, as well as the success of organizations in communicating such information to the public. Milwaukee Area Technical College has remained among the top contributors to the areas quality of life for the past three years. Over the past two years, the college has been in the top ten. Milwaukee Public Television. WMVS/WMVT-TV, is affiliated with MATC and remains among the top five area contributors. Table 5 The top organizations as rated by Metro Milwaukee residents: 1994 1995 1996 1994 1995 1996 1. Pettit Foundation 8.59 8.47 8.40 68% 68% 70% 2. Milwaukee World Festival 7.95 7.86 8.30 66% 62% 74% 3. Harley Davidson 8.15 7.77 8.06 68% 60% 66% 4. Bradley Foundation 7.88 7.78 8.05 57% 49% 59% 5. Milwaukee Public Television 8.00 7.91 8.00 66% 62% 64% 6. Medical College of Wisconsin 7.75 7.52 7.69 52% 46% 50% 7. Milwaukee Area Technical College 7.36 7.50 7.58 49% 51% 55% 8. Midwest Express Airlines 7.12 7.33 7.57 44% 43% 50% 9. Miller Brewing 7.58 7.25 7.48 56% 46% 53% 10. University of Wisconsin-Milwaukee 7.27 7.31 7.47 50% 48% 54% The report goes on to say; Furthermore, MATC is on of the few organizations whose ratings have increased steadily in each of the past three years. Additionally, the school receives a superlative rating of 10 from one-fifth of all residents and an extremely low share of poor ratings - 3%. Handsworth Colleges value to its local community is represented by its rapid growth over the past few years as shown by the numbers of students enrolling each year. Table 6: Handsworth College Enrollments 1990 - 97 Year Number of % Growth Enrollments 1990/91 5,442 1991/92 7,018 29.0% 1992/93 9,650 37.5% 1993/94 15,063 56.1% 1994/95 19,193 27.4% 1995/96 22,372 11.6% 1996/97 24,000 (est.) 10.7% The rapid growth in enrollments between 1991 and 1995 is mostly due to Handsworth College becoming more closely involved with local community groups. Many of the population in the area immediately surrounding the college are of Indian, Pakistani and Afro-Caribbean origin who have had limited integration with the rest of the population, and are unlikely to enroll at public colleges. However, a considerable amount of cultural education did take place within community groups and at religious centers. Handsworth College has pioneered schemes that enabled these groups to obtain national accreditation for the study that takes place. The college works in close partnership with them in developing the curriculum and has been able to secure government (FEFC) funding to enable the work to develop. The college provides teacher training to the tutors and offers other support and guidance to ensure that the groups operate effectively and efficiently. Following this success of these activities, the schemes have been expanded to include other community groups such as Chinese and Vietnamese and groups outside the immediate local area. Furthermore, the colleges expertise and good infrastructure has allowed the model to expand nationally and now groups all over the country are being supported by the college. In the last two years, the numbers have become stabilized as the provision has become consolidated into systematic college activity. Comparative Results for our Colleges As is shown in tables 3 and 4, the indicators used at Handsworth are much more biased towards the financial health of the college than those at Milwaukee. This is because funding follows the performance indicators in the UK and therefore will be more likely to address financial issues. These financial issues are addressed in the US via annual independent financial audits. The calculation below shows a comparison for the income received for each full-time equivalent student in each college in 1995/96. Table 7 Funding and FTE Student Numbers 1995/96 Milwaukee Handsworth Ratio Operational Expenditure $113,165,000 $20,899,000 FTE students 12,307 6,492 Expenditure per FTE $9,195 $3,219 2.86:1 Gross Domestic Product (GDP) per head of population $29,600 $20,900 Expenditure per FTE as a percentage of GDP per head 31.1% 15.4% 2.02:1 Hence, in Milwaukee, further education is funded at a much higher rate than Handsworth, and even after taking relative costs into account, the rate is double. Table 8 shows the costings based on a head-count of students rather than FTEs. Table 8 Funding and Student Headcount numbers 1995/96 Milwaukee Handsworth Ratio Student headcount 63,270 22,372 Expenditure per student $1,789 $934 1.92:1 Expenditure per student as a 6.04% 4.47% 1.35:1 percentage of GDP per head (It must be stated that Handsworth College is poorly funded even by UK standards as it has the lowest Average Level of Funding in England. The average expenditure per FTE in further education in England in 1995/96 was $4,435. Handsworth fares so badly due to historical factors resulting from low levels of local funding prior to the FEFC being established in 1993 and also by the way the funding methodology handled the colleges large growth in 1993/94. However, the FEFC's policy of 'convergence' should increase Handsworth Colleges ALF.) It is useful to attempt to compare the graduation rates of each of the two colleges. In 1995/96, MATC had a graduation rate of 36% compared to the state average for Wisconsin of 43%. In the most recently published figures for 1994/95, Handsworth College was reported as having an achievement rate of 92% compared to the average for England of 75%. However, these comparisons are grossly misleading as the published English figures are based on the achievement rate of completing students (and not enrollments) and also include qualifications of all types. This particularly distorts the result for Handsworth College where a large numbers of short programs are delivered, normally with high achievement rates. It is better to compare the broadly similar programs using US terminology of Associate Degrees, Two-Year Diplomas, One-Year Diplomas and Short Programs. These are calculated by comparing the numbers graduating in 1995/96 against those enrolling as shown below: Table 9 Graduation Rates 1995/96 Type of Program Milwaukee Handsworth Associate Degrees 39% n/a Two-Year Diplomas 32% 32% One-Year Diplomas 37% 46% Short Programs n/a 79% By comparing the data in table 9 with the costings in table 7, indicative estimates of the funding per graduating student have been obtained as shown in table 10. For both colleges, average costs have been applied to subsets of the data. The actual audited costs may differ from these values. Table 10 Funding per Graduating Student 1995/96 Type of Program Milwaukee Area Technical Col Handsworth College $ % of GDP/head $ % of GDP/head Associate Degrees $47,200 159% n/a Two-Year Diplomas $57,500 194% $20,100 96% One-Year Diplomas $24,900 84% $7,000 33% Short Programs n/a $730 3.5% In this study, we have not been able to compare the depth and breadth of the academic content in the programs in the two colleges. However, Handsworth College is about to commence a program of Associate Degrees recognized in the US, where it was found that the depth of the existing Handsworth College programs were very similar to those in the US but additional study of about 25-30% of the program needs to be added to give sufficient breadth. Conclusions In both colleges, hard practical reality decrees that the principal use of performance indicators contribute towards determining each colleges annual funding budget. In Milwaukee, the student FTE number is used but the colleges budget depends to a great extent on factors outside the colleges control. At Handsworth, the funding received from the FEFC is based on pre-defined criteria which involve the following:: * the ratio of the actual volume of education delivered compared to target * the colleges average level of funding * the colleges contribution to the FEFCs main priorities, which up until now has been growth Student retention rates and graduation rates and integral features of the calculation of the volume of education delivered and qualifications which contribute to the national targets (NTETS) have higher funding unit values. Hence the college funding is related to good scores in the FEFCs performance indicators PI 1, PI 3, PI 4, PI 5 and PI 6 that were defined earlier in this paper. In addition, colleges who receive poor inspection gradings are not allowed to grow in the areas that are criticized. Milwaukee is apparently substantially better funded than Handsworth. Using the most appropriate indicator which is colleges income per FTE student as a percentage of the GDP per head of population, Milwaukee is better funded by a ratio of two to one. There are several reasons for this. The programs of study at Milwaukee have a greater breadth which probably requires greater resources to deliver. The unemployment rate in Handsworth (and Birmingham and the UK) is much higher than Milwaukee meaning that the requirement for training employed students will be lower. Generally the student participation rate at Handsworth is lower, suggesting that the population local to Handsworth places less emphasis on further education than does Milwaukee. (This is probably also true if the UK were to be compared to the US) The graduation rates of both colleges are broadly similar suggesting that the colleges educational standards are comparable. Both colleges have a close involvement with and add value to their local communities. These cannot be compared directly, but Milwaukees Quality of Life shows the level of satisfaction in the local community as does the large growth in Handsworths working with local community groups. Handsworth differs from Milwaukee in that there is strong local competition from other colleges. In Birmingham there are eight general further education colleges who recruit from the same potential student base. Each college runs aggressive marketing campaigns to try to increase its student numbers. In many large cities in England colleges have recently merged and this may yet happen in Birmingham. Milwaukee Area Technical College, at this time, has little competition from publicly funded colleges. The further education colleges in the UK are firmly attached to funding driven by performance indicators. The history and lessons that are being learned in the UK should be examined by educators from other countries, such as the US, that may be moving toward performance based funding. A fair warning to educators on the US side of the Atlantic was given by Austin R. Gilbert, chairman of South Carolinas Commission on Higher Education: We are faced with a crisis in credibility. If South Carolinas public colleges want more state money, he said, they must submit to high-stakes performance evaluations and demonstrate that they have improved. Educators in all states should be watching this closely. Discussion There are several issues that manifest themselves in regard to performance indicators. These have to do with the questions; What are we measuring? And why are we measuring it? If we are measuring our outcomes, what are they? If we are measuring quality, what is quality? Why are we measuring these things? There appears to be two justifications arising for developing performance indicators. One is for funding purposes and the other is for program improvement. Currently, the funding objective is more evident and better defined and implemented in the UK than in the US. Program improvement, on the other hand, is an often stated objective of performance indicators in the US, but the link between data gathering and actual program improvement is most often mired in complex political processes both in and outside of the various colleges. This is further complicated by the fact that many performance indicators are not indicators of performance at all but rather management or accounting tools. The danger is that there may be a growing distinction between what we do and how we look. What we do in further education will probably continue to respond in some way to local markets while how we look, that is how we portray ourselves to outside agencies, will be determined by the unique subsets of information they may require for funding or for accreditation. Our challenge will be to plan and deliver what is best for our students (customers) within the context of outside funding and accreditation criteria (performance indicators). In recent years there have been discussions of quality in education largely as the result of business practices and terminology that have migrated into education. In many places quality has replaced excellence as the catch word for the nature of the goal of our endeavor. For our purposes we will use the term excellence to mean either quality or excellence. What has been traditionally measured in the assessment of education have been profiles of the colleges. Such things as library holdings, academic preparation of faculty, resources available per student have been used to determine accreditation and rank. These indicators are not outcome indicators but may still be considered as performance indicators within the following discussion. There are two kinds of indicators that help us define and manage our colleges and subsequently portray our colleges to others. The first, are prerequisites to excellence. These are the above mentioned traditional indicators. They are the kinds of things most of us agree will be needed for the college to create and environment in which excellence can occur. The second indicators are evidences of excellence. These are the outcomes that tell us that we are doing well. In a general way, the prerequisites to excellence are the old indicators and the evidences of excellence are the new indicators. Our task is to maintain the integrity of our institutions relative to their missions while at the same time responding to these outside forces. It seems that most often these outside indicators are thrust upon us and our task becomes trying to determine how these indicators can help us better understand and manage our institutions. The indicators themselves seldom grow out of our wish to better understand and manage our institutions and therefore may not be well suited for that activity. Funding, on the other hand, poses problems because it is attempting to emulate the free market using the indicators as a proxy for the market place. While the free market players are organized to earn money the public sector service providers are organized to spend money. The mismatch is obvious. In addition the messages inherent in this activity are garbled. This is further complicated by the different nature and orientation of the training of chief financial officers and chief academic officers. As long as performance indicators are tied to funding we will not use them to manage the college we will use them to manage the budget. Suggestions for Further Study In any study, the more the authors explore the topics, the more the possibilities for further work appear, often at a much greater rate than the authors can handle. This is particularly the case when the authors are based 4,500 miles apart and also have to manage the many differences in concepts and terminology that exist in the educational systems between our countries.. This study could be extended in several ways that include: * consider the numerical values of other performance indicators listed in this study * include more colleges in the US, UK and other countries * consider the issues of quality and excellence in greater depth. Glossary of Terms AACC American Association of Community Colleges ALF Average level of funding (UK) EI Equalization index (US) ESL English as a second language FE Further education FEFC Further Education Funding Council (UK) FTE Full time equivalent (US and UK) GDP Gross Domestic Product MATC Milwaukee Area Technical College MDI Management Decisions Incorporated (US) NTETS National Training and Education Targets (UK) Bibliography (1995). A primer on performance indicators. Association of Universities and Colleges of Canada, Research File, Vol. 1, No. 2 Ewell, Peter T. (1993) Accountability at the crossroads: implications for institutional research. AIR Forum, Chicago, IL Hull, David. (1995). Farmyard PIs. Further Education Staff College, Management Information Officers Workshop. McDaniel, Olaf C. (1996). The paradigms of governance in higher education systems. Higher Education Policy, Vol. 9, No. 2, pp 137-158. Nedwek, Brian. (1996) The promises and pitfalls of performance indicators. Rocky Mountain AIR, Park City, UT. Roberts, Keith and Rieley, James B. (1995) Applying a continuous quality improvement model to assess institutional effectiveness. The Center for Continuous Quality Improvement, Milwaukee, WI The Economic Intelligence Unit. (!996). The World in Figures. The Economist World in Figures 1997, pp 95. The Further Education Funding Council. (1995). Handsworth College. Report from the Inspectorate. The Further Education Funding Council. (1995). Management Statistics for Colleges. Consultation circular 95/28. The Further Education Funding Council. (1996). College Responsiveness. National Survey Report from the Inspectorate The Further Education Funding Council. (1997). Measuring Performance. Trow, Martin. (1996). Trust, markets and accountability in higher education: a comparative perspective. Higher Education Policy, Vol. 9, No 4, pp 309-324. page 20 uvxt|& ( < z&&'(6/B/3366BBHHzKKlXX&[-[d]]]]]]]]w``jbbbbbbbbeeefffggk0kGgjuȐߐ!#ߙٟ4kx[R[vQho5CIMVy4NPgtjH,? EFLMOPVXZB. "$3Q\^`bdfhtvz|46tvxz|AC& ( * , : < c e ')HJ24 V"` *YN D`bpgi0]} @Bf=k : s !6!_!!!!=""""o#q#### $_$$$$$$$%%&&k&n&p&ep&r&t&x&z&&&''( ()),, . .2/4/6/B/D///113 3333336666;8=8;;<<|=~=>>)@+@AABBBCCEEFFGGHHHH8I, yfюӎՎȐ#%xjĒrɓ "X  hX X ZS RTۙݙߙ HJuwMOǝɝACd؞ڞܞޞ~џӟ՟<>np=?A{}ܧާAbdgPR߫F$gikxX X X Yxz#%u;= AC+- "LN#ikmo3d04CKŽX X X X X X X X X X ?KMNPPR`bPR<>@BERTVXZ`X X !F )9r ZZmZo2 Tms Rmn `Symbol HelvTimes New Roman !"#%Z"Z-vHWhat are Performance Indicators and Can they Be Used to Manage Colleges?Handsworth collegeHandsworth college