WPCo L<hUϳovw?BL uA,!=IT!^0>q^X'I~n](<1v gMų ˩xNYxsX:uXw Pl w.8.ֿO"Q~t&iM{-d!0n?5x.(pHLME|{hȩV6V?:Doe 1Ö$Z2[{lgO]范%{&Aco/ ~pgy4{adǣ< %ޜ"2 LH qH x6id n_JuymHyɗ ],'# "UN- %{ 0(U>w@4';JUHR-U:@ oV b]]@ 6]oV@]@]@]@]@]@]@]@]@]@]@]@]@]@] 0DRRR D+ 0K AҴHP LaserJet 4Si/4Si MXHPPCL5E,,,,0 (9 Z 6Times New Roman RegularX($<( 9Z &Courier Regular VVDUJoK"3|x,v Z 0Book Antiqua Regular ##Xd#,2 AZ"Arial Regular WPC'u u 9!u Uƚ %`o * ~&f & &$TNPPMicrosoft PowerPoint & TNPPf & &TNPP  v 345&W?"Arial Rounded MT Bold-.  32 Licensure Examination Resultst4rrthtTr0ds4tsD4ts0rht4Dh?"Arial Rounded MT Bold-.  (2 yas Outcome Indicators:rh0tDrtr0<tx4rrDtUh<?"Arial Rounded MT Bold-.  '2 Issues and Challenges<hhtrh0rtx0tr44rtxrh & &`Hf "Arial Rounded MT Bold-.  .2 "Dr. Gita Wijesinghe PitterS1#Y(BiC<CFDBK'(B1"Arial Rounded MT Bold-.  '2 [bMs. Claudia H. Lanham]=#SCDFBV#CCCDCc"Arial Rounded MT Bold-.  '2 $eMrs. Donna McGalliard]1=#SDCDC]BYCB1F"Arial Rounded MT Bold-. "Arial Rounded MT Bold-.  02 #Office of Academic ProgramsL 99: E99<9U9@*:<*9U4"Arial Rounded MT Bold-.  :2 K"State University System of Florida@"9"9I:49*4"4@44"9U: ::*<9"Arial Rounded MT Bold-"Arial Rounded MT Bold-. "Arial Rounded MT Bold-.  f2 ?Paper presented at the 37th Annual Forum of the Association for502/#2$/+00/201/0/09100000#1F1009+,000010$"Arial Rounded MT Bold-.  *2 Institutional Research,0,0100:/,/0#/1"Arial Rounded MT Bold-.  !2  Orlando, Florida?#002011#20"Arial Rounded MT Bold-.  2 ) May 1997C/,/0/0 & &Hj--"System-jH & TNPP.  -- &WordPerfect 6.1 . D- H'--j --$I_0ywQW)7 o?zrjjjPr zpP( )Qy0_Fv( P5pMe} P?o}eM57 WwvF- - - %I_0ywQW)7 o?zrjjjPr zpP( )Qy0_Fv( P5pMe} P?o}eM57 WwvF- -- $ M  %%5=-MM- - - % M  %%5=-MM- -- $e(0 ]HeHme- - - %e(0 ]HeHme- -- $tmmmmmeeee]]UMMEEEE====555x5p5p5p5h5h=h=`=`E`E`E`M`U`]`]`e`ehehmhmpmpmpmxuxmmmmmuuuuu}x}p}puhuhuhu`u`m`mXmXeXeX]X]XUXUXMXMXEXEX=X=`5`5`5h5h-h-p-p-x-x-----5555===EEMMUU]eeemmm- - - %tmmmmmeeee]]UMMEEEE====555x5p5p5p5h5h=h=`=`E`E`E`M`U`]`]`e`ehehmhmpmpmpmxuxmmmmmuuuuu}x}p}puhuhuhu`u`m`mXmXeXeX]X]XUXUXMXMXEXEX=X=`5`5`5h5h-h-p-p-x-x-----5555===EEMMUU]eeemmm- -- $@}5555555===EEMMMUUUMMEEE=======UUUUUUUU]]]eeemuu}}}uummee]]]]]}}- - - %@}5555555===EEMMMUUUMMEEE=======UUUUUUUU]]]eeemuu}}}uummee]]]]]}}- -- $u--uu- - - %u--uu- -- x$:u-%%%%'%'%'-/-/-7-75757=7=?E?E?M?M?U?U?]7]/]/]/U7U7M7M7E/E/=/=/5/5'5'-----5me'e/e/]7]7]7e7e/m/m'm'mmu- - - x%:u-%%%%'%'%'-/-/-7-75757=7=?E?E?M?M?U?U?]7]/]/]/U7U7M7M7E/E/=/=/5/5'5'-----5me'e/e/]7]7]7e7e/m/m'm'mmu- -- "$OeGOo5g5OWEg5o5MMo=WMW]Oe- - - "%OeGOo5g5OWEg5o5MMo=WMW]Oe- -- $IoO0yagQOI/91)!!!p)P189IQayxhXP0HOHoHHHPXh&x>^v8Pp/Ogv^>&- - - %IoO0yagQOI/91)!!!p)P189IQayxhXP0HOHoHHHPXh&x>^v8Pp/Ogv^>&- -- $x8V0N8F8F@F@>@>@>@6@.@.@&8&8800((   &.> F N N V V ^^^ffffffff^^VVNNFF>>66...&&...66>>FFNNVVVV^^VVVNNF>..&  ((0088@@@H&H&H.H.H6H6H>H>HFHFHN@N@N8V8V- - - %x8V0N8F8F@F@>@>@>@6@.@.@&8&8800((   &.> F N N V V ^^^ffffffff^^VVNNFF>>66...&&...66>>FFNNVVVV^^VVVNNF>..&  ((0088@@@H&H&H.H.H6H6H>H>HFHFHN@N@N8V8V- -- $ 0(0- - - % 0(0- -- "$( o  (- - - "%( o  (- -- $ _Www(0H(W _- - - % _Www(0H(W _- -- $ (0 0@(0- - - % (0 0@(0- -- l$4@@HPPXXX```hhpppxxxxxxyxyxypqpqpihi8A0I`qhyhyhppphhh``XXXPHH@q- - - l%4@@HPPXXX```hhpppxxxxxxyxyxypqpqpihi8A0I`qhyhyhppphhh``XXXPHH@q- -- $ qH)P!9xAI`1iq- - - % qH)P!9xAI`1iq- -- $999- - - %999- -- $!- - - %!- -- $ 0 ``(0``08pp0 - - - % 0 ``(0``08pp0 - -- $@xpx- - - %@xpx- -- $y- - - %y- -- $7?'- - - %7?'- -- $ OoWW_O- - - % OoWW_O- -- $ 9 !A9- - - % 9 !A9- -- $xqyy'''///''yqqqiia'a'a'Y/Y/Y7Y7Y?Y?YGYGaOaOiWiWiWqWqWyWy__WWWWOOGGOOOOOyOyOqGqGiGi?i?a7a7a7a/i/i/i/q/q/y/77777777//'yyqq- - - %xqyy'''///''yqqqiia'a'a'Y/Y/Y7Y7Y?Y?YGYGaOaOiWiWiWqWqWyWy__WWWWOOGGOOOOOyOyOqGqGiGi?i?a7a7a7a/i/i/i/q/q/y/77777777//'yyqq- -- $ 'GowWO''- - - % 'GowWO''- -- $yWWWOOGGGG G G(G0G0G8G8O@O@W@W@_@_@_@g@g@g8g0g gooooooww  ((00888@@@88000((  wwwww w0w8w8w@o@oHoHgHgO_OWOWOOOOOGHGHGH?@?8?8?0?07(7 7 ?????GGOOOW- - - %yWWWOOGGGG G G(G0G0G8G8O@O@W@W@_@_@_@g@g@g8g0g gooooooww  ((00888@@@88000((  wwwww w0w8w8w@o@oHoHgHgO_OWOWOOOOOGHGHGH?@?8?8?0?07(7 7 ?????GGOOOW- -- $ GooOOGGo- - - % GooOOGGo- -- $ GogoOG7G- - - % GogoOG7G- -- $7&7&w^onF>g^''77- - - %7&7&w^onF>g^''77-   --$f^^ff- - - %f^^ff-  --$f~vnf- - - %f~vnf-  --$www- - - %www-  --$ooooo- - - %ooooo-  --$ggggg- - - %ggggg-  --$___W_- - - %___W_-  --$WWWWW- - - %WWWWW-  --$ggg_gg- - - %ggg_gg-  --$ooogo- - - %ooogo-  --$~ww~w~~- - - %~ww~w~~-  --$ffffff- - - %ffffff-  --$VV^VVV- - - %VV^VVV-  --$V^^VV- - - %V^^VV-  --~$'/??GGGGGOW_gggoooowww (8@HWow&&.6>FVVVV^^VNNVNF>w6o.g&g_WWWOOGGGGG???77/''''/7/''''wg_'W'H'H@0((  xxxhpphXXPPXXPH@0( ((( - - - ~%'/??GGGGGOW_gggoooowww (8@HWow&&.6>FVVVV^^VNNVNF>w6o.g&g_WWWOOGGGGG???77/''''/7/''''wg_'W'H'H@0((  xxxhpphXXPPXXPH@0( ((( -  - &%`pp `P0 8H@_Xg``X- - % 8@@WX_`w- - (%(8@H`_pwp`X8- - % 000@Pp8- - %@8@H- - %H0HP`p- - 00h- - 00- - 0 80- - 0(@0- - 0P@H- - 0`8H- - 08- - 08- - &%6F- - %- - (%.>FVVF6V6- -- $0080- - - %0080- -- $p p`p - - - %p p`p - -- $- - - %- --$ (`(P0H@8@00 0 (- - - % (`(P0H@8@00 0 (- - !@8- - %8@8HH@8@- --$ HOH_Xg``X`ppw`_HO- - - % HOH_Xg``X`ppw`_HO- --$ `P@88HHP`- - - % `P@88HHP`- -- $N6- - - %N6- --,$F>&6F- - - ,%F>&6F- -- $- - - %- --$ 0W0H8( 8H8W0W- - - % 0W0H8( 8H8W0W- - %IoO8oyWi?a'QIAAAAxAXI@Q aiyxh8hO`o``hhx6Nfv @Xx'?WovfN6- '-''' & TNPP--' & &TNPP & --    WPC'u u 9!u Uƚ %`o * ~&f & &$TNPPMicrosoft PowerPoint & TNPPf & &TNPP  v 345&W?"Arial Rounded MT Bold-.  32 Licensure Examination Resultst4rrthtTr0ds4tsD4ts0rht4Dh?"Arial Rounded MT Bold-.  (2 yas Outcome Indicators:rh0tDrtr0<tx4rrDtUh<?"Arial Rounded MT Bold-.  '2 Issues and Challenges<hhtrh0rtx0tr44rtxrh & &`Hf "Arial Rounded MT Bold-.  .2 "Dr. Gita Wijesinghe PitterS1#Y(BiC<CFDBK'(B1"Arial Rounded MT Bold-.  '2 [bMs. Claudia H. Lanham]=#SCDFBV#CCCDCc"Arial Rounded MT Bold-.  '2 $eMrs. Donna McGalliard]1=#SDCDC]BYCB1F"Arial Rounded MT Bold-. "Arial Rounded MT Bold-.  02 #Office of Academic ProgramsL 99: E99<9U9@*:<*9U4"Arial Rounded MT Bold-.  :2 K"State University System of Florida@"9"9I:49*4"4@44"9U: ::*<9"Arial Rounded MT Bold-"Arial Rounded MT Bold-. "Arial Rounded MT Bold-.  f2 ?Paper presented at the 37th Annual Forum of the Association for502/#2$/+00/201/0/09100000#1F1009+,000010$"Arial Rounded MT Bold-.  *2 Institutional Research,0,0100:/,/0#/1"Arial Rounded MT Bold-.  !2  Orlando, Florida?#002011#20"Arial Rounded MT Bold-.  2 ) May 1997C/,/0/0 & &Hj--"System-jH & TNPP.  -- &WordPerfect 6.1 . D- H'--j --$I_0ywQW)7 o?zrjjjPr zpP( )Qy0_Fv( P5pMe} P?o}eM57 WwvF- - - %I_0ywQW)7 o?zrjjjPr zpP( )Qy0_Fv( P5pMe} P?o}eM57 WwvF- -- $ M  %%5=-MM- - - % M  %%5=-MM- -- $e(0 ]HeHme- - - %e(0 ]HeHme- -- $tmmmmmeeee]]UMMEEEE====555x5p5p5p5h5h=h=`=`E`E`E`M`U`]`]`e`ehehmhmpmpmpmxuxmmmmmuuuuu}x}p}puhuhuhu`u`m`mXmXeXeX]X]XUXUXMXMXEXEX=X=`5`5`5h5h-h-p-p-x-x-----5555===EEMMUU]eeemmm- - - %tmmmmmeeee]]UMMEEEE====555x5p5p5p5h5h=h=`=`E`E`E`M`U`]`]`e`ehehmhmpmpmpmxuxmmmmmuuuuu}x}p}puhuhuhu`u`m`mXmXeXeX]X]XUXUXMXMXEXEX=X=`5`5`5h5h-h-p-p-x-x-----5555===EEMMUU]eeemmm- -- $@}5555555===EEMMMUUUMMEEE=======UUUUUUUU]]]eeemuu}}}uummee]]]]]}}- - - %@}5555555===EEMMMUUUMMEEE=======UUUUUUUU]]]eeemuu}}}uummee]]]]]}}- -- $u--uu- - - %u--uu- -- x$:u-%%%%'%'%'-/-/-7-75757=7=?E?E?M?M?U?U?]7]/]/]/U7U7M7M7E/E/=/=/5/5'5'-----5me'e/e/]7]7]7e7e/m/m'm'mmu- - - x%:u-%%%%'%'%'-/-/-7-75757=7=?E?E?M?M?U?U?]7]/]/]/U7U7M7M7E/E/=/=/5/5'5'-----5me'e/e/]7]7]7e7e/m/m'm'mmu- -- "$OeGOo5g5OWEg5o5MMo=WMW]Oe- - - "%OeGOo5g5OWEg5o5MMo=WMW]Oe- -- $IoO0yagQOI/91)!!!p)P189IQayxhXP0HOHoHHHPXh&x>^v8Pp/Ogv^>&- - - %IoO0yagQOI/91)!!!p)P189IQayxhXP0HOHoHHHPXh&x>^v8Pp/Ogv^>&- -- $x8V0N8F8F@F@>@>@>@6@.@.@&8&8800((   &.> F N N V V ^^^ffffffff^^VVNNFF>>66...&&...66>>FFNNVVVV^^VVVNNF>..&  ((0088@@@H&H&H.H.H6H6H>H>HFHFHN@N@N8V8V- - - %x8V0N8F8F@F@>@>@>@6@.@.@&8&8800((   &.> F N N V V ^^^ffffffff^^VVNNFF>>66...&&...66>>FFNNVVVV^^VVVNNF>..&  ((0088@@@H&H&H.H.H6H6H>H>HFHFHN@N@N8V8V- -- $ 0(0- - - % 0(0- -- "$( o  (- - - "%( o  (- -- $ _Www(0H(W _- - - % _Www(0H(W _- -- $ (0 0@(0- - - % (0 0@(0- -- l$4@@HPPXXX```hhpppxxxxxxyxyxypqpqpihi8A0I`qhyhyhppphhh``XXXPHH@q- - - l%4@@HPPXXX```hhpppxxxxxxyxyxypqpqpihi8A0I`qhyhyhppphhh``XXXPHH@q- -- $ qH)P!9xAI`1iq- - - % qH)P!9xAI`1iq- -- $999- - - %999- -- $!- - - %!- -- $ 0 ``(0``08pp0 - - - % 0 ``(0``08pp0 - -- $@xpx- - - %@xpx- -- $y- - - %y- -- $7?'- - - %7?'- -- $ OoWW_O- - - % OoWW_O- -- $ 9 !A9- - - % 9 !A9- -- $xqyy'''///''yqqqiia'a'a'Y/Y/Y7Y7Y?Y?YGYGaOaOiWiWiWqWqWyWy__WWWWOOGGOOOOOyOyOqGqGiGi?i?a7a7a7a/i/i/i/q/q/y/77777777//'yyqq- - - %xqyy'''///''yqqqiia'a'a'Y/Y/Y7Y7Y?Y?YGYGaOaOiWiWiWqWqWyWy__WWWWOOGGOOOOOyOyOqGqGiGi?i?a7a7a7a/i/i/i/q/q/y/77777777//'yyqq- -- $ 'GowWO''- - - % 'GowWO''- -- $yWWWOOGGGG G G(G0G0G8G8O@O@W@W@_@_@_@g@g@g8g0g gooooooww  ((00888@@@88000((  wwwww w0w8w8w@o@oHoHgHgO_OWOWOOOOOGHGHGH?@?8?8?0?07(7 7 ?????GGOOOW- - - %yWWWOOGGGG G G(G0G0G8G8O@O@W@W@_@_@_@g@g@g8g0g gooooooww  ((00888@@@88000((  wwwww w0w8w8w@o@oHoHgHgO_OWOWOOOOOGHGHGH?@?8?8?0?07(7 7 ?????GGOOOW- -- $ GooOOGGo- - - % GooOOGGo- -- $ GogoOG7G- - - % GogoOG7G- -- $7&7&w^onF>g^''77- - - %7&7&w^onF>g^''77-   --$f^^ff- - - %f^^ff-  --$f~vnf- - - %f~vnf-  --$www- - - %www-  --$ooooo- - - %ooooo-  --$ggggg- - - %ggggg-  --$___W_- - - %___W_-  --$WWWWW- - - %WWWWW-  --$ggg_gg- - - %ggg_gg-  --$ooogo- - - %ooogo-  --$~ww~w~~- - - %~ww~w~~-  --$ffffff- - - %ffffff-  --$VV^VVV- - - %VV^VVV-  --$V^^VV- - - %V^^VV-  --~$'/??GGGGGOW_gggoooowww (8@HWow&&.6>FVVVV^^VNNVNF>w6o.g&g_WWWOOGGGGG???77/''''/7/''''wg_'W'H'H@0((  xxxhpphXXPPXXPH@0( ((( - - - ~%'/??GGGGGOW_gggoooowww (8@HWow&&.6>FVVVV^^VNNVNF>w6o.g&g_WWWOOGGGGG???77/''''/7/''''wg_'W'H'H@0((  xxxhpphXXPPXXPH@0( ((( -  - &%`pp `P0 8H@_Xg``X- - % 8@@WX_`w- - (%(8@H`_pwp`X8- - % 000@Pp8- - %@8@H- - %H0HP`p- - 00h- - 00- - 0 80- - 0(@0- - 0P@H- - 0`8H- - 08- - 08- - &%6F- - %- - (%.>FVVF6V6- -- $0080- - - %0080- -- $p p`p - - - %p p`p - -- $- - - %- --$ (`(P0H@8@00 0 (- - - % (`(P0H@8@00 0 (- - !@8- - %8@8HH@8@- --$ HOH_Xg``X`ppw`_HO- - - % HOH_Xg``X`ppw`_HO- --$ `P@88HHP`- - - % `P@88HHP`- -- $N6- - - %N6- --,$F>&6F- - - ,%F>&6F- -- $- - - %- --$ 0W0H8( 8H8W0W- - - % 0W0H8( 8H8W0W- - %IoO8oyWi?a'QIAAAAxAXI@Q aiyxh8hO`o``hhx6Nfv @Xx'?WovfN6- '-''' & TNPP--' & &TNPP & --(3$ !  'dxd( $ Figure  1  Level 1Level 2Level 3Level 4Level 5% $# I&mage <=8C HKKKK !   (#(#XXXX  HXX |H 7XXdd7  #H |# H# X X d#   X Xm&9)%`|0V^& ` %E^&^&Z5&m #.  M##XX.# XXXX |X X                  +&$ LicensureExaminationResultsasOutcomesIndicators:IssuesandChallenges#H ||##XXH#Abstract  ( x     Asaccountabilityandperformancebasedbudgetingbecomeincreasinglyprevalentinhighereducation,quantifiableoutcomesmeasureshavegainedinpopularity.Ameasurethathasinitialappealislicensureexaminationresults.However,themeasureisnotyetwidelyused,dueprimarilytodifficultiesinobtainingdata,andisfraughtwithissueswhichneedtobeaddressedinordertoobtainmeaningfulinformation.Thepaperdiscussestheextenttowhichthemeasureisusedinseveralsystemsofhighereducation,theissuessurroundingpassratesonexaminations,andprovidesalistofnationalorganizationswhichadministerlicensureexaminations,toassistresearchersinterestedintheuseofthismeasure. Introduction  h% "   Outcomesmeasureswhichmaybeobtainedwithrelativeeaseandnotmuchadditionalexpenditurehavegainedpopularityasincreasedpressurehasbeenappliedtocollegesanduniversities +X&( toprovidequantifiabledataabouttheirprogramsandgraduates.Theresultsoflicensureexaminationsasanindicatorofoutcomeshasaparticularappealbecauseitisaquantitativemeasureandisdirectlyrelatedtotheemployabilityofaninstitutionsgraduates.Whilelicensureexaminationresultscanproveusefulasanoutcomesmeasureandasameanstocurricularimprovementsinacademicprograms,theeasewithwhichthedatamaybeobtainedvarieswidelybystateandbydiscipline.Inaddition,anumberofissuesshouldbetakenintoconsiderationwheninterpretingthedatainordertounderstandthemeaningoftheinformationgathered.  Thepaperpresentsanddiscussesissueswhichshouldbetakenintoconsiderationandthechallengesfacedintheuseoflicensureexaminationresultsasameasureofoutcomesininstitutionsofhighereducation.Thediscussionisbasedontelephonesurveysofseveralstateuniversitysystems,nationalentitiesresponsibleforcertificationexaminations,facultyexpertsinvariousdisciplines,reviewofpublicationsanddocumentsregardinglicensure,andexperiencesinFlorida,astatewhichhashadgreateraccesstolicensureinformationthanmanyothers.̜  ,`'* CertificationandLicensureExaminations     Whatarecertificationandlicensureexaminations,andwhatisthedifferencebetweenthetwo?Bothtypesofexaminationstesttheknowledgeofanindividualinaparticulardiscipline,toassureacertainlevelofcompetence.Licensureisgrantedbythestate,whereascertificationisgrantedbyaprofessionalorganization,andacertificationexaminationmaybedevelopedandadministeredbyanationalentitysuchasacommission,acounciloranassociation.Somenationalassociationscontractwithatestingservice,suchastheEducationalTestingService(ETS)todevelopandadministertheexamination.Passinganationalcertificationexaminationisoftenonecriteriaforobtainingstatelicensure.Inthecaseofsomedisciplines,astatemayuseanationalexaminationandsupplementitwithastatespecificportionoftheexamination,usuallydealingwiththestatelawsrelatingtotheprofession.Suchexaminationsmaybecalledlicensureexaminations.Someprofessionsmayhavecertificationexaminations,butpassageoftheexaminationmaynotberequiredforemploymentandlicensuremaynotexist(suchasvocationalrehabilitationcounselingandhealthinformationmanagementinFlorida.)Inthiscase,practitionersseekcertificationnotbecauseofnecessity,buttoreassureclients ,`'* oftheircapabilities.̜  Itisimportanttounderstandthepurposeoflicensureandcertificationexaminationsbeforeusingpassratestomakeassumptionsaboutthequalityofeducationalprograms.InmostϜprofessions,certificationorlicensureisintendedtoensureathresholdofcompetenceonthepartoftheexamineetosafeguardthepublic.Whileapassratethatfallssignificantlybelowthenationalaverageshouldbecauseforconcernandfurtheranalysis,ahighpassratedoesnotnecessarilysignalahighqualityprogram.Theeducationalprogramsshouldbepreparingstudentstodofarmorethansimplymeetathresholdofcompetence.Itispossiblethattwoprogramsofsimilarlyhighpassrateshavevastlydifferentlevelsofquality,whereoneprogramsimplypreparesthestudentstopasstheexamination,whiletheothergoesmuchfurther. DisciplineswithNationalExaminations      Disciplinesofferedatthebachelorslevelorhigher,inwhichnationalexaminationsareavailableandwidelyusedincludeaccounting,architecture,landscapearchitecture,interiordesign,clinicallaboratorysciences(medicaltechnology),clinicalsocialwork,dentistry,dieteticsandnutrition,engineering,law,marriageandfamilytherapy,medicine,mental ,`'* healthcounseling,nursing,nursinghomeadministration,occupationaltherapy,pharmacy,physicaltherapy,psychology,schoolpsychology,radiologicalsciences,rehabilitationcounseling,respiratorytherapy,speechlanguagepathologyandaudiology,teachereducationandveterinarymedicine.Otherdisciplineswhichareofferedatanassociatedegreelevelalsohavenationalcertificationexaminations.Insomeinstances,disciplinesmayhavenationalexaminationscorrespondingtoprogramsattheassociateandatthebachelorslevel.Inafewdisciplines,suchasnursing,bothassociateandbaccalaureatedegreestudentssitforthesameexamination.AppendixAcontainsalistofthenationalentitiesresponsibleforwidelyusedcertification/licensureexaminations. UseofExaminationDatabyStates  `   Aninformaltelephonesurveyofseveralstateuniversitysystemsrevealedthatsomestatesarealreadyusinglicensurepassratesasaperformanceindicatoratthestatelevel,eitheraspartofanaccountabilityreportorperformancebasedbudgeting.ThesestatesincludeNorthCarolina,Tennessee,TexasandWisconsin.AcompletelistofstatessurveyedisincludedinAppendixB.NorthCarolinausespassratesasacomponentoftheUniversityAssessmentPlan.Theyarealsoconsideringusingit ,`'* asameasurefortheIncentiveFundingPlan.Tennesseeusespassratesasameasureforperformancebasedfunding.Itisoneof10variables,accountingfor5%ofthebudget.Tennesseehasdevisedaformulaforthispurposewhichusespassratesandaslidingscalebasedonthenumberoffirsttimetesttakers(passrateswithmanytakersareweighedmoreheavily).Theformulageneratespointsforperformancebasedfunding.PassratesareameasureintheperformancebasedfundinginTexas,althoughnofundsaretieddirectlytothismeasure.Texasisplanningtoincorporatepassratesintotheprogramreviewprocess.InWisconsin,passratesareoneindicatorontheaccountabilityreport.Allthesestatesonlyusepassratesoncertification/licensureexaminationsinselecteddisciplines,suchasnursing,lawandmedicine.Thedecisiononwhichdisciplinestouseisbasedoneaseofobtainingthedataandhowcriticalpassageoftheexaminationistopractice.  InFlorida,SouthCarolina,andColorado,passratesareoneϜindicatorbeingconsideredforperformancebasedfundinginthenearfuture.Floridausedpassratesasanindicatorintheaccountabilityreportin1993,buttheyhavenotbeenusedsinceϜthen.However,passratescontinuetobeusedasoneindicatorofprogramqualityinsystemwideprogramreviews(apracticeof ,`'* longstanding).AfewstatessuchasCalifornia,ArizonaandGeorgiahaverejectedtheconceptofusinglicensurepassratesasanindicator.  Problemsencounteredbystatesystemsinclude:thelimitedavailabilityofpassratesfromnationalentities;thefinancialburdenofobtainingpassratesfromsomeagencies;theinterpretationoftheratesonceobtained;andthenonexistenceofsometypeofnationaldatabaseorsoftwarecoursefilewheredatacanbeentered,configured,andusedforspecificresearchinterests. DataConsiderations  p   Accessibility .Nationalandstateagenciesresponsiblefor P theexaminationshavetraditionallyviewedtheexaminationsastoolsforassessingthecompetenceofindividualtesttakers,notasindicatorsofperformancefortheprogramsandinstitutionswhichpreparedthetesttakers.Infact,severalagenciescontactedforinformationcautionedagainsttheuseofpassratesforevaluatingprograms.Becauseofthisviewpoint,obtainingdataforprogramevaluationcanbequiteachallenge;manyagenciesarenot geareduptoprovidesuchdata.Theextenttowhichusefuldataiscollected,recorded,andmadereadilyavailablevariesgreatlybydisciplineandagency.Somenational ,`'* associationsroutinelyprovidethedatatotheinstitutions.Otherseitherdonotcollectdatainaformthatwouldbeusefultotheprogramsordonotmakeapracticeofprovidingthedatatotheprograms.Somechargeafeetoprovidethedata.̜  Statesystemsoruniversitycentraladministrationsmayeitherrequesttheaffectedprogramstoprovidethepassratestothem,orobtainthemdirectlyfromthenationalentityorstateagencyresponsiblefortheexamination.Thebenefitofrequestingdatafromtheprogramsisthatthisisprobablytheeasiermethodsinceprogramshavebetteraccesstotheinformationthanacentraladministrationorsystemoffice.Thebenefitofacentralofficeobtainingdatadirectlyisthatonehasmorecontrolofthedataincludedinthereport.  InFlorida,obtainingdatafromtheappropriatestateagenciesisoftenachallenge.Theidealsituationwouldbefortheappropriateagenciestocollecttherelevantinformation,Ϝstoreitinaneasilyaccessibleformonadatabase,wherebyinstitutionsofhighereducationcouldtapintothedatabasetonotonlyobtainpassratesbutalsoconductmoreindepthstudieswhichcouldexaminethepassratesbycohort,byraceandgender,subtests,theaveragenumberofattemptstopassallsubtests,andothervariables.InFlorida,theStateUniversitySystem, ,`'* theDivisionofCommunityColleges,andsomeprivateuniversitiesareattemptingtoworkwiththeDepartmentofBusinessandProfessionalRegulation(DBPR)tobringaboutaccesstothedataϜthroughadatabase.  Subtests. Manyexaminations,suchasthoseinpsychology, 0  clinicalsocialwork,medicaltechnology,speechpathologyandaccounting,consistoftwoormoresubtests.Ofteninsuchcasesasingleoverallpassrateforallsubtestsattemptedwillbereported.Thismeansthatapersonwhositsfortwosubtestsandpassesthembothwillbecountedasapass,andapersonwhositsforthreesubtestsandpassestwowillbecountedasafail.However,insomedisciplinessuchasarchitecture,onlythepassratesforeachsubtestarereported.Examineesmayretakeonlythesubteststheyfailed.Thesefactorsmakeitdifficulttointerpretpassratesforrepeattakers.Also,reportingpassratesonanumberofsubtestsperdisciplinedoesnotlenditselftobrevityandsimplicitywhichareimportantinreportingpassratestothegeneralpublicandlegislators.Forpublicreporting,itmaybebesttoprovideonlytheoverallpassrateoffirsttimetakers(ifavailable),ratherthanresultsonsubtestsandpassratesforrepeattakers.Informationonindividualsubtestsisusefulforcurricularreviewpurposes. ,`'* М   Testtakers. 󛀀Individualswhositforanexaminationforthe  firsttimearereferredtoas firsttimetakers.Thosewhoretaketheexaminationorsubteststhattheyfailedonthefirsttryare repeaters.Dataismostusefulifitdistinguishesbetweenfirsttimetakersandrepeaters.Somenationalentitiesandstateagenciesprovidedataonthesetwotypesoftesttakersseparately,butsomewillonlyreportfirsttimetakers,andotherswillcombinethetwo.CombiningfirsttimetakersandϜrepeattakersisnotadvisablebecauseitskewsthedataandmakesusefulinterpretationdifficult.UsuallytherepeaterstendtohavesignificantlylowerpassratesthanthefirsttimeϜtakers.Also,therelationshipbetweenthequalityoftheeducationreceivedintheprogramandperformanceonthetestisnotasclearsincetimeandotherexperienceshaveintervened.Themostuseful,mostwidelyreported,andsimplestmeasuretoreportisthepassrateoffirsttimetakers.Thatis,whatpercentageofthegraduatesfromaparticularprogram,takingtheexaminationforthefirsttime,receivedapassingscoreontheexamination?Inadditiontothismeasure,itmaybeusefultoknowwhatproportionofyourgraduatesdosucceedinpassingtheexaminationwithinareasonableperiodaftergraduation(e.g.5years).Inascertainingtheeconomicimpactofaninstitutions ,`'* programsonthestateorlocalcommunityitmaybeusefultohavedataonallgraduateswhoeventuallybecomecertifiedorlicensedinafield.̜   Annualdata .Mostexaminationsareadministeredmorethan P  onceperyear.WhenobtainingdataitisadvisabletorequestϜsittingsforacompleteyear.Reportingagenciesmaynotalwaysspecifyiftheyareprovidingyouwithdataonallthesittingsforagivenyear;itisbesttoprobe,andalsotospecifywhatconstitutesthebeginningandendingofthereportingyear.̜ EvaluatingPassRates   М  Aprogramspassrateforagivenyearisthepercentageofgraduatesfromthatparticularprogramwhoreceivedapassingscoreonallsubtestsattemptedwithinthegivenyear.Oncepassratesareobtained,whatdoesonedowiththem?Itisnotveryilluminatingtosimplyreportpassratesforvariousdisciplinesforagiveninstitution,withoutanypointsofreference.Forexample,whethera75%passrateisgoodnewsorbadnewswilldependonthediscipline,andwhetheritisbetterorworsethanpreviouspassratesforaparticularprogram.Thetworeferencepointsthataregenerallyusedare1)nationaland/orstatepassratesfortheexamination,and2)previoushistoryofpassratesforgraduatesoftheparticularprogram.Thepassing ,`'* scoresofthenationalexaminationsformostdisciplinesaresetbythenationalagencyresponsiblefordeveloping,administeringandscoringtheexamination.Individualstatesusuallyacceptthepassingscoresuggestedbythenationalentity.Ifthereisachangeinpassingscore,itisusuallyincreasedby5points,therebysettingahigherstandardforstatecompetencyandpassageoftheexamination.Additionally,somestateshavedevelopedtheirownsectionoftheexaminationwhichtestscandidatesonthelawsandrulesofthatstatewhichgoverntheparticulardiscipline.Normally,thepassingscoreforthissectionoftheexaminationissimilar,ifnotthesame,asthatsetforthenationalexamination.Therefore,oncethepracticeinagivenstatewithregardtopassingscoresinvariousexaminationsisdetermined,onecanmakeuseofthenationalandstatepassrateforcomparisonpurposes.Inmostcases,nationalpassratesforvariousexaminationsdoprovideausefulpointofcomparison.Thesepassratescanbeobtainedthrougheachnationalentityresponsiblefortherespectiveexaminations.Alistofmanyofthesenationalentities,theiraddressesandtelephonenumbersappearsinAppendixA.Nationalpassratesvaryconsiderablybydiscipline,rangingfromhighratesof90%andaboveforseveralalliedhealthdisciplines,tolowratesof ,`'* 20%suchastheCertifiedPublicAccountant(CPA)examination. CriticalityforPracticeintheProfession   М  Howcriticalispassingthecertificationorlicensureexaminationforgettingonesfirstjobintheprofession?Thisisanimportantquestionwhichshouldbeaskedbeforedecidingwhichdisciplinestoincludeinameasureusingpassratesasanoutcomesindicator.Criticalitycanhaveaconsiderableimpactonthepassratesthemselves,aswellashowimportantameasureofprogramqualitythepassratesare.Ifpassingtheexaminationisnotcriticaltoobtainingtheirfirstjobintheprofession,studentsarenothighlymotivatedtotaketheexaminationimmediatelyupongraduation,ortoperformverywellonit.Thismayaffectthepassratesthroughnofaultoftheeducationalprogram.Criticalitycanvarysomewhatbystate,dependingonthestatelawsgoverningpracticeinthevariousprofessions.Thecriticalityandotheracademic/licensureinformation,ofeachdiscipline,islistedinAppendixC.  Ingeneral,thereseemstobeaninterestingcorrelationbetweenthenationalpassratesandcriticalityforemployment.Professionsinwhichcriticalityishighgenerallyhavehighnationalpassrates,andprofessionsinwhichcriticalityismoderatetolowgenerallyhavemoderatetolownationalpass ,`'* rates.Inmostofthehealthdisciplines,criticalityishigh.Inthesedisciplines,unlessyoupasstheexaminationyoucannotpracticeataprofessionallevelinthediscipline,andthefinancialconsequencestotheindividualaresevere.Theexpectationofalmostall(ifnotall)studentsenteringsuchprogramsisthattheywillpasstheexaminationupongraduation.Incontrast,indisciplinessuchasarchitectureandlawthecriticalityismoderate,andindisciplinessuchasengineeringandaccounting,thecriticalityisrelativelylow.Inlaw,forexample,itispossibletobecomeemployedinmanyprofessionaljobsinthefieldoflaw,otherthanpracticeasanattorney,withoutpassingthebarexamination.InengineeringmoststudentsdonotneedtotaketheFundamentalsofEngineeringexaminationforentrylevelpositionsinEngineering.Oneregisteredengineermayhavealargenumberofnonregisteredengineersworkingunderhis/hersupervision.ThecriticalityofbeingaregisteredengineerishigherinsomeengineeringϜdisciplines,however,suchascivilengineering.Inaccounting,employersdonotexpectgraduatestohavepassedtheCPAexaminationpriortotheirfirstjob.Inbothengineeringandaccounting,notpassingtheexaminationmaybeabarriertoadvancementincertaintypesofjobs.Thereforeindividualsmay ,`'*  7XXdXXd7   7XXdXXd7  taketheexaminationsafewyearsaftergraduation.Thisbringsuptheissuediscussednext,i.e.thetimeelapsedbetweengraduationandtakingtheexamination.̜ Timeelapsed  0    Somedisciplines,suchasarchitecture,engineering(tobecomearegisteredengineer),anddieteticsrequireaninternshiporsomepracticalexperienceinthefieldpriortosittingfortheexamination.Thisexperienceisincorporatedintosomedegreeprograms,whileinotherstheexperienceoccursaftergraduation(thisvariesbydisciplineaswellasbyprogramwithinagivendiscipline).Thedurationoftheexperiencevariesbydiscipline.Inprogramswheretheexperienceisgainedaftergraduation,andthedurationoftheexperienceisayearormore,themoretenuousaretheconclusionsonecandrawabouttheprogrambasedontheexaminationresults.Thelikelihoodthatasignificantportionofthegraduatesmaynotbereportedbackaspartoftheinstitutionspassratesincreases,particularlywhengraduates(perhapsthemostableones)obtainworkexperienceinotherstatesorothercountries.Also,theexaminationmaybepracticebased,andtheresultsmaybeasmuchareflectionoftheinterveningpracticalexperiencesofthestudentsasofthe ,`'*  7XXdXXd7  academicprogramfromwhichtheygraduated. Conclusion     Opinionsvaryastothevalidityofusingcertificationandlicensureexaminationresultstoassessacademicprograms.Theoverwhelmingresponsetotheauthorsqueryoftestingagenciesregardingtheusageofexaminationresultstoimprovecurriculawasoneofcaution.Theyfeltthattheexaminationsaremoreameasureofthestudentsabilitythanofprogramquality.Certainagenciesexpressedtheirconcernovertheuseoflicensurepassratesasameasureofprogramquality,stressingthattheirexaminationisdesignedtotestthecandidatespracticumexperiencesaswellastheireducationalbackground.Forexample,theexaminationinarchitectureisdesignedtotesttheknowledgeofthecandidateafterthetwotothreeyearpracticumthatisrequiredofallexaminationcandidates.SomestatesystemsofhighereducationandmanyacademicprogramswithintheFloridaStateUniversitySystemhavequestionedtheuseofexaminationpassratestojudgetheacademicqualityofprograms.Thediscussionssurroundingthisissueinregardtoaccountabilityandperformancebasedbudgetingareintheearlystages;inadditiontotheproblemsofobtainingdata,thereisuneasinessonthepartofentitiesresponsibleforthe ,`'* examinationsaswellastheacademiccommunityregardingtheinterpretationofthedata.Basedonexperiencesinprogramreviewandaccountability,examinationofdata,discussionswithentitieswhichadministerexaminations,faculty,andotherstatehighereducationsystems,theauthorsofferthefollowingthoughtsandrecommendations:1.Passratesabovethenationalorstateratesarenotnecessarilyindicatorsofhighqualityeducationalprograms.ϜCertificationandlicensureexaminationsaregenerallyintendedtoensureathresholdofcompetencefortheprotectionofthepublic.Ontheotherhand,passratessignificantlybelowthenationalorstateratesshouldalwaysbecauseforselfscrutinyinaprogram,andactasatriggertoexaminetheprogramfordeficiencies,providedthenumberoftesttakersisnotsolowastomakethepassratemeaningless.Theremaybereasonsforthelowpassratesotherthanaproblemwiththequalityoftheprogram,butoneshouldexaminetherelevantinformationbeforedrawingconclusionsordismissingthepassrates.2.Itisalmostalwaysusefulforprogramstoreviewtheexaminationresultsandpassratesoftheirgraduates.Somedisciplines,inwhichpassingtheexaminationiscriticalforpractice,aremoreaccustomedtosuchselfscrutinythanothers. ,`'* Evenindisciplineswherepassingtheexaminationisnotcriticalforpractice,butmanystudentsenterwiththegoalofbecomingcertifiedorlicensed,programscangleanusefulinformationbyexaminingtheresults,andshoulddoso.Resultsforanyoneyearmaybeananomaly;atrendoverseveralyearswouldbemoreuseful.Aftercarefulanalysisofexaminationresultsandotherrelevantfactors,programsshouldbewillingtomakecurricularchangeswhennecessary.3.Tothisend,thereshouldbeagreaterwillingnessandabilityonthepartofentitiesresponsiblefordeveloping,administeringandreportingexaminations,tocollectandproviderelevantaggregatedatatoprograms.OrganizationssuchastheAssociationforInstitutionalResearch(AIR)andlegislatorsinterestedingreateraccountabilitycouldbeofgreathelpincreatinganationalmilieuwhereaggregatedataonexaminationresultsaremorereadilyavailable.Itwouldalsobeofassistancetoinstitutionalresearchersiftherewasacentraldatabankwhichcontainedthenationalpassratesinvariousdisciplinesagainstwhichtocompareprogrampassrates.3.Adistinctionshouldbemadebetweenscrutinizingpassratesattheprogramlevelandreportingpassratesatastatelevelforaccountabilityandperformancebasedbudgetingpurposes.At ,`'* theprogramlevel,themanycomplexitiesandcaveatspeculiartoaparticulardiscipline,suchaslowcriticality,timeelapsedduetofieldexperiencerequirements,etc.canbetakenintoaccountwheninterpretingthepassrates.Examiningpassrateswithinthecontextofprogramreviewswhichinvolveexternalexpertsinthedisciplineisquiteusefulandbringsalevelofobjectivitytointerpretingaprogramspassrates.Atastatelevel,whereexaminationpassratesareoneofseveralmeasuresreported,andtheaudienceisnotgiventomullingthecomplexitiesofindividualdisciplines,itisbesttolimitreportingtodisciplineswherethecriticalityofpassingtheexaminationishighandnosignificanttimeintervenesbetweengraduationandtheexaminationduetoarequiredfieldexperience.Herethereportingcanberelativelystraightforward,andshouldincludetheannualpassrateforfirsttimetakersandareferencepointsuchasanationalorstatepassrateforfirsttimetakers.AttemptingtoincludeinformationthatdoesnotlenditselftosimplicityandbrevityinanarenaruledbysoundbitesislikelytofosterϜmisunderstanding.4.Ifpassratesreportedforaccountabilityandperformancebasedbudgetingarelimitedtodisciplinesinwhichthedatais ,`'* relativelystraightforward(highcriticality,nosignificanttimelapseaftergraduation),isitreasonabletojudgetheentireinstitution,andperhapsallocateaportionoftheinstitutionsbudget,basedontheperformanceofstudentsinsuchalimitednumberofprograms?Themajorityoftheprogramsarelikelytobeinonediscipline(health),whichfurtherlimitstheapplicability.5.Ratherthanindividualsremovedfromhighereducationmakingjudgementsaboutentireinstitutionsbasedonsimplepassratesforafewprograms,itmaybepreferabletoholdinstitutionsaccountable,andtheyinturnholdprogramsaccountable,forcarefullyscrutinizingthepassratesinrelevantdisciplines,andmakingchangeswhentheyarewarranted.  0   .XX  .AppendixA P    p AppendixB  P AppendixC  P AppendixD  #      ~   XX ~AppendixD    NOTESONFLORIDASEXPERIENCESINOBTAININGPASSRATEDATA     FloridasDepartmentofBusinessandProfessionalRegulation(DBPR)includesaBureauofTestingwhichproducesexaminationstatisticsformorethan50professions.Eachprofessionhasaboardthatcreatestheirownrules,regulationsandmethodsforadministration.DifferentdepartmentswithinDBPRhavedifferentfunctions;Psychometricsdesignsthestateexams,ExamsServicesadministerstheexams,andSystemsgeneratestatisticsfromtheexamresults.Thereportsgeneratedprovideinformationonbiographics,education,pass/failrates,schools,andinstate/outofstatestatus.CopiesofthereportsareobtainedfromtheListingandLabelsdepartmentandareavailableonpaperordiskette.Unfortunately,thereportscurrentlygeneratedprovideverylimitedfeedbackwhentryingtotracebacktoouruniversities.Thebiographicssuchasrace,sex,andageareasumofallthetesttakersandarenotpresentedbyuniversity.TheStateUniversitySystem(SUS)hasbeenabletoselectFirstTimetakerswhograduatedfromouruniversities,butnotthedateofgraduation.TheSUScollectionofinformationisatthemercyofthesender.Duringthecourseofayear,formostprofessions,therearemultipleopportunitiestositforanexam.Reportsforaparticulartimeperiodcanberequested,butiftheyarenotincludedwemustassumethattherewerenoothertesttakersthanthereportsthatwerecieved.Thelicensurereportsdonottelluswhetherapersonevermanagestopassalltherequiredsubtestswithintheprescribedtimelimittoearnalicense.Thisisimportanttolearnhowmanyofourgraduatesareeverabletopracticewithintheirfield,iflicensingisarequirement.Manyexamsaremadeupofsubtests.Ifthetesttakerisrequiredtositforallthesubtests,thenthereportiscountingequally.But,iftesttakersareallowedtositforpartoftheexams,thenitchangesthemeaningofthepass/failrates.Forinstance,ifaprofessionhasfivesubtestsandapersonsitsforfourandpassesonlythreetheyaremarkedasafail.Ifanotherpersonsitsfortwosubtestsandpassesboth,theyareconsideredapass.Thefirsttimeacandidatesitsforthetest,theyarelabeledafirsttimetaker.Ifittakesthreesittingsbeforesuccessfullypassingallthesubtests,theyarecountedasafailforFirsttimetaker,onefailandonepassforrepeattakers.Someexamsrequirethatthepersonhavecertainexperiencesinthefieldbeforebeingeligibletositfortheexam.Inthesecasespass/failratesshouldnotbeusedasaninstrumenttoevaluatethequalityoftheeducationalprogram.Thecontentoftheselicensureexamsarebasedontheprofessionalactivititeswhichneedtobeperformedcorrectlybyanewlylicensedpractitionerinordertopreventanyharmtothepublic.Emphasisisgiventotestingthecandidatesclinicalor ,`'* practicalknowledgeessentialtoprotectingthepublic.Therefore,licensureexamsrequiringexperienceinfieldarenotlimitedtoknowledgegainedsolelyfromtheeducationalprogram.Thegreaterthetimethathaspassedsincecompletionoftheprogram,thecandidatesperformanceontheexamdecreases.Ratherthanareflectiononthequalityoftheprogramitindicatesthatwhatwaslearnedwasnotretainedovertime.Oncertainexamsrecentgraduateswhohavelesspracticalexperiencewillgenerallyperformlesswellthantheircounterpartswithmoreexperience.TheStateUniversitySystemhasbeenfortunatethatwearenotcurrentlyrequiredtoreportlicensureforAccountability.ItisunderconsiderationforinclusioninPerformanceBasedFundingReports.However,otherlevelsofpostsecondaryinstitutionsarerequiredtoreportlicensure,suchasthecommunitycollegesystem,theIndependentcollegesanduniversities,andthevocationalinstitutions.TheBoardofRegentsisoftenaskedtofurnishlicensuredataforthelegislativestaff.Theinformationisalsousedwhenconductingprogramreviewsorexaminingprogramsforvariousotherreasons.Ithasbeenourexperiencethatgettinginformationfromthestateagencyisverydifficult.However,inrecentyearstheagencyhasbegunbuildingacomputerizeddatabase.Atthepresenttime,weareworkingtowardsacquiringpermissiontotapintotheirdatabasetorunstatisticalreportsthathavemeaning.Ideally,wewouldliketosharethedatabasewiththeotheragencieswhoarerequiredtofurnishsimilarreports.Curently,thefocusisonthepassratesoffirsttimetakers.Ifwecouldtapintoadatabasewewouldbeabletoexpandourknowledgeandcomparepassratesonsubtests,tofollowcandidatesthroughtheentireprocess.Wecouldmoreeasilycompareourgraduatestocandidatesmovingintoourstatewhoalsositfortheexam.Wecouldrunstatisticsonraceandsexandage.Wecouldcomputeanaveragelengthoftimeandnumberofsittingsthatittakescandidatestocompletethelicensureprocess.Inotherwords,wewouldgainagreaterunderstandingofhowlicensureaffectstheprocessofourgraduatesabilitytoworkintheirfield.# X XXXo##   X Xp#  p&!# m& 9)%`|0V^& ` %E^&^&Z5&mm& 9)%`|0V^& ` %E^&^&Z5&m̛