|
A brief review of the Accreditation history reveals that colleges have been performing comprehensive self-studies of their institutions religiously. However, what these self-studies often do not reveal is the extent and rigor with which student services have been assessed. As the ACCJC (The Accrediting Commission for Community and Junior Colleges) is moving toward outcome based self-study, the student services personnel feel increasingly under pressure to assess student services. Based on surveys collected at several student services assessment workshops conducted by the author, student services at California community colleges have shown an increase in the level of awareness and numbers of colleges that conduct regular student services assessment. However the depth, defined by the number of areas covered, is yet to show growth.
From a budgetary point of view, student services traditionally share about 20% of the total expenditure. From a service utilization perspective, a typical student is half-time, taking two or three classes, and the fact is no matter how many classes s/he takes, s/he definitely interfaces with the Student Services. S/he must apply for admission, seek counseling, go through assessment testing and orientation (if s/he is taking more than 3 credit units), visit the health services if needed, purchase text books, dine at the cafeteria, utilize parking, discuss career and transfer issues at the career or transfer center, and apply for financial aid. The list can go on. Even though not all these services are considered student services in all colleges, they represent a large part of the interaction between the college and the students that has traditionally remained hidden and under-studied. The limelight has been on teaching, grades, and learning outcomes, namely, instruction.
It has become clear that a student's college experience is no longer defined by time spent in the classroom. Their experience starts the moment they apply for enrollment, further enhanced by their interaction with counseling, assessment, financial aid, and others - all before they even start their first day of class. They will continue to interact with and rely on these services to continue their study. These interactions therefore influence their learning outcome. For example, before they obtain counseling, they must receive appropriate orientation so that they know where to go for help. If they do not receive quality counseling, they will end up taking the wrong classes and possibly prolong their time-to-degree. If they are placed in the wrong class, they will be wasting their time for the whole term. In sum, there will be no quality instructional outcomes if a learner is not well informed and adequately serviced. In a cohort tracking study done by Barr and Rasor (1999), they found that students who used student services (EOP&S, DSPS, MESA, etc.) persisted longer and achieved more than those who did not, including Transfer Ready status.
There is no doubt that student services is a mission critical function of today’s learning. When learners demand quality education, they are talking about both learning that takes place inside a classroom as well as the process with which they obtain learning. It is not atypical to receive copious statements from students concerning non-classroom related inconveniences and problems that affect their learning. Practitioners in research have been saying all along that close to a quarter of the reasons students left college were related to counseling, A&R, and financial aid (Woods, 1995). As another example, a recent study by South Texas Community College stated similar findings.
Many theories and models exist to either describe the out-of-classroom factors’ impact on learning, or strategies to remedy the negative influences. To name a few, Terenzini’s Student Learning Model identifies the importance of both classroom experience and out-of-class experiences; Rootman, Pace and Holland’s College Fit Model focuses on the match between students’ expectations and their college experience; Weidman’s model of undergraduate socialization demonstrates the impact of college environment on students’ growth.
There are also studies conducted on retention and persistence using academic factors and student demographics, while leaving out student services, such as the “pathways to degree” study by Cabrera et al (2001). Without the inclusion of student services variables (may be called student affairs in some institutions) does not mean they are irrelevant. Quite contrary, the difficult job of separating student services variables from the academic variables speaks in volumes their long held importance. All of the theoretical and empirical knowledge points to one thing, that student services is an integral part of student learning; that as teaching is to learning, student services is to the learning experience; that good student services will exert fine influences on student learning experience.
Higher education is gradually moving out of the traditional in-class lecturing mode to an anytime anywhere mode as evidenced by burgeoning distance/online learning. Students may never appear in a traditional classroom at all, evidenced by the number of online degrees offered by reputable institutions. Booming is a more appropriate word to describe the growth of distance/online learning courses and demands in recent years. In order to successfully conduct the transfer of knowledge in this changing world, successful instruction has to rely more and more on services that it used to treat as supportive and secondary. These services ought to be viewed as essential components for they determine the effectiveness of course delivery. In the past, a college's reputation was determined by outcome measures developed for instruction, which largely concerned with instructors' ability to teach. Now, a need exists to establish outcome indicators to measure the effectiveness of non-instructional services, particularly student services.
A MODEL IMPLEMENTED
What is a good way of assessing student services? First, assessing student services need not reinvent the wheel and most existing evaluation methodologies should apply. Second, successful assessment of student services needs to begin at the individual component level, i.e., counseling, registration. It is the sum of individual components of student services that collectively determines the institutional effectiveness, which manifests in terms of transfers, graduates’ employability, degrees awarded, success and retention, among others. The entire suite of diverse student services components jointly influences the student learning experience.
What should be assessed in each component? Again, wheels need be reinvented. Since evaluation in essence is about answering the question of whether a particular goal has been achieved by the person/entity that proposed to achieve it, then the correct approach is to examine the processes and results of the achievement of goals or objectives set forth by each individual component. The goals or objectives set by the components are contracts and essential obligations they have between themselves and their service recipients, in this case, registered students. Assessment of student services then is the on-going process of examining what the service provider promises to provide and what the service recipient receives. Just to provoke more thinking on this subject, following is a conceptual model of the steps it takes to assess a student services component in a typical community college in California.
STEP ONE: INFORMATION COLLECTION - Who are we?
The step starts with an explanation of the purpose followed by specific tasks of gathering background information, which is crucial to understanding the individual component.
· Historical review of the component (e.g. how and when established)
· Organization chart
· Personnel (full-time, part-time, student, volunteers)
· Summary of duties (e.g. scope of authority, services provided, qualifications of employees, ratio, etc.)
· Equipment (e.g. technology, etc.)
· Hours of operation
STEP TWO: ACCOUNTABILITY - What do we do?
In some institutions, this step may take place first, but it would remain highly abstract without knowing first the "who" from Step One. This step includes the following:
- Mission or philosophy
- Specific goals and objectives
- Relationships with students and other student services and instruction of the college
STEP THREE: ASSESSMENT - How have we done?
This is broken into two parts: Process and Outcome. Process is an examination of the process of service delivery - "Have we done what we planned to do?" This part solicits lots of "yes" and "no" answers. It is followed by quantitative and measurable data, which help to answer the question, "How have we done it?" Outcome asks the questions: Are students satisfied? What areas have been rated high or low? Process translates the objectives decided by the Mission and Goals to questions that need a "yes" or "no" answer. Outcome relies on statistics, e.g., specific numbers of using services of admissions, health services, and service feedback surveys, e.g. interest, needs, satisfaction, awareness, etc. Both process and outcome evaluations compensate each other to provide feedback to student services personnel to improve and enhance their work.
STEP FOUR: PLANNING - How can we do better?
This is the last step to close the loop of Planning, Implementation and Evaluation. It materializes in the form of a report. However, the report can only be meaningful if the previous steps are done well. The report should contain the following:
- Candid evaluation of service delivery both process and outcome
- Future needs
- Recommendations/ new goals
- Resources needed (e.g. dollars, materials, personnel, facilities, etc.)
- Timetables and persons responsible
These four steps help illustrate the basic flow of the process of conducting student services assessment.
Upon implementing this assessment process at Cabrillo College, Chair of the Counseling Division, Arturo Canto, remarked a typical sentiment of the student services personnel, “The process, after we’ve done it, is very helpful for me as a manager and informative to other employees. It is rooted in our job duties, which we can easily relate to, it is logical with clearly laid out steps, and it builds a connection between our jobs and students’ learning experience as well as the connection between our day-to-day work and the statistics.” Another manager, Joe Nugent, Director of Maintenance & Operations, wrote, “The practice of defining and understanding goals, strategies and objectives are critical to progressive management.”
CONCLUSION
About 16 years ago, a document entitled, “They Said It Couldn’t Be Done” was published. It was a joint effort of close to 1,000 personnel, both practitioners and institutional researchers, in the California Community College System. This particular document is still being cited today as a comprehensive and reasonable approach to evaluating student services. Its core principal is harmonious with the ideas expressed in this article, arguing for many of the same points. It has sample timelines, milestones, and survey forms that help guide the practice. The web site for this document is at: http://www.cabrillo.cc.ca.us/oir/ssevaluation/sservpg1.htm. In addition, another excellent source of guidance and motivation comes from the duo, Lee Upcraft and John Schuh, who authored a series of books on assessment in student affairs.
Assessing student services needs immediate attention from the colleges taking a comprehensive approach to college assessment. Assessing student services helps to complete the entire picture of assessment of learning. This task can be accomplished if the practitioners and institutional researchers work together. The theoretical discussion and pragmatic solutions expressed herein attempt to provide some framework, structures and practical ideas about how to start evaluating and assessing student services. It conceptualizes a process that is true to the nature of the student services and is based on actual practices in student services assessment.
Bibliography
Barr, J., & Rasor, R. (1999) Freshmen Persistence As Measured by Reaching Academic Achievement Benchmarks. Presentation at the Annual Conference of the Research and Planning Group for California Community Colleges. Lake Arrowhead, CA
Cabrera, A., Nasa, S., & Burkum, K. (2001) Pathways to a Four-Year Degree: The higher education story of one generation. Center for the Study of Higher Education. Penn State University.
Luan, J., Robles, H., & Ramirez, D. (1999) Assessing Non-instructional Programs-Student Services. Proceedings from the California Assessment Institute. Long Beach, California, 1999.
Nichols, J. (1995) The Departmental Guide and Record Book for Student Outcomes Assessment and Institutional Effectiveness. Agathon Press, New York.
Office of Institutional Research and Effectiveness (1998) Retention at South Texas Community College: A delicate balance. Special Report No.9. South Texas Community College.
Pacheco, D. (1999) Culture of Evidence. Assessment and Accountability Forum. Page 14.
Seybert, J(1996) Assessment at Two-Year Institutions. Assessment Update Vol 8., No. 4., Assessment Update - First Ten Years (1999). Trudy Banta, Editor. Page 280.
They Said It Couldn’t Be Done. Student Services Program Review Project-Final Report. Santa Ana, California, 1986.
Upcraft, L., & Schuh, J. (1996) Assessment in Student Affairs – A Guide for Practitioners. Jossey-Bass Publications, San Francisco.
Woods, E. G. (1995) Reducing the Dropout Rate. School Improvement Research Series (SIRS). North West Regional Educational Laboratory.
|
