The Big Picture – A Four-Year Cycle for Institutional Effectiveness at Dixie College

Credit Where Credit is Due:

During the summer of 1995, Joe Peterson (Dean of Arts, Letters & Science) attended a conference in Nashville sponsored by James O. Nichols and his group, Institutional Effectiveness Associates. At that conference, Dean Peterson learned a broad conceptual approach to institutional effectiveness, which he then brought to campus planners and adapted to Dixie College’s needs. This approach can be found in Nichols’ book, A Practitioner’s Handbook for Institutional Effectiveness and Student Outcomes Assessment Implementation, 3rd ed., Agathon Press, 1995.

Institutional Effectiveness Defined:

Validating that in whatever endeavor we find ourselves (teachers, secretaries, administrators, librarians, grounds workers, etc.), we accomplish the objectives of that role.

Tasks involved in institutional effectiveness:
• Establish or Define Intended Outcomes
• Conduct the Activity or Task with Outcomes in Mind
• Assess the Outcomes
• Implement Improvements to Enhance Effectiveness

The above steps result in the following four-year “cycle” of institutional effectiveness:
1. Year one – establish the mission statement, the institution-wide statement of purpose and goals.
2. Year two – divide the institution’s employees into groups or units and ask each group to establish “statements of intended outcome” that link thematically to the goals of the mission statement. Also, ask each group to devise methods to assess the group’s intended outcomes.
3. Year three – the groups implement the assessment methods they have devised and determine how effectively the outcomes are achieved. Also, the groups plan improvements to processes, in order to enhance effectiveness.
4. Year four – the mission statement is re-visited in light of the assessment activities and improvements implemented, and the statement is updated and re-drafted.

Purposes and Assumptions of Institutional Effectiveness Activities:

Institutional effectiveness activities are undertaken with two broad purposes: IMPROVEMENT and ACCOUNTABILITY:

Improvement:
• This is by far the most important purpose of institutional effectiveness. Finding ways to better achieve intended outcomes will make Dixie College, according to its Vision Statement, “recognized as the finest community college in the nation.”

Accountability:
• Dixie College must be answerable to entities that provide its resources, and to the citizens of the state of Utah. Additionally, in our last accreditation report, evaluators wrote, “Recommendation One: It is strongly recommended that the college move immediately and aggressively to develop and implement an institution wide quality assurance/institutional effectiveness program. The college is clearly not fully in compliance with standard I.”
• Following are some entities that require accountability:
  • Legislature, Commissioner’s Office, Parents, Students, Accreditors, etc.

Certain assumptions underlie institutional effectiveness activities, including –

• The main purpose of Institutional Effectiveness is program improvement.
• These assessment activities have nothing to do with the evaluation of individual employees. The purpose of Institutional Effectiveness is program improvement, not remediation of any individual employee. No employee will be terminated as a result of Institutional Effectiveness assessment activities.
• While we must conduct assessment activities to satisfy the demands of accreditors, the legislature, and the constituency that pays for public education, accountability is secondary to program improvement.

Year One – Establishing The Mission Statement, the Foundation of Institutional Effectiveness:

The mission statement articulates particulars of institutional purpose, goals, and identity. It establishes a shared vision of the institution, a foundation upon which other departmental or unit goals rest. It is important that departments or units of the college base their sense of purpose and their intended outcomes on the mission statement.

During the 1994/95 academic year, a Strategic Planning Committee was established to draft a mission statement. This committee conducted a “Climate Survey,” a gathering of demographic data intended to describe the larger economic, social, and political environment in which Dixie College’s Strategic Plan would be set forth. Also, this committee drafted Dixie College’s MISSION STATEMENT (click here to see it).

Year Two – Establishing the Department or Unit Outcomes:

During the 1995/96 academic year, with the mission statement approved and in place, the institutional effectiveness process moved to the department or unit level. Following were tasks undertaken this year:

• Departments and units were identified for institutional effectiveness purposes, and from within each group, a representative was appointed. This representative was designated the contact person and coordinator for institutional effectiveness activities.
• Each department or unit identified between three and five statements of intended outcomes, goals or objectives that the group determined to be most important and fundamental to the group.
• For each statement of intended outcome, the group devised two or more means of assessment, methods of verifying how effectively or successfully the statement of intended outcome was achieved by the unit or department.

Readers are invited to see the “department or unit outcomes and means of assessment” site to see Dixie College’s statements of intended outcome and planned means of assessment.

CRITERIA: The statements of intended outcome that departments and units have submitted are evaluated according to the following criteria:

• The statement has an obvious link to the mission statement.
• The statement is based on outcomes, not processes.
• The statement is written at an appropriate level of specificity.
• The statement is reasonable, given the ability of students or the current situation.
• The statement is clear and accomplishment ascertainable.

The statement has an obvious link to the mission statement.

It is very important that the intended outcomes of individual departments or units within the college have an obvious link to some aspect of the Mission Statement.

• If the Mission Statement says, “Provide quality applied technology education that meets training needs for students desiring certificates and/or Associate of Applied Science Degrees. . .” a department’s Statement of Intended Outcomes might say, “Students of a technical program will find ready employment in the field for which they prepared,” or “Employers of technical program graduates will report that they are pleased with all aspects of the education received by their employees.”
• If the Mission Statement says, “Maintain and care for campus facilities and grounds at the highest levels possible. . .” a unit’s Statement of Intended Outcomes might say, “Students will report that they are ‘ fully satisfied with the cleanliness of grounds and buildings,’” or “Faculty and staff will report that they are ‘ fully satisfied with the cleanliness of grounds and buildings.’”
• If the Mission Statement says, “Provide quality learning resources and student support services, including . . . career counseling, . . .” a unit’s Statement of Intended Outcomes might say, “Students will receive individual or group assistance with career entry skills, including resume preparation and job interview skills, such that they will be fully prepared to make the transition to the world of work.”

The statement is based on outcomes, not processes.

• PROCESS: This type of statement focuses on means for attaining outcomes, not on outcomes themselves -- “We will teach five sections of FCIS 220 each year,” or “Three new technical programs will be added to the current curriculum.”
• OUTCOMES: This type of statement focuses on results of processes, not on the processes themselves: “Students will score at or above the national average on the FCIS Standards Exam.”

When departments and units established their “outcomes,” they were asked to focus on the outcomes, not the processes.

The statement is written at an appropriate level of specificity.

Each department or unit is limited to from three to five statements. Thus, these statements should focus on a small number of outcomes that the departments or units value highly. However, statements should be neither too specific or too general.

For example, if a university’s Education Department were writing statements. . .

• TOO SPECIFIC: “Students will be able to compose true/false statements.”
• TOO GENERAL: “Students will be able to prepare a good test.”
• MORE APPROPRIATE: “Students will be able to utilize alternative forms of evaluation in classroom teaching situations.”

The statement is reasonable, given the ability of students or the current situation.

In the Statements of Intended Outcomes, departments or units should set a benchmark for success beforehand, so that after they have conducted assessment activities, they will have some criteria for judging success. The benchmarks that departments and units set in their statements may be --

• NOT CHALLENGING ENOUGH: “Students will score at an average of the 30th percentile on a national exam.”
• REASONABLE BENCHMARKS: “Students will score at an average of the 75th percentile on a national exam.”
• TOO CHALLENGING: “Students will score at an average of the 96th percentile on a national exam.”

The statement is clear and accomplishment ascertainable.

The statements should be understood by persons within the discipline of the department or unit. Thus, discipline specific jargon is all right, so long persons within that discipline would find the statement clear. The statement should also facilitate assessment. If the statement does not lend itself to ascertaining accomplishment, it should probably be re-drafted.

Year Two (continued) – Devising Means of Assessment:

After the departments or units on campus have composed from three to five Statements of Intended Outcomes, the next step involves designing Means of Assessment – ways to determine if the outcomes are in fact being achieved. For each Statement of Intended Outcomes, the departments or units will design two means of assessment.

Following is a possible example:

Year Three – Implementing Assessments and Planning Improvements:

Having defined their intended outcomes and designed methods of assessment, during the 1996/97 academic year, the departments or units will implement assessment activities and gather data. Readers are invited to visit the “implementing assessments and planning improvements” site to see results, as soon as they become available.

Year Four – The Mission Statement Re-Visited:

The institutional effectiveness cycle begins anew in this year. The mission statement, now three or four years old, will be reviewed and the strategic planning process will undertake a re-drafting of the mission statement, in light of past institutional effectiveness activities. Readers are invited to visit the “mission statement” site to see our present mission statement. As soon as changes are made, new drafts will be posted in this site.

E-Mail: Email to Joe Peterson, Institutional Effectiveness at Dixie College, peterson@dixie.edu