Principles: Standard African Humanities University Curriculum Review

Chair:

Members: 

The Curriculum Review Committee is charged with the task of evaluating the _________________________________ program and curriculum, as well as making recommendations for improvement.

Goals of Evaluation Research in Curriculum Review

1. To ensure that the needs of the population being served by the curriculum are met

• a. Monitor and improve performance
• b. Assess accountability, efficiency, appropriateness of the curriculum
• c. Assess the extent to which the curriculum addresses the needs of the student population and area-wide community
• d. Awareness evaluation. Who knows about the program? What do they know?
• e. Quality assurance. Are minimum and accepted standards being routinely and systematically provided? How can quality be monitored and demonstrated?
• f. Criterion-referenced evaluation. Extent to which specific objectives have been achieved at the desired level of attainment (i.e. at the criterion).

1. To evaluate the quantity of resources consumed by the Department of _________________________________ and/or to quantify the consequences of curriculum delivery
1. Used to evaluate cost of instruction, departmental events and activities, and supplies
2. Types of economic evaluations to be undertaken in this curriculum:

• i. Cost-effectiveness - Relation between program costs and benefits (benefits expressed in non-monetary terms);
• ii. Cost-benefit - the benefits in terms of expense, are compared as to their worth (benefits expressed in monetary terms); and
• iii. Cost-utility - the states of health associated with outcomes are valued relative to one another.

1. To understand and improve operations and the delivery of the curriculum (process focused)
1. Understand

• i. Use qualitative measures that describe the views, feelings, and perceptions of the persons involved (evaluate systems, policies, and services)
• ii. Observational studies
• iii. Case-controlled studies identify persons, populations, or organizations that exhibit a specific characteristic and compare the outcomes of this group with groups of persons, populations, or organizations that do not exhibit these characteristics. (UKMT _________________________________ majors with majors at other 4-year academic institutions)

1. Improve

• i. Findings used to improve the curriculum and/or the way in which interventions will be used in the future
• ii. Assess an intervention while it is being developed

Process in Carrying Out Curriculum Review

1. Identify the Goals of the Curriculum Review:
1. To ensure that the needs of the population being served by the curriculum are met
2. To evaluate the quantity of resources consumed by the Department of _________________________________ and/or to quantify the consequences of curriculum delivery
3. To understand and improve operations and the delivery of the curriculum (process focused)
2. Identify the Objectives of the Curriculum Review (including _________________________________ outcomes and indicators)
3. Identify Review outcomes, indicators, and benchmarks
4. Define measures (data) to satisfy outcomes, indicators, and benchmarks
5. Decide on important objectives; Curriculum Review Committee Members rate the importance of each of the potential objectives
6. Assess what is currently available to meet the important objectives; in some cases this can be combined with the previous step by having committee members rate current availability, and if possible give information about how needs are currently being met.  Alternatively, information can be sought from records or through a small-scale survey.
7. Select final set of objectives.  Judgment must now be exercised in selecting a set of objectives which rate high on importance and low on current availability.  Other criteria such as feasibility may also be incorporated. 
8. Define the study (review process)
1. Determine and describe what needs to be evaluated; exclude what will not be evaluated
2. Identify factors that may influence the effectiveness of the evaluation; and

Research design

1. Data Selection (Characteristics of Data): 
1. Identify data that will satisfy the goals and objectives of the curriculum review
2. Identify the sources from which the data will be gathered and the people/departments/offices from whom data will be gathered
2. Data Measurement
1. Identify data measures In assessing a measure, it is important to consider its utility.  That is, the information it provides should be helpful in the various phases of helping: the assessment of a problem, the monitoring of the intervention process, and the evaluation of evaluation outcomes.
3. Validity and Reliability
1. Validity

• i. Does the process enable the Committee members to satisfy objectives, outcomes, benchmarks, and indicators?
• ii. Are the measures objective?

1. Reliability

• i. Can the review process be repeated utilizing the same methods and generate the same results?
• ii. Is the process objective?

1. Data Collection
1. Identify data collection forms and instruments, if they already exist; if they do not, they must be devised

• i. Include elements necessary to collect data likely to address the objectives (outcomes, indicators, and benchmarks) into a user friendly format
• ii. Design instruments to produce data in a format that is compatible with computer entry coding

1. Establish validity of measurement instruments/forms; this allows the identification of elements or questions that are not relevant to answer the question and may identify data elements that need to be added.

• i. Devise generalized data collection forms that can be modified to fit the exact circumstances and questions being studied
• ii. Include data collection instruments in published reports

1. Gather appropriate data
1. Data Analysis
1. Analysis methods and techniques are determined by the design of the review process
2. For purposes of this review, the analysis will consist largely of descriptions of nominal and categorical level data, presented in the form of tables or figures

Findings

Information usually is described as the findings or results of the study. In the process, the data are organized and converted into information framed to satisfy the objectives that the Curriculum Review was designed to answer. The methods used were outlined during the research design process. Additionally, there may be data that have been collected during the data gathering process that may offer information that is outside of the boundaries initially established in the design process. This may be true particularly in the process of data collection, as some of the questions asked during the interview process or collected during a survey process may have been modified during data collection.

Efforts should be made that lead to institutionalization of indicators.

When possible, cost (economic, social, and emotional) should be addressed. The costs associated with the project should be summarized in a manner to address the efficiency with which the study was conducted.

There are many more aspects to data analysis than just examination of those data that relate directly or indirectly to the questions posed. This information should not be lost.

Conclusions

Conclusions consist of the interpretation of the results of the study in a similar way as do the discussion and conclusion sections of a journal article. At the very least, the conclusions must address the Review goals and objectives, as well as satisfy the outcomes.

If the goals and/or objectives could not be met, the reasons for not obtaining them should be defined, and suggestions as to how future curriculum reviews should be conducted that more likely would provide answers to the questions. The answers to the questions posed at the beginning of the project and the conclusions drawn from the findings must have implications for the conduct and evaluation of future responses.

The limitations to the Curriculum Review also should be defined, with an aim towards avoiding these problems in future efforts.

Of substantial importance in framing conclusions is what the findings mean in relation to the vision and mission of the department and the College.  Do they contribute to the realization of University of Kmt' vision and mission, as well as those of the Department of _________________________________? Do they not contribute to or are they not in alignment with the College's vision and mission?

Similarly, conclusions should be generated that address each of the additional elements and factors listed in the sections above. Each has particular importance in furthering curriculum review efforts and understanding.

Recommendations

The final element of a Curriculum Review consists of recommendations for future actions.  The recommendations resulting from the study should suggest how the finding of the evaluation can be applied to continuous improvement cycles.  Recommendations should not be a rehash of the conclusions. Thus, positive aspects of the findings should be reinforced and negative aspects (responses that could be improved as a result of the study) should lead to suggestions for improvement.

In addition, recommendations should include advice for future evaluators in terms of how the curriculum review process study could have been improved, what questions remain that would address the same or similar problems, and suggestions offered as to how these (new) questions potentially could be answered. Recommendations always should address the indicators selected in terms of their utility and suggested modifications. This includes identification or modification/adaptation of thresholds.

• 1. Recommendations should
• a. be clearly derived from the data; and
• b. be practical (i.e. capable of implementation)
• 2. Distinguish between
• a. Findings: information about the situation;
• b. Interpretations: interpretations (explanations offered about the findings). Interpretations are necessarily specula­tive, and may be the subject of dispute.
• c. Judgments: values brought to bear on the data. Criteria for the making of judgments should be made explicit.
• d. Recommendations: suggested courses of action. The reasonableness of a recommendation depends on its being logically backed up by a set of findings, a reasonable interpretation of the findings and a judgment applied to them. Resolution should be sought through discussion, and returning to the findings, to ensure that inter­pretations are grounded in these findings.
• i. The process of generating recommendations takes time. A carefully carried out evaluation can be ruined by seeking to produce recommen­dations under severe time pressure. There should be the opportunity to discuss provisional recommendations.
• ii. Consider presenting recommendations as a set of options. Given a list of findings, there may be several reasonable interpretations and different value positions which will generate a whole range of options. Each should be fully documented, showing fairly how the findings, interpretations and judgments support the option.
• iii. The nature of recommendations should be negotiated with deci­sion-makers, or whomever forms the audience for the report, at an early stage. This will have implications for the kind of evaluation carried out and the questions addressed.

Curriculum review

A quality educational program must

• be consistent with its institution's mission,
• have clearly defined outcomes it intends to produce,
• use the best combination of learning experiences to help each learner achieve these results,
• include an assessment process that shows whether the results are being achieved, and
• use the findings of assessment to improve program effectiveness.

An approach to continuous program improvement that asks the right questions can provide academic administrators, faculty members, and others with the information they need to develop an appropriate, effective, and efficient academic program. The focus here is on undergraduate programs, but identical principles apply to curricula at the graduate level as well.

Listed below are a number of key questions to ask when reviewing curricula. Most of them are germane whether a curriculum is in general education or a specialized field. Although designed for reviewing curricula that already exist, many of these questions also can be helpful when beginning to design a new curriculum.

Asking these questions can help ensure the maximal learning possible with available resources and minimal waste.

A. Being clear about purpose and desired results: Mission statements, goals, and objectives

1. Is the curriculum consistent with and does it naturally flow from the institution's or unit's mission statement?
2. What assumptions have been made about entering students' developmental levels, knowledge, skills, and affective characteristics - all important inputs for the curriculum and each course. Have entering students been carefully assessed to ensure these assumptions are correct?
3. Does the curriculum have a formal set of intended learning outcomes that articulate the knowledge, skills, attitudes, and values it proposes to introduce or reinforce and that every student should have achieved upon graduation?
4. Are these intended outcomes written in specific language that is understandable in the same way to students, faculty members, and all other users?
5. Are the intended outcomes stated in terms of effective goals and objectives that permit assessment of students' success in achieving them?
6. When identifying and developing these intended outcomes, was there appropriate input from all concerned stakeholders, depending on the type of curriculum, such as faculty members, professionals in various fields, employers, and alumni?
7. Have agreed-upon intended learning outcomes been identified for each of the major areas within the curriculum - for the humanities, natural and social sciences, and fine arts within general education or for a major field curriculum within the disciplines?
8. Does each course have a set of clearly stated intended outcomes derived from the intended outcomes of the curriculum?
9. Do these outcome goals and objectives prominently include higher-order cognitive and other complex behaviors as appropriate?
10. Will achieving each course's intended outcomes materially contribute to learners' achieving the outcomes of the curriculum in a deliberate and predetermined way?
11. When the intended outcomes of all the curriculum's courses are considered together, will every student have ample opportunity to achieve each of the specific intended outcomes of the curriculum itself?
12. For a major field in which certification or accreditation exists, are all of the outcome goals and objectives required for certification built into the curriculum's intended outcomes; or if requirements are stated in terms of courses, are all of these courses part of the curriculum?
13. Is the curriculum carefully sequenced such that the learning outcomes of prerequisite courses provide all required inputs for successive courses?

B. Monitoring program quality: Knowing and improving actual results

1. Is there an assessment plan that can ensure graduates of the program have the knowledge, skills, attitudes, and values described as intended outcomes of the curriculum?
2. Is the curricular program being assessed as a whole and not merely by assessment of the intended outcomes of each of its individual courses?
3. Are diverse methods of assessment being used as appropriate for each type of learning engaged in and outcome desired?
4. Is there a close alignment between the intended outcomes of each course and the ways in which students are assessed in the course? Are a variety of assessment techniques being used?
5. Are intended outcomes being measured directly in both curriculum and courses - as opposed to surveying students' opinions about their learning - to reveal clearly what graduates know and can do, including their important affective qualities?
6. Are the findings of assessment made public and effectively communicated, as appropriate, to all interested stakeholders in a timely manner and in language they can understand?
7. Specifically, how have faculty members used information generated by assessment to improve the amount of learning produced?
8. Do faculty members collect data from students about their perceptions of and their level of satisfaction with the courses they have taken? Specifically, how is this information being used to improve courses?

C. The education process: Producing learning

1. Are the educational processes employed to help students learn in each course or activity fully consistent with research on learning and student development and thus appropriate for reaching both the course's or activity's specified outcomes and those of the curriculum?
2. Has the curriculum been designed such that each student has the sustained opportunity to apply to important issues, situations, and problems the knowledge, skills, attitudes, and values that have been identified as intended outcomes?
3. What percentage of class time do students spend passively listening to traditional lectures?
4. Are students consistently actively involved in learning, not only in their courses but also through such methods as participating in internships, practical, and work-study and study-abroad programs?
5. Do students understand the purpose, structure, and processes of the curriculum, their responsibilities for learning, and how their progress will be assessed? Is each student helped to understand these things at the beginning of the curriculum and throughout every course?
6. Is the formal academic curriculum specifically linked to noncourse-based opportunities for learning on campus such as orientation, developmental academic advising, the cocurriculum, residence life, and employment?

D. Other Important Considerations

1. Are students able to enroll in both required and elective courses as needed? Are they completing the curriculum in a timely manner? If not, specifically why not?
2. Is the program attracting an adequate number of students to support accomplishment of the institution's or unit's mission and to make it cost-efficient? If not, specifically why not?
3. Are students completing the program and each of its courses at a high rate? If not, specifically why not?
4. Do the dropout or failure rates in the curriculum as a whole and in each of its courses indicate a problem and, if so, has the problem been identified and is it being appropriately addressed?
5. Do graduates find appropriate employment in their major fields after graduation? If not, specifically why not?