This blog is for educators, academicians, students and those who are interasted to integrate technology in class room.

Archive for the ‘Curriculum Design and Development’ Category

Curriculum Design and Development : A Scientific Process

The ADDIE Model

ADDIE is an acronym for Analysis, Design, Development, Implementation, and Evaluation. This model guides you through the process of creating effective educational courses and materials for your audience. While there are variations of this model in the industry, the concepts are the same. As a professional, this model is more than just an acronym. It is a blue print for success.

The ADDIE model is the blue print for instructional designers.

The Analysis is the most important step in the process. It helps you to determine the basis for all future decisions. A mistake that many beginners make is not conducting a proper analysis at the beginning. It is this analysis that helps you identify your audience, limitations or opportunities, or other important points that will be useful in the design process.

The Design process is the brainstorming step. This is where you use the information obtain in the Analysis phase to create a program or course that meets the needs of your customer or audience. There are many forms of the design process and it can be very tedious at times. Testing your concepts in the design phase will save you time and money.

The Development phase focuses on building the outcome of the design phase. This process consumes much of the time spent in creating a sound educational program or course. It includes various steps such as initial drafts, reviews, re-writes, and testing. For larger corporations, this phase can involve numerous individuals to include subject matter experts (SME), graphic artists, and technical experts. For elearning courses, this phase could require additional assistance for managing server space and technology.

The Implementation phase includes more processes than simply presenting the materials developed. While the concepts and materials have been tested throughout the process, the implementation phase can uncover topics that require further development or re-design work. The processes for this phase vary based on the size of the organization, the complexity of the program or course, and the distribution of the materials. This includes such concepts as test pilots, train-the-trainer sessions, and other delivery methods to present the materials.

The Evaluation phase plays an important role in the beginning and at the end of the process. Evaluation objectives reflect much of the discoveries found in the Analysis process. These discoveries include the objectives and expectations of the learner. When looking at the process, you must avoid the thought that it is structured in a chronological order. Rather, the ADDIE Model is a continuous circle with overlapping boundaries. Of all of the process phases, the evaluation phase is the lest understood.

Source:http://www.instructionaldesignexpert.com/addie.html

Introduction to Instructional Design

ADDIE Model | ADDIE Phases | Other Models | More Info

Technology is defined as applied science.  Engineering is the technology of applying physics to the design of buildings, chemicals, electrical systems, etc..  Instructional design, in comparison, is the technology of applying learning and instructional theory to the design of quality instruction.s

Quality of instruction is often measured along three dimensions: effectiveness, efficiency, and cost.  The effectiveness of the instructions deals with how well did the instruction enable learners to achieve the expected outcomes.  Efficiency has to do with time and energy expended to complete the instruction.  Cost of the instruction encompasses all expense incurred throughout the design and delivery of the instruction.

Since many faculty are not formally trained in instructional design principles, the purpose of these tutorials is to help fill the gap between being a subject matter expert and being a teacher.

The remainder of this module will provide a primer to instructional design principles to lay a foundation to draw on as we move towards implementing these principles in the design of an online course.


Basic Design Model: ADDIE (return to top)

Instructional design uses a systematic approach to teaching. Design models are used to represent the processes involved in designing instruction.  While there are many design models, and the list is continually growing, most instructional models are a derivative of the ADDIE model in that they include such core elements as analysis, design, development, implementation, and evaluation (ADDIE).

imgThese unifying elements ensure congruence among goals, strategies, evaluation, and the effectiveness of the resulting instruction.

While the core elements of the ADDIE model remain constant, the ADDIE activities typically are not organized in a linear, step-by-step manner. Instead, the instructional design process is iterative and self-correcting in nature (Gustafson & Branch, 2002). Experienced faculty members can use a modified ADDIE model in their courses.

Note however that two elements of the ADDIE, analysis and evaluation, are constantly omitted in design and training process for a variety of reasons such as lack of time and lack of awareness. Literature indicates that doing an analysis (needs assessment) and evaluation (reflection and improvement), even simple ones, can contribute significantly to your course design.


Phases of the ADDIE Model (return to top)

img2 Analysis

  • Analyze student characteristics
    • Prerequisite knowledge
    • Online experience
    • Etc.
  • Analyze learning context (being online, what is and is not reasonable to do in this course?)
    • Learners’ bandwidth
    • Resources available to learners for completion of assignments, etc.
  • Analyze content to be learned
  • Define learning goals and objectives

Design

  • Decide on the main instructional strategies (how will learners interact with content?)
  • Design an assessment plan
  • Create detailed course outline

Development

  • Construct a detailed syllabus
  • Develop and/or acquire course materials
  • Develop assignment/project resources
  • Develop assessment items (traditional exams, rubrics for projects, papers, or portfolios, etc.)

Implementation

  • Learners complete the course as designed
  • Learners are assessed according by the designed assessments
  • Instructor monitors, guides, and facilitates as learners complete the course

Evaluation

  • Formative- instruction effectiveness is monitored while being taught
    • Adjust instructional strategies according to students’ interaction with the content, the instructor, and the peers
  • Summative- overall course effectiveness is measured at course completion
    • Collect course feedback via online survey, email, or other media.
    • Validate content accuracy and completeness, teaching methods, and communication approach, among others.
    • Revise as necessary

References:
Gustafson, K. K., & Branch, R. M. (1997). Survey of instructional development models (3rd ed.).
Syracuse University, NY: ERIC Clearinghouse on Information Resources. 


Other Instructional Design Models (return to top)

See if you can identify the ADDIE elements in these instructional models:

img4
img6
img8

* The graphics in this page are from United States Military Academy at West Point.

 Source:http://itle.okstate.edu/fd/online_teaching/intro_to_id.html

Assessment Primer: Goals, Objectives and Outcomes

Outcomes Pyramid

The assessment literature is full of terminology such as “mission”, “goals”, “objectives”, “outcomes”, etc. but lacking in a consensus on a precise meaning of each of these terms.  Part of the difficulty stems from changes in approaches to education – shifts from objective-based, to competency-based, to outcomes-based, etc. education have taken place over the years with various champions of each espousing the benefits of using a different point of view.  The Outcomes Pyramid shown below presents a pictorial clarification of the hierarchicalrelationships among several different kinds of goals, objectives, and outcomes that appear in assessment literature.

Outcomes Pyramid Image

The ‘pyramid’ image is chosen to convey the fact that increasing complexity and level of specificity are encountered as one moves downward.  The pyramid structure also reinforces the notion that learning flows from the mission of the institution down to the units of instruction.

Outcomes Pyramid Definitions

Mission Statements of the University, School/College, and Program

Mission Statement is a general, concise statement outlining the purpose guiding the practices of an institution or school/college.  Accrediting bodies expect that student learning outcomes flow from the mission statements of the institution and school/college; i.e., the school/college mission should be in harmony with the mission statement of the institution.

See “How To Write Missions” for more detail (2 page  pdf document)

Goals of the Program (or Department)

Goals are broad, general statements of what the program, course, or activity intends to accomplish.  Goals describe broad learning outcomes and concepts (what you want students to learn) expressed in general terms (e.g., clear communication, problem-solving skills, etc.)  Goals should provide a framework for determining the more specific educational objectives of a program, and should be consistent with the mission of the program and the mission of the institution.  A single goal may have many specific subordinate learning objectives.

See “How To Write Goals” for more detail (2 page  pdf document)

Objectives

Goals and Objectives are similar in that they describe the intended purposes and expected results of teaching activities and establish the foundation for assessment.  Goals are statements about general aims or purposes of education that are broad, long-range intended outcomes and concepts; e.g., “clear communication”, “problem-solving skills”, etc.  Objectives are brief, clear statements that describe the desired learning outcomes of instruction; i.e., the specific skills, values, and attitudes students should exhibit that reflect the broader goals.

There are three types of learning objectives, which reflect different aspects of student learning:

  • Cognitive objectives: “What do you want your graduates to know?
  • Affective objectives: “What do you want your graduates to think or care about?
  • Behavioral Objectives: “What do you want your graduates to be able to do?

Objectives can also reflect different levels of learning:

  • Mastery objectives are typically concerned with the minimum performance essentials – those learning tasks/skills that must be mastered before moving on to the next level of instruction.
  • Developmental objectives are concerned with more complex learning outcomes – those learning tasks on which students can be expected to demonstrate varying degrees of progress.

Instructional Objectives describe in detail the behaviors that students will be able to perform at the conclusion of a unit of instruction such as a class, and the conditions and criteria which determine the acceptable level of performance.

What are the differences between Goals and Objectives?  Both goals and objectives use the language of outcomes – the characteristic which distinguishes goals from objectives is the level of specificity.  Goals express intended outcomes in general terms and objectives express them in specific terms.

See “How To Write Objectives/Outcomes” for more detail (6 page pdf document)

Outcomes

Learning Outcomes are statements that describe significant and essential learning that learners have achieved, and can reliably demonstrate at the end of a course or program.  Learning Outcomes identify what thelearner will know and be able to do by the end of a course or program – the essential and enduring knowledge, abilities (skills) and attitudes (values, dispositions) that constitute the integrated learning needed by a graduate of a course or program.

The learning outcomes approach to education means basing program and curriculum design, content, delivery, and assessment on an analysis of the integrated knowledge, skills and values needed by both students and society.  In this outcomes-based approach to education, the ability to demonstrate learning is the key point.

What are the differences between Objectives and Outcomes?  Objectives are intended results or consequences of instruction, curricula, programs, or activities.  Outcomes are achieved results or consequences of what was learned; i.e., evidence that learning took place.  Objectives are focused on specific types of performances that students are expected to demonstrate at the end of instruction.  Objectives are often written more in terms of teaching intentions and typically indicate the subject content that the teacher(s) intends to cover.  Learning outcomes, on the other hand, are more student-centered and describe what it is that the learner should learn.

An effective set of learning outcomes statements informs and guides both the instructor and the students:

For teaching staff:  It informs:

  • the content of teaching
  • the teaching strategies you will use
  • the sorts of learning activities/tasks you set for your students
  • appropriate assessment tasks
  • course evaluation.

For students: The set of learning outcomes provides them with:

  • a solid framework to guide their studies and assist them to prepare for their assessment
  • a point of articulation with graduate attributes at course and/or university (i.e. generic) level.

Learning Outcome statements may be broken down into three main components:

  • an action word that identifies the performance to be demonstrated;
  • learning statement that specifies what learning will be demonstrated in the performance;
  • a broad statement of the criterion or standard for acceptable performance.

For example:

ACTION WORD
(performance)
LEARNING STATEMENT
(the learning)
CRITERION
(the conditions of the performance demonstration)
Produces documents using word processing equipment
Analyzes global and environmental factors in terms of their effects on people
Examples of Goals, Objectives, and Outcomes
Goal Objective How this objective might be reformulated as a Learning Outcome
(Geology)  To develop knowledge, understanding and skills related to the recognition and interpretation of igneous and metamorphic rocks. To explain the different magma geochemistries derived from partial melting of the mantle in different tectonic regime. Students should be able to demonstrate how magma geochemistry relates to partial melting of the mantle by contrasting the outcomes of this process in different tectonic regimes through the critical analysis of specific case studies.
(Biochemistry) To explain the biochemical basis of drug design and development. To demonstrate the application of molecular graphics to drug design. Students should be able to apply the principles underpinning the use of molecular graphics in the design of drugs to illustrate general and specific cases through a computer-based presentation.
(English)  To introduce students to modes of satiric writing in the eighteenth century. To familiarize students with a number of substantive eighteenth century texts.  Students will be trained in the close reading of language and its relation to literary form. Students should be able to analyze the relationship between the language of satire to literary form by the close examination of a selected number of eighteenth-century texts in a written essay.
(Engineering) This course introduces senior engineering students to design of concrete components of structure and foundation and the  integration of them into overall design structures. The student is able to function in teams. Functioning as a member of a team, the student will design and present a concrete structure which complies with engineering standards.
(Geology) Become acquainted with topographic maps and their usage. Use topographic maps and employ these maps to interpret the physiography and history of an area. Students should be able to
  • Locate and identify features on topographic maps by latitude and longitude and township and range.
  • Contour a topographic map and construct a topographic profile.
  • Identify major landform features on topographic maps and relate them to basic geologic processes of stream, groundwater, glacial or marine erosion and deposition.
  • Interpret geologic maps and geologic cross-sections.

More Examples of Mission, Goals, Objectives and Outcomes

An example based on material from Eastern Kentucky University Social Work program (2 page pdf document)

An example based on material from California State University, Sacramento Gerontology program (1 page  pdf document)

An example based on material from IUPUI Mechanical Engineering which takes the step of defining Measurable Outcomes (2 page  pdf document)

Translating Course Goals Into Measurable Student Outcomes

Assessment can measure the extent to which course goals have been achieved, but only if those goals aremeasurable.  For the most part, course goals are too broad or too abstract to measure directly.  Once goals have been formalized, the next step is to translate the often abstract language of course goals into a set of concrete measurable student outcomes.

Measurable Outcomes Diagram

Example: Lack of specificity of Goals

Introductory Astronomy Course Goal = Students understand the seasons.How does one measure ‘understand’? No idea! This Goal can be made more measurable by identifying specific Outcomes one would expect from a student who “understands” the seasons.Course Outcomes =  The student can define seasons
The student can distinguish the importance of different factors such as tilt and distance.

Measurable student outcomes are specific, demonstrable characteristics – knowledge, skills, values, attitudes, interests–that will allow us to evaluate the extent to which course goals have been met.

Example: translating a course goal (in the context of dental health) into measurable student outcomes

Dental Health 101

Course Goal

Measurable Student Outcomes

The Student:

  • Understands proper dental hygiene
The Student can:

  • Identify the active ingredient in toothpaste
  • Explain why teeth should be cleaned at least twice per year
  • Describe how poor dental hygiene can lead to poor overall health
Example: showing a link between objectives and assessment.

Example: Refining a Goal into Measurable Objectives

Goal: Students will be familiar with the major theories of the discipline.Does this goal convey any information?

  • Would a student know what was expected of his/her work?
  • Would a colleague know the focus of your department’s teaching?
  • Would an employer know what your students could do?

Refining the goal into a
measurable objective

Explanation of the process

Students will be familiar with the major theories of the discipline Objective = verb (active behaviors)
+
object (products, skills/performances, content/knowledge, attitudes/dispositions)Objective = (be familiar with) + (major theories of the discipline)Start with the object aspect of the objective.  Suppose five major approaches (theories) to conflict resolution are: withdrawal, smoothing, forcing, compromising, and problem solving.
Students will be familiar with withdrawal, smoothing, forcing, compromising, and problem solving Specifying what the department views as the major approaches (theories) is an improvement in the wording of the objective.
Students will be familiar with withdrawal, smoothing, forcing, compromising, and problem solving Sharpening the verb will also make it better – what does “be familiar with” imply about a student’s knowledge or skills?Objective = (be familiar with) + (withdrawal, smoothing, forcing, compromising, …)

  • Avoid vague phrases:
    appreciate, understanding, have an awareness of, etc.
  • Use action verbs:
    generalize, produce, evaluate, etc.

Action oriented verbs make objectives more concrete

This objective might be revised into two objectives

  • Students will summarize …
  • Students will choose and defend …
Objectives obtained through the revision of the original Goal:

  • Students will summarize the five major approaches to conflict resolution: withdrawal, smoothing, forcing, compromising, and problem solving
  • Students will choose and defend a conflict resolution approach appropriate for a given situation
Checklist to Review Program-Level Outcome Statements

Checklist to Review Program-Level Draft of Learning Outcome Statements
(Based on Assessing for Learning: Building a sustainable commitment across the institution by Maki 2004)

Outcome #1 Outcome #2 Etc.
Describes what students should represent, demonstrate, or produce?
Relies on active verbs?
Aligns with collective intentions translated into the curriculum and co-curriculum?
Maps to curriculum, co-curriculum, and educational practices?
Is collaboratively authored and collectively accepted?
Incorporates or adapts professional organizations’ outcome statements when they exist?
Can be assessed quantitatively and/or qualitatively?

Assessment Primer:
Learning Taxonomies

Beginning in 1948, a group of educators undertook the task of classifying education goals and objectives.  The intention was to develop a classification system for three domains:

  • Cognitive domain (intellectual capability, mental skills, i.e., Knowledge)
  • Affective domain (growth in feelings, emotions, or behavior, i.e., Attitude)
  • Psychomotor domain (manual or physical skills, i.e., Skills)

This taxonomy of learning behaviors can be thought of as the goals of training; i.e., after a training session, the learner should have acquired new skills, knowledge, and/or attitudes.

Cognitive Domain – Bloom’s Taxonomy

Work on the cognitive domain was completed in 1956 and is commonly referred to as Bloom’s Taxonomy of the Cognitive Domain, since the editor of the volume was Benjamin S. Bloom, although the full title was Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain, 1956 by Longman Inc. with the text having four other authors (Max D. Engelhart, Edward J. Furst, Walker H. Hill, and David R. Krathwohl).

Bloom identified six levels within the cognitive domain, from the simple recall or recognition of facts, as the lowest level, through increasingly more complex and abstract mental levels, to the highest order which is classified as evaluation.

Six Levels Diagram

A description of the six levels is given here (1 page  pdf document).

Bloom, et al indicated …

“[Bloom’s] Taxonomy is designed to be a classification of the student behaviors which represent theintended outcomes of the educational process.  It is assumed that essentially the same classes of behavior may be observed in the usual range of subject-matter content of different levels of education (elementary, high school, college), and in different schools.  Thus a single set of classification should be applicable in all these circumstances.

What we are classifying is the intended behaviors of students – the ways in which individuals are to think, act or feel, as a result of participating in some unit of instruction.  (Only such of those intended behaviors as are related to mental acts of thinking are included in the part of the Taxonomy developed in the handbook for the cognitive domain.)

It is recognized that the actual behaviors of the students after they have completed the unit of instruction may differ in degree as well as kind from the intended behavior specified by the objectives.  That is the effects of instruction may be such that the students do not learn a given skill to any degree.

We initially limited ourselves to those objectives referred to as knowledge, intellectual abilities, and intellectual skills.  (This area, which we named the cognitive domain, may also be described as including the behavior; remembering; reasoning, problem solving; concept formation, and to a limited extent creative thinking.)”

In essence, the authors foreshadowed what has come to be known as outcomes-based assessment(Assessment in Higher Education by Heywood 2000)

Examples of learning objectives at each of the Bloom levels:

Example of Learning Objectives at each of the levels of Bloom’s taxonomy

(based on Assessment in Higher Education by Heywood 2000 and
Eder, Douglas J., “General Education Assessment Within the Disciplines”,
The Journal of General Education, Vol. 53, No. 2, pp. 135-157, 2004 )

Bloom’s level Learning goal: Students will understand the major theoretical approaches within the discipline
Knowledge Students can list the major theoretical approaches of the discipline
Exam question at this level: Name the muscles of the rotator cuff.
Medical faculty questions at this level: What was the heart rate?  Where is the primary lesion?
Comprehension Students can describe the key theories, concepts, and issues for each of the major theoretical approaches
Exam question at this level: How does the rotator cuff help you to raise your arm?
Medical faculty questions at this level: When would you use that type of hernia repair?  Why is the fracture in the same place it was before?
Application Students can apply theoretical principles to solve real-world problems
Exam question at this level: Why does throwing a curve ball cause rotator cuff injury?
Medical faculty questions at this level: You are watching the patient and she falls – what would you do?  Here is a lady with no vibratory sensation – what problem does this pose?
Analysis Students can analyze the strengths and limitations of each of the major theoretical approaches for understanding specific phenomena
Exam question at this level: How does the throwing motion stress each component, in turn, of the rotator cuff?
Medical faculty questions at this level: What are the most significant aspects of this patient’s story?  That is a curious bit of information – how do you explain it?
Synthesis Students can combine theoretical approaches to explain complex phenomena
Exam question at this level: Design a physical therapy program to strengthen each component of the rotator.
Medical faculty questions at this level: How would you summarize this?  What are your conclusions?
Evaluation Students can select the theoretical approach that is most applicable to a phenomenon and explain why they have selected that perspective
Exam question at this level: Evaluate another physical therapist’s program to strengthen the rotator cuff.
Medical faculty questions at this level: Why is that information pertinent?  How valid is this patient’s story?

The following graphics depict how courses in a curriculum reflect Bloom’s levels. Namely, the higher levels of learning are addressed in advanced course work taken by students.

Bloom Lower Division Image

Bloom Upper Division Image

Affective Domain – Krathwohl’s Taxonomy

Bloom’s Taxonomy second domain, the Affective Domain, was detailed by Bloom, Krathwhol and Masia in 1964 (Taxonomy of Educational Objectives: Volume II, The Affective Domain).  Bloom’s theory advocates this structure and sequence for developing attitude – also now commonly expressed in personal development as ‘beliefs’.

Affective Domain Taxonomy Image

Krathwohl’s affective domain taxonomy is perhaps the best known of any of the affective taxonomies. A description of the levels is given here (1 page  pdf document ).

Psychomotor Domain

Various people have since built on Bloom’s work, notably in the third domain, the ‘psychomotor’ or skills, which Bloom originally identified in a broad sense, but which he never fully detailed.  This was apparently because Bloom and his colleagues felt that the academic environment held insufficient expertise to analyze and create a suitable reliable structure for the physical ability ‘Psychomotor’ domain.  As a result, there are several different contributors providing work in this third domain, such as Simpson and Harrow which are described below.

Harrow’s Taxonomy

The psychomotor domain taxonomy due to Harrow is organized according to the degree of coordination including involuntary responses as well as learned capabilities.  Simple reflexes begin at the lowest level of the taxonomy, while complex neuromuscular coordination make up the highest level.

Psychomotor Taxonomy Image

Simpson’s Taxonomy

The psychomotor domain includes physical movement, coordination, and use of the motor-skill areas.  Development of these skills requires practice and is measured in terms of speed, precision, distance, procedures, or techniques in execution.  Simpson’s seven major categories listed from the simplest behavior to the most complex are shown here (1 page  pdf document) . Source:http://assessment.uconn.edu/primer/taxonomies1.html

Assessment Primer:
Writing Instructional Objectives

(Based on Preparing Instructional Objectives by Mager 1962 and Preparing Instructional Objectives: A critical tool in the development of effective instruction by Mager 1997)

An objective

  • Is an intent communicated by a statement describing a proposed change in a learner
  • Is a statement of what the learner is to be like when he/she has successfully completed a learning experience

An instructional objective describes an intended outcome.  A usefully stated objective is stated in behavioral, or performance, terms that describe what the learner will be doing when demonstrating his/her achievement of the objective.  An instructional objective must

  • Describe what the learner will be doing when demonstrating that he/she has reached the objective; i.e.,

What should the learner be able to do? (Performance)

  • Describe the important conditions under which the learner will demonstrate his/her competence; i.e.,

Under what conditions do you want the learner to be able to do it? (Conditions)

  • Indicate how the learner will be evaluated, or what constitutes acceptable performance; i.e.,

How well must it be done? (Criterion)

Course objective:

  • What a successful learner is able to do at the end of the course
  • Is a description of a product, of what the learner is supposed to be like as a result of the process

The statement of objectives of a program must denote measurable attributes observable in the graduate of the program; otherwise it is impossible to determine whether or not the program is meeting the objectives.  Tests or examinations are the milestones along the road of learning and are supposed to tell the teacher and the student the degree to which both have been successful in their achievement of the course objectives.

An advantage of clearly defined objectives is that the student is provided the means to evaluate his/her own progress at any place along the route of instruction; thus, the student knows which activities on his/her part are relevant to his/her success.  A meaningfully stated objective is one that succeeds in communicating to the reader the writer’s instructional intent and one that excludes the greatest number of possible alternatives to your goal.

BAD” words
(open to many interpretations)

GOOD” words
(open to fewer interpretations)

To KNOW To WRITE
To UNDERSTAND To RECITE
To ENJOY To IDENTIFY
To APPRECIATE To DIFFERENTIATE
To GRASP THE SIGNIFICANCE OF To SOLVE
To COMPREHEND To CONSTRUCT
To BELIEVE To LIST
To COMPARE
To CONTRAST

The idea is to describe what the learner will be doing when demonstrating that he/she “understands” or “appreciates”.

Steps to write objectives that will describe the desired behavior of the learner:

  1. Identify the terminal behavior or performance by name; i.e., specify the kind of behavior that will be accepted as evidence that the learner has achieved the objective.
  2. Define the desired behavior further by describing the important conditions under which the behavior will be expected to occur.
  3. Specify the criteria of acceptable performance by describing how well the learner must perform to be considered acceptable.
Step [1] Identifying the terminal behavior
Scheme to fulfill Step [1]:
Write a statement describing one of your educational intents and then modify it until it answers the question: “What is the learner doing when he/she is demonstrating that he/she has achieved the objective?”
A useful objective identifies the kind of performance that will be accepted as evidence that the learner has achieved the objective.  An objective always states what a learner is expected to be able to do and/or produce to be considered competent.  Two examples:
Be able to ride a unicycle.         => the performance stated is ride
Be able to write a letter.   => the performance stated is writing, the product is a letter
Performances may be visible, like writing, repairing, or painting; or invisible, like adding, solving, or identifying.  If a statement does not include a visible performance, it isn’t yet an objective.
Overt (visible) performance
To identify the kind of performance associated with the objective, you need to answer the question: What will the learner be DOING when demonstrating achievement of the objective?
Example:
Given all available engineering data regarding a proposed product, be able to write a product profile.  The profile must describe and define all of the commercial characteristics of the product appropriate to its introduction to the market, including descriptions of at least three major product uses.
=> performance = “write a product profile
Covert (invisible) performance
Some performances are not visible to the naked eye, such as solving, discriminating, and identifying.
Statements such as
Be able to solve …
Be able to discriminate …
Be able to identify …
are inadequate because they don’t describe a visible performance.  Whenever the main intent of the objective is covert, you need to add an indicator behavior to reveal how the covert performance can be directly detected.  An indicator behavior is one that tells you whether a covert performance is happening to your satisfaction.
Example:
Consider the covert performance ‘Be able to discriminate counterfeit money’.  An indicator behavior would be for this performance could be to ‘sort the money into two piles’, counterfeit and genuine.  Thus, a suitable objective could be “Be able to discriminate (sort) counterfeit money.”

Examples:

Stated in behavioral terms Stated in performance terms
“To develop an appreciation for music” “The learner correctly answers 95 multiple-choice questions on the history of music”
“To be able to solve quadratic equations”
“To be able to repair a radio” “To be able to write a summary of the factors leading to the depression of 1929”
“To know how an amplifier works”
“To know the rules of football”
Step [2] further defining the terminal behavior
Scheme to fulfill step [2]:
Given an objective and a set of test items or situations, accept or reject each test item on the basis of whether the objective defines (includes) the behavior asked for.  If you must accept all kinds of test items as appropriate, the objective needs to be more specific.  If the objective allows you to accept those items you intend to use and allows you to reject those items you do not consider relevant or appropriate, the objective is stated clearly enough to be useful.
To state an objective that will successfully communicate your educational intent, you will sometimes have to define terminal behavior further by stating the conditions you will impose upon the learner when he/she is demonstrating his/her mastery of the objective.  As a simple example:
(a) “To be able to solve problems in algebra.”
vs. (b) “Given a linear-algebraic equation with one unknown, the learner must be able to solve for the unknown without the aid of references, tables, or calculating devices.”
In (b) we clearly see a more well-defined statement of the conditions under which solving an algebraic equation will occur.
You should be detailed enough to be sure the target behavior would be recognized by another competent person, and detailed enough so that other possible behaviors would not be mistaken for the desired behavior.  You should describe enough conditions for the objective to imply clearly the kind of test items appropriate for sampling the behavior you are interested in developing.
Examples:
“Given a list of 35 chemical elements, be able to recall and write the valences of at least 30.”
Given a list‘  – Tells us something about the conditions under which the learner will be recalling the valences of elements.
at least 30‘ – Tells us something about what kind of behavior will be considered ‘passing’; 30 out of 35 is the minimum acceptable skill.
Given a product and prospective customer, be able to describe the key features of the product.”
The performance is to occur in the presence of a product and a customer; these are the conditions that will influence the nature of the performance, and so they are stated in the objective.
To avoid surprises when working with objectives, we state the main intent of the objective and describe the main condition under which the performance is to occur.  For example, “Be able to hammer a nail …” is different from “Given a brick, be able to hammer a nail …”.
Miscommunications can be avoided by adding relevant conditions to the objective by simply describing the conditions that have a significant impact on the performance – in other words, describe the givens and/or limitations within which the performance is expected to occur.  Some simple examples:
With only a screwdriver …
Without the aid of references …
Given a standard set of tools and the TS manual …

Guiding questions:

  • What will the learner be expected to use when performing (e.g., tools, forms, etc.)?
  • What will the learner not be allowed to use while performing (e.g., checklists or other aids)?
  • What will be the real-world conditions under which the performance will be expected to occur (e.g., on top of a flagpole, under water, in front of a large audience, in a cockpit, etc.)?
  • Are there any skills that you are specifically not trying to develop? Does the objective exclude such skills?

Some simple examples:

(i) Objective: “When asked a question in French, the student must be able to demonstrate his/her understanding of the question by replying, in French, with an appropriate sentence.”
Inappropriate test situations:
“Translate the following French sentences.”
“Translate the following French questions.”
Appropriate test situation:
“Reply, in French, to the following questions
(ii) Objective: “To be able to solve a simple linear equation.”
: Inappropriate test situation
“If seven hammers cost seven dollars, how much does one hammer cost?”
Appropriate test situation:
“Solve for x in the following 2 + 4x = 12
Key point: If you expect the student to learn how to solve word problems, then teachhim/her how to solve word problems. Do not expect him/her to learn to solve word problems by teaching him/her how to solve equations. The only appropriate way to test to see whether they have learned to solve equations (as stated in the objective) is to ask them to solve equations
(iii) Objective: “Given a DC motor of ten horsepower or less that contains a single malfunction, and given a standard kit of tools and references, the learner must be able to repair the motor within a period of 45 minutes.
Test question: “Given a motor with trouble in it, locate the trouble.”
Appropriate (Yes or No)?:
No! The objective asked for repairing behavior rather than locating behavior. ‘Repair the motor’ means to make it work.  Making it work is the desired behavior.  The test item sampled only a portion of the behavior called for by the objective
Step [3] stating the criterion
Scheme to fulfill step [3]:
Ask the following questions of statements used to assess performance:
(a)  Does the statement describe what the learner will be doing when he/she is demonstrating that he/she has reached the objectives?
(b) Does the statement describe the important conditions (givens or restrictions) under which the learner will be expected to demonstrate his/her competence?
(c) Does the statement indicate how the learner will be evaluated? Does it describe at least the lower limit of acceptable performance?
You can increase the ability of an objective to communicate what it is you want the learner to be able to do by telling the learner how well you want him/her to be able to do it.  If you can specify at least the minimum acceptable performance for each objective, you will have a performance standard against which to test your instructional programs; you will have a means for determining whether your programs are successful in achieving your instructional intent.  Indicate in your statement of objectives what the acceptable performance will be, by adding words that describe the criterion of success.
Some examples of ways in which minimum acceptable performance can be specified:
(i) time limit
Ex.: “The student must be able to correctly solve at least seven simple linear equations within a period of thirty minutes.”
(ii) minimum number of correct responses that will be accepted
or number of principles that must be applied
or number or principles that must be identified
or number of words that must be spelled correctly
Ex: “Given a human skeleton, the student must be able to correctly identify by labeling at least 40 of the following bones (list of bones inserted here).”
(iii) indicate the percentage or proportion
Ex.: “The student must be able to spell correctly at least 80% of the words called out to him/her during an examination period.”
(iv) define the important characteristics of performance accuracy
Ex.: “… and to be considered correct, problem solutions must be accurate to the nearest whole number.”
An objective describes the criteria of acceptable performance; that is, it says how well someone would have to perform to be considered competent.  For example,
“Given a computer with word-processing software, be able to write a letter”
could have a criteria of “all words are spelled correctly, there are no grammatical or punctuation errors, and the addressee is not demeaned or insulted”.  Thus, you complete your objective by adding information that describes the criterion for success keeping in mind that if it isn’t measurable, it isn’t an objective.

Questions to answer leading to a useful objective:

  • What is the main intent of the objective?
  • What does the learner have to do to demonstrate achievement of the objective?
  • What will the learner have to do it with or to? And what, if anything, will the learner have to do it without?
  • How will we know when the performance is good enough to be considered acceptable?
Summary
  • A statement of instructional objectives is a collection of words or symbols describing one of your educational intents.
  • An objective will communicate your intent to the degree you have described what the learner will be doingwhen demonstrating his/her achievement and how you will know when he/she is doing it.
  • To describe terminal behavior (what the learner will be doing)
    • Identify and name the overall behavior act.
    • Define the important conditions under which the behavior is to occur (givens or restrictions).
    • Define the criterion of acceptable performance.
  • To prepare an objective
    • Write a statement that describes the main intent or performance expected of the student.
    • If the performance happens to be covert, add an indicator behavior through which the main intent can be detected.
    • Describe relevant or important conditions under which the performance is expected to occur.  Add as much description as is needed to communicate the intent to others.
  • Revise as needed to create a useful objective, i.e., continue to modify a draft until these questions are answered:
    • What do I want students to be able to do?
    • What are the important conditions or constraints under which I want them to perform?
    • How well must students perform for me to be satisfied?
  • Write a separate statement for each objective; the more statements you have, the better chance you have of making clear your intent.

Samples of Professional Association Learning Goals/Objectives/Outcomes

American Psychological Association Learning Goals for Psychology (5 page  pdf document)
American Sociological Association Learning Goals for Sociology (2 page  pdf document)
ABET, Inc. Learning Goals/Outcomes for Engineering (1 page  pdf document)

Samples of Program Learning Objectives From Other Institutions

University of Colorado at Boulder (2 page  pdf document)
University of Illinois at Urbana-Champaign (3 page pdf document)
San Diego State University (3 page  pdf document)

Source:http://assessment.uconn.edu/primer/objectives1.html\

Assessment Primer:
Curriculum Mapping

Plan for designing and delivering learning outcomes:
In designing course outcomes

  • Start first with the broad outcomes expected of all students
  • Then work backward to design academic program outcomes
  • Finally design course outcomes that will lead to the achievement of both program and institutional outcomes

When the program is delivered, students experience the system in reverse

  • Students first participate in experiences that address lesson outcomes
  • The learning that results from these experiences accumulates as students proceed through the courses and other experiences in the program
  • The curriculum is designed so that it provides a coherent set of experiences leading to the development of desired knowledge and skills – students show increasing levels of sophistication and integration of skills as they progress through the program

(Learner-Centered Assessment on College Campuses: shifting the focus from teaching to learning by Huba and Freed 2000)

mapping

Curriculum mapping makes it possible to identify where within the curriculum learning objectives are addressed.  In other words, it provides a means to determine whether your objectives are aligned with the curriculum.

Alignment – the curricula must be systematically aligned with the program objectives.  Alignment involves clarifying the relationship between what students do in their courses and what faculty expect them to learn.  Analyzing the alignment of the curricula with program objectives allows for the identification of gaps which can then lead to curricular changes to improve student learning opportunities.

Approach to determining the alignment of courses with the program objectives – create a matrix:

Curriculum Alignment Matrix
(Assessing Academic Programs in Higher Education by Allen 2004)
Course Program Objective 1 Program Objective 2 Etc.
100 I
101 P
102 D P
103 I D
Etc.
  I = introduced, P = practiced, D = demonstrated

Aligning course objectives to program objectives may be accomplished by a curriculum alignment matrix which maps each onto the other; a checkmark indicating coverage or an indication of the level of coverage can be used.

Similarly, a course alignment matrix may be used to indicate where course objectives support the overall objectives of the program.

Course Alignment Matrix
(Assessing Academic Programs in Higher Education by Allen 2004)
Course Objectives Program Objective 1 Program Objective 2 Program Objective 3 Etc.
Course Objective 1 B
Course Objective 2 B B
Course Objective 3 B
Course Objective 4 I
Etc. A
B= basic, I = intermediate, A = advanced expectation for this objective

Mapping of outcomes to educational experiences may also be done:

Program- or Institution-level Map
(Assessing for Learning: Building a sustainable commitment across the institution by Maki 2004)
Learning Outcomes Course or Educational Experience #1 Course or Educational Experience #2 Etc.
Outcome #1 I
Outcome #2 E R
Outcome #3 R
Outcome #4 I E
Etc.
I = introduced, R = reinforced, E = emphasized

An example outlining the connections between program objectives and courses:

Example of curriculum mapping
(Based on “Defining Outcomes for Programs and Courses”, Higher Learning Commission Workshop
Making a Difference in Student Learning: Assessment as a Core Strategy by Pagano 2005)

Martha Stewart College
Degree: Bachelor of Arts
Major: Party Planning

Program Objectives: All students with a major in Party Planning will be able to:

  • Develop and execute parties for a variety of situations and for diverse clientele.
  • Create complete menus for a variety of events.
  • Demonstrate an understanding of the biochemical properties of foods and liquids.
  • Plan, price, and budget a variety of parties.
  • Develop successful marketing strategies for a party planner.
  • Anticipate and respond to emergencies in parties they are running.
  • Train and manage staff.
Party Planning
Core Courses:
PP 110 Introduction to Party Planning
PP 200 Party Budgeting and Purchasing
PP 201 Fundamentals of Catering
PP 240 Home Decorations
PP 260 Crisis Management
PP 290 Capstone Course/Internship
Details on one of the courses: PP 201: Fundamentals of Catering
By the end of the semester, students should be able to

  • Create and develop a food and beverage menu for a variety of parties
  • Budget and price menus for a variety of parties
  • Develop realistic timelines for delivering and preparing food and ancillary party accoutrements.
  • Demonstrate an understanding of food varieties and appropriateness for different occasions.
  • Make appropriate decisions regarding staffing at a variety of parties.
(Example Continued)Martha Stewart College
Degree: Bachelor of Arts
Major: Party Planning
Program Objectives
#1
Develop and execute parties for a variety of situations and for diverse clientele.
#2
Create complete menus for a variety of events.
#3
Demonstrate an understanding of the biochemical properties of foods and liquids.
#4
Plan, price, and budget a variety of parties.
#5
Develop successful marketing strategies for a party planner.
#6
Anticipate and respond to emergencies in parties they are running.
#7
Train and manage staff.
PP 110
Introduction to Party Planning
I I I
PP 200
Party Budgeting and Purchasing
I P
PP 201
Fundamentals of Catering
D I
PP 240
Home Decorations
P D
PP 260
Crisis Management
I D D
PP 290
Capstone Course
D P P D D
B = basic, I = intermediate, A = advanced expectation for this objective
(Example Continued – Mapping of the objectives of a single course)Martha Stewart College
Degree: Bachelor of Arts
Major: Party Planning
Program Objectives
#1
Develop and execute parties for a variety of situations and for diverse clientele.
#2
Create complete menus for a variety of events.
#3
Demonstrate an understanding of the biochemical properties of foods and liquids.
#4
Plan, price, and budget a variety of parties.
#5
Develop successful marketing strategies for a party planner.
#6
Anticipate and respond to emergencies in parties they are running.
#7
Train and manage staff.
PP 110
Objective #1
B B I
PP 200
Objective #2
B A A
PP 201
Objective #3
B B A
PP 240
Objective #4
I B
PP 260
Objective #5
B A
B = basic, I = intermediate, A = advanced expectation for this objective

Curriculum Mapping Example – Business Program

An example showing the mapping of learning competencies with Business courses (3 page  pdf document)

Curriculum Mapping Example – Engineering Program

An example of a UCLA Engineering program curriculum mapping (2 page  pdf document)

Source:http://assessment.uconn.edu/primer/mapping1.html

Getting Started

Virtually every program already is doing many kinds of informal assessments. Formalizing these activities into a viable and valuable assessment plan can be made relatively easy using the following guidelines.

1. Read all the pages in this section to get an overview of assessment, accountability, and student learning.

2. Participate with your program faculty in brainstorming discussions on these questions:

  • What particular skills, knowledge, or abilities should graduates of your program be able to demonstrate upon graduation?
  • At what levels of expertise should they be able to demonstrate such knowledge, skills, and abilities?
  • As specifically as possible, identify how you can assess whether students have acquired these abilities.

3. Based on the discussions from #2, write a list of specific program learning objectives. Include both discipline-specific learning objectives and across-the-curriculum developmental or integrative objectives.
(See more about writing program objectives)

Wherever possible use verbs to frame learning objectives as specific actions.
Example: “Graduates should be able to explain the impacts of various taxes on the economic decisions of producers and consumers.”
list of “verbs” to use in objective statements (NCGIA)

4. For each learning objective, identify at least one (more are better) actual learning outcome which will be measured or observed to provide evidence of how well the objective has been met by each student.

5. Organize the set of learning objectives around common themes; use these themes to define tentative program goals. In addition to defining discipline-specific goals and objectives, program goals should also reflect the continuing development of Western’s general education learning objectives throughout each major.

6. Integrate program goals into a tentative mission statement.

7. Repeat 3, 4, 5 to integrate mission, goals, and objectives and make them congruent.

At this point you have a mission statementgoals statementlearning objectives, and learning outcomes; what remains is to “close the loop” by establishing procedures and assigning responsibilities for:
a) Measuring actual outcomes and comparing them with intended objectives;
b) Implementing program changes based on assessment results; and
c) Assessing, documenting, and reporting the effectiveness of changes introduced during the previous assessment cycle.

8. Done!

Developing a Program Assessment Plan

Academic departments at Western show considerable variation in levels of development of their assessment programs. Many, especially those forced to establish assessment procedures to meet the professional accreditation requirements of their disciplines, have quite highly developed plans for assessing program outcomes, including especially student learning outcomes. Many others have not had such incentives, and have developed only vestigial assessment plans at best. Even those programs with considerable experience with assessment do not necessarily share a common view of the importance of various learning outcomes or a common format for documenting their assessment activities or reporting their findings.

It is useful to acknowledge this range of experience with program assessment by identifying three stages of development of program assessment plans: beginning, intermediate, and integrated.

The Planning stage is the beginning level of implementation. It is characterized by tentativeness and uncertainty; mission and goals are not clearly defined; program learning objectives are not clearly defined and may not be congruent with goals; outcomes measures are not good estimators of program objectives; assessment data are being collected or analyzed only sporadically; classroom assessment procedures are not congruent with stated program goals; or collected data has either not been analyzed or results have not applied for program improvement.

The Emerging stage is the intermediate level of implementation. It is characterized by familiarity, growing confidence, and growing commitment to assessment; faculty members are increasingly engaged in collecting and applying assessment data; assessment results are increasingly used in decisions about course sequencing, faculty allocations, teaching methods, program curricula, choice of instructional resources, planning and budgeting, and program improvement; and faculty are increasingly engaged in an ongoing conversation about program improvement based on assessment findings.

The Maturing stage is the integrated level of implementation. It is characterized by: continued development of the processes of the “emerging” level, the increasingly important role of student learning and teaching excellence in defining program effectiveness and guiding program changes, and the full engagement of faculty in an active “culture of evidence” dedicated to improving student learning, performance, involvement, and achievement.

Western’s goal is for all academic program assessment plans to evolve to the “maturing” stage. This website is to assist program faculty in the development, implementation, and improvement of unit assessment plans, and to establish a unified annual reporting format which summarizes departmental assessment activities. In addition, staff at the Center for Instructional Innovation and Assessment and the Office of Institutional Assessment are available for assistance.

Mission Statement: Defining What’s Most Important

The Mission Statement is the initial point of reference for any program or course. It is a concise statement of the general values and principles which guide the curriculum. In broad strokes it sets a tone and a philosophical position from which follow a program’s goals and objectives; therefore the mission statement is also a statement of program vision. The mission statement can and should be brief. However, it is not an isolated document. Rather, it is the cornerstone of a the curricular structure , defining the very broadest curricular principles and the larger context in which more specific curricular goals will fit. The program mission statement should define the broad purposes the program is aiming to achieve, describe the community the program is designed to serve, and state the values and guiding principles which define its standards.

Program mission statements must also be consistent with the principles of purpose set forth in the University’s mission and goals statements; therefore, a good starting point for any program mission statement is to consider how the program mission supports or complements the University mission and strategic goals.

Paraphrasing from several versions of Western’s Mission Statement:

The mission of Western Washington University is to provide to Washington State students a high quality undergraduate education which nurtures the intellectual, ethical, social, physical, and emotional development of each student, through:

  1. A common, broad-based mastery of the fundamental concepts, history, perspectives, and significance of the arts, sciences, social sciences, and humanities; and
  2. Baccalaureate and master’s degree major programs of a practical and applied nature directed to the educational, economic, and cultural needs of Washington State residents.

These mission elements are further elaborated in Western’s Strategic Plan, which emphasizes three broad goals of educational quality, multicultural enrichment, and community service.

The program mission statement must serve as a link between departmental goals and objectives on the one hand, and University mission and goals on the other; it must also demonstrate logical internal consistency among program mission, goals, objectives, and outcomes.

As a result, writing the mission statement is an iterative process of successive approximations:

  • first approximation of mission
  • first approximation of goals
  • first approximation of objectives
  • second approximation of mission, etc.

Therefore, in the initial stages of mission development, a rough listing of the main purposes of a program, and how it fits into the larger mission and goals of the University, might be adequate before moving on to first approximations of program goals and objectives.

Program Goals: Focusing the Mission Statement

The main function of the goals statement is to form a bridge between the lofty language of the Mission Statement and the concrete-specific nuts and bolts of program objectives. In the goals statement, the broad principles of the Mission are narrowed and focused into the specific categories of skills, knowledge, and abilities which will characterize graduates of your program including those that are specific to your discipline as well as those which represent the broader general competencies implied by Western’s mission and strategic goals.

The goals statement is essentially becomes a blueprint for implementing the mission by answering the following questions:

  • How do program goals relate to the program mission?
  • How does this program fit into a student’s overall development?
  • What general categories of knowledge and abilities will distinguish your graduates?
  • For each principle of the mission, what are the key competency categories
    graduates of the program should know or be able to do?

As discussed above in the “overview” section, general competency goals might include the four integrative abilities being considered as possible statewide required accountability goals (writing, information technology literacy, quantitative reasoning, and critical thinking), as well as the fourteen areas of alumni satisfaction Washington State currently wants assessed in alumni surveys—satisfaction with Western’s contribution to the graduate’s ability for:

writing effectively
speaking effectively
critical reading
quantitative reasoning
arts appreciation
scientific principles
civic rights and responsibilities
problem solving
working cooperatively
learning independently
cultural and philosophical diversity
interaction of society and environment
readiness for career
readiness for graduate study
developing satisfying meaning for life

Each major department must take responsibility for promoting and assessing student development across the range and level of abilities appropriate to its programs, including both majors and general education students. Therefore the program goals statement should include all of the key competency areas which the program or its courses address, for both majors and non-majors.

Program Objectives: Identifying Intended Learning Outcomes

Program objectives are brief, clear, focused statements of specific intended learning outcomes. Each objective can be linked directly to one or more program goals. Each objective should be defined with outcomes assessment criteria in mind for “measuring” how well each objective has been accomplished. Operationally, it is very helpful to formulate each objective statement to include.

Stating each objective in the form of an “action verb” combined with a description of a very specific ability helps translate objectives into learning outcomes students can actually demonstrate and faculty can actually measure. The use of the verb form emphasizes that objectives can be assessed by examining very specific products or behaviors students can actually do. By implication, each objective must have associated criteria for evaluating the success of the program in terms of the actual accomplishments of its graduates. For example, here are some sample learning objectives from the Human Services program:

  • Examine the history and philosophies of human services
  • Identify what constitutes genuine and empathic relationship
  • Analyze the role of conflict in individual and societal systems
  • Demonstrate a broad range of relevant communication skills & strategies
  • Design integrated services using innovative practices in diverse settings

Two kinds of learning objectives: mastery and development
There are two general categories of learning objectives. Mastery objectives establish minimum criteria for the acquisition and demonstration of foundational skills or knowledge. Mastery implies the achievement of a minimal or threshold level of competence, and also implies that what is important is the attainment of a minimum or threshold level of competence. Mastery objectives are measured on a binary scale: pass/fail, satisfactory/unsatisfactory, etc.

In contrast, developmental objectives imply a sequential continuum of integrative abilities. In general these include two distinct categories of abilities to be assessed as student learning objectives: general, across-the-curriculum abilities, and abilities specific to the major. Developmental objectives form a hierarchy of sequential skill levels which become the basis for particular course sequences within a program.

Because developmental objectives are best represented as a sequence of checkpoints for student learning, it is important and useful for departments to establish criteria for defining and assessing several different levels of developmental abilities, and to associate the attainment of sequential levels of such abilities with specific courses or groups of courses in their programs. In this way program objectives can be integrated meaningfully into individual courses, and learning objectives for one course become prerequisite knowledge for more advanced courses.

For example, a sequence of developmental objectives might include:

  • Demonstrate observational skills
  • Draw reasonable inferences from observations
  • Demonstrate perception of important relationships in observations
  • Analyze structure and organization
  • Select and apply appropriate theoretical constructs to observations

Both mastery and developmental objectives can be associated with a wide variety of competencies:

  • Knowledge
  • Cognitive development–area and level
  • Technical skill development–skill and level
  • Process skill development–skill and level
  • Comprehension–type and level
  • Application
  • Analysis
  • Synthesis
  • Evaluation
  • Integrative thinking/ creativity
  • Attitudes, behaviors, and values
  • Development of desirable personal/professional qualities

Assessment and Outcomes

Selecting Measurable Learning Outcomes

Learning outcomes are observable indicators or evidence of actual student learning. Each program must select an array of assessment tools, which can include both direct measures of student knowledge and performance, and indirect measures of changes in student behavior, attitudes, or values.

Direct measures include national standardized tests; licensing or certification exams; local content or competency exams, papers, or projects; skills tests, projects, reports, demonstrations, or performances; portfolio analysis; capstone projects, experiences, or performances; email or online discussion board content; and so forth.

Indirect measures include surveys of students, alumni, or employers; student or graduate profiles, interviews, or focus groups; transcript analysis; periodic review of syllabi, textbooks, exams, or other curricular materials; and so forth.

Each program will have its own unique needs and its own set of outcomes. What is important is that each outcome provides evidence about the accomplishment of a particular program objective. Ideally, each objective will be assessed by multiple outcomes measures so that:

  • Each outcome is a measurable estimator of a program objective
  • Outcomes selected are feasible measures given the resources available
  • Outcomes link actual student learning to intended post-graduate abilities
  • Outcomes accurately reflect ability and knowledge
  • Outcomes can be direct or indirect measures

back to Introduction

Measurement, Evaluation, and Reporting

The whole point of assessment to establish an ongoing, systematic mechanism for assessing, reviewing, and improving programs. Therefore each program assessment plan must include explicit procedures for determining which outcomes will be measured; when they will be measured; who will measure them; who will analyze them; what results will be reported, to whom; and how results have been implemented.

This is the step in the assessment cycle that makes assessment relevant, and it is the step which is likely to be most scrutinized by outside agencies. The “accountability” aspect of assessment is the requirement to document how assessment findings have been used to guide program improvement.

Currently Western is using an annual survey of academic departments to gather information on program assessment plans. This procedure is likely to be modified in the future into a uniform reporting format that includes program mission, goals, objectives, outcomes, and procedures, along with a cumulative listing of program improvements that have been made as a result of assessment findings.

Therefore, this section of each plan must show not only how results have been applied to program improvement in each annual cycle, but also must analyze what results say about program effectiveness and about the impact of assessment-induced changes on program effectiveness over time.

Reporting
At present Western has no common assessment activity reporting requirement or format for academic units. In the past, assessment information has been gathered from units via an online survey, and data from the survey has been collated and used to construct assessment reports for accreditation and for the State.

In the future it is quite likely that Western will adopt some common reporting format for academic units, which will generally follow the structure described in this section and shown in the figure below:

Outcomes Assessment Plan

  1. Program mission statement
  2. Program goals consistent with mission statement
  3. Multiple learning objectives (intended learning outcomes) for each goal
  4. Measurement of multiple outcomes for each learning objective
  5. Assessment criteria for each learning objective
  6. A framework for data analysis and program improvement
  7. Documentation of how assessment results have improved both programs andassessment criteria and procedures

Source:http://pandora.cii.wwu.edu/cii/resources/outcomes/measurement.asp

Models and Resources for Developing Outcomes

A number of departments and units at Western have already made a serious commitment to developing program and course assessment plans, and are using assessment data in many different ways to improve student learning. The information presented here is meant to show some of the diversity of work in progress in different programs across campus. It is by no means exhaustive, nor are any of the results shown here “final” in any sense. Rather, they are snapshots of the ongoing evolution of the assessment of student learning at Western.

  1. Engineering Technology (PDF)
    This is an example of a departmental model designed in response to the requirements of Industrial Advisory Boards, appropriate accrediting agency, and several professional organizations. It features learning outcomes in particular skill areas of analysis, communication, teamwork, technology, creative problem-solving, ethics, and professionalism.
  2. Physical Education Outcomes Assessment Plan (PDF)
    Recreation Program Outcomes Assessment Plan
     (PDF)
    Based on a learning outcomes development process created at California State University at Chico, these outcomes assessment models for Physical Education and Recreation identify Learning Objectives, Learning Processes, Assessment Techniques, Status/Outcomes/Results, and Decisions/Plans for Future Recommendations.
  3. Environmental Studies Introductory Course Assessment (PDF)
    Huxley College of the Environment moved to revise its core curriculum and entry-level GUR course to place greater emphasis on problem-solving and interdisciplinary integration of subject matter. The course revisions were designed as part of a larger curricular change, which includes this entry-quarter core experience, to be followed by a choice of other substantive courses to complement the student’s major, and finally, another integrative, problem-based capstone course. Perry analysis of student papers was conducted along with student self-evaluation to assess student learning.
  4. College of Business and Economics MBA Outcomes Assessment(PDF)
    This study in the MBA program uses a “Post-Then methodology” to ask students to rate their learning in the program by comparing their levels of expertise in various areas with those upon entry into the program. Both the rationale and the results of the assessment are presented in this preliminary report.
  5. Geology Program Trial of Critical Thinking Rubric (PDF)
    In Spring of 2002 the Geology Department faculty tested a critical thinking rubric adapted from one developed at WSU, using a number of raters. Although ratings varied unpredictably between some raters on some questions, most faculty found the rubric promising and useful, and plan to develop it further.
  6. Woodring College of Education State Program Approval for Math(PDF) / Woodring College of Education State Program Approval for Health & Fitness (PDF)
    Washington State places special requirements on Education programs for assessment of student learning among future teachers, with different assessment plans required for different specialties. Here are the competencies for two of those programs, together with outlines of where in the curriculum these competencies are to be learned, and how they are assessed.

Source:http://pandora.cii.wwu.edu/cii/resources/outcomes/models.asp

Bloom’s Revised Taxonomy Wheel

Bloom’s Taxonomy Wheel


Designing Learning

Writing Learning Objectives

Definition of learning objectives

Learning objectives describe what students are expected to learn and what they will be assessed on as a result of participating in a course. It is important that course objectives are written for specific forms of learning that students are able to demonstrate as part of their assessments.

Benefits of learning objectives

• Learning objectives maximise student study efforts and encourage independent learning by making the teacher’s focus and decision-making for assessment transparent.
• They provide lecturers with a guide for what should be assessed
• They provide the basis for lecturers and tutors to link teaching design and teaching activities with desired student results
• Writing learning objectives provides course teams with the opportunity to demonstrate which graduate skills are developed in their course
• They provide the basis for evaluating course effectiveness in relation to student learning.

How to write Learning Objectives

1. Begin your list of 3-5 learning objectives with a one-time lead-in statement like —

“Upon successfully completing this course, students will be able to – “

“After completing this course, you should be able to – “

“This course will prepare you to be able to do the following:”

2. Then, for each learning objective that follows, start with an action verb that reflects the appropriate behavior students should be able to demonstrate, e.g.,

Knowledge verbs what students will need to know (cognitive behaviors).

Attitude verbs describe what students should care about (affective behaviors).

Skill verbs describe what students should be able to do (psychomotor behaviors).


These verbs also describe the level of learning that is expected by students, so make sure you select a behaviour appropriate to the level you expect.

NOTE: Your action verbs must reflect observable and measurable behaviors.

3. After each action verb, include a qualifier to restrict the conditions and terms under which the objectives are met.

• How often? for example: at least once per hour, at the start of every cycle, before starting the task or after
• How well? for example: exactly 7%, no more than 1 error, accurate to three decimal points
• How many? for example: identify at least 16 items, produce 4 items,
• How much? for example: 100 meters long, 1/2 block before turning
• How will we know it is OK? for example: until the left hand is touching, by speaking only after the customer has spoken
• Combination – for example: produce at least 15 per hour (how many and how often), until the ditch is 300 feet long with tapering slopes (how much and we know it is OK)
• What is given/not given? for example: by checking a chart, by looking at photo, by referring to the manual, without reference to the manual, with no supervision
• What are the variables? for examples: no matter how upset the customer becomes

Learning Objectives Checklist

 Read each learning objective – Yes/No
Does it speak directly to the learner? (refer to what student might achieve, not what teacher will do)
Is it measurable?
Does is target one specific aspect of expected performance?
Does it use an effective action verb?
Does it match instructional activities and assessments?
Is it written in terms of observable behavioral outcomes?

Source:http://www.business.uts.edu.au/teaching/staff/objectives.html

Android Apps for writing Learning Outcomes:

(1)Bloom’s Revised Taxonomy:

unnamed (1)unnamed

https://play.google.com/store/apps/details?id=appinventor.ai_orionmbeadling.Blooms_Revised_Taxonomy&feature=search_result#?t=W251bGwsMSwyLDEsImFwcGludmVudG9yLmFpX29yaW9ubWJlYWRsaW5nLkJsb29tc19SZXZpc2VkX1RheG9ub215Il0.

(2) Bloom Taxonomy:

unnamed (2)

unnamed (3)

 

 

 

 

 

 

 

 

 

https://play.google.com/store/apps/details?id=com.widged.android.bloomTaxonomy&feature=search_result#?t=W251bGwsMSwyLDEsImNvbS53aWRnZWQuYW5kcm9pZC5ibG9vbVRheG9ub215Il0.

 

 

Advertisements