ºÚÁÏÉçÇø

Who has access to the data: Instructor

Delivery method: Online or in-class

Description: The Bare Bones Questioning Technique (also known as Stop-Start-Continue) is a five-minute evaluation process in which ºÚÁÏÉçÇøs type online, write on note cards, or verbalize their answers to these three questions:

Use the following three questions:

• What (if anything) is interfering with your learning? (STOP);
• What suggestions do you have to improve your learning? (START);
• What is your instructor doing that helps you to learn? (CONTINUE).

Delivery: For in-class delivery, instructors may write the questions on the board/screen, read them aloud, or pass them out on printed paper.

For online delivery, instructors may import the Bare Bones instrument directly to their course Canvas site using these steps:

• Log in to your myMiami account.
• Go to Canvas; then select Canvas Commons on the left menu bar.
• In the Search bar that appears, type "Miami Midcourse Evaluation CTE & eLearning.”
• Click the link to "Miami Midcourse Evaluation CTE & eLearning.”
• Click “Bare Bones Questioning Technique.”
• Click “Import/Download.”
• Select your course from the list that appears by checking the box beside it.
• The Bare Bones tool will appear in the Modules section of your course Canvas site.

Instructors may announce the survey to ºÚÁÏÉçÇøs via Canvas, and ºÚÁÏÉçÇøs’ responses will appear in an Excel file.

Who has access to the data: Instructor

Delivery method: Online

Description: The Student Assessment of Their Learning Gains (SALG) is an online survey originally developed through a National Science Foundation grant. The SALG is designed for instructors in all disciplines to get feedback from their ºÚÁÏÉçÇøs on various elements of a course.

Delivery: Instructors may use this  (bit.ly/CTE_SALG; the link is case-sensitive) with their preferred survey platform (Canvas, Qualtrics, GoogleForms, SurveyMonkey, etc.). The template allows you to choose or modify the statements as you see fit in order to assess ºÚÁÏÉçÇøs’ perceived achievement of your learning goals. Students take the survey online through a link you provide.

For further reference, the Center for Teaching Excellence has created a video seminar on how to use the SALG: .

Who has access to the data: Instructor, Chair, University

Delivery Method: Online

Description: The midcourse evaluation will be delivered via the same delivery tool (i.e., What Do You Think) as end of course evaluations. Please contact courseevals@miamioh.edu for details on how to use the survey feature of "What Do You Think" as a tool for creating and delivering mid-course evaluations.

Who has access to the data: Instructor

Delivery method: Online

Description: The SEEQ is a comprehensive ºÚÁÏÉçÇø rating form providing useful information about teaching effectiveness. This tool focuses on learning environment, enthusiasm, organization, group interaction, individual rapport, breadth, examination, assignments, and overall (general) evaluation. This tool can be administered online or printed and distributed during a class session.

Learning Environment

1. The following statements are rated on the scale Very Poor, Poor, Moderate, Good, Very Good, or Not Applicable:
   • You find the course intellectually challenging and stimulating.
   • You have learned something which you consider valuable.
   • Your interest in the subject has increased as a consequence of the course.
   • You have learned and understood the subject materials in this course.
2. Do you have any comments to add about the Learning Environment of the course?

Who has access to the data: Instructor

Delivery method: In class

Description: For a QCD, the instructor meets with a faculty developer to discuss objectives and any changes to the basic protocol. The instructor prepares the class for the 15-minute experience and leaves the room during the QCD. Later, the instructor meets with the faculty developer to review the data and to plan improvements.

For the processing, the faculty development team (one person to ten, depending on the class size) greets the ºÚÁÏÉçÇøs and explains the procedures. Students are asked to write on an index card a number from one to five indicating their satisfaction level with the course and a word or phrase to clarify their experience ("awesome," "confusing," etc.). For the report, these data are dropped into a histogram that displays the number of ºÚÁÏÉçÇøs and lists each number and the associated words or phrases.

The team can then display for ºÚÁÏÉçÇøs, via a projector or printed copies, a numbered list of the ºÚÁÏÉçÇø learning outcomes (SLOs) for the course. On the reverse side of the index card, the ºÚÁÏÉçÇøs indicate (by recording their numbers) the two SLOs they felt were best met and the two that were least fulfilled. During the final stage of the QCD, ºÚÁÏÉçÇøs form groups of five to seven and on a highly structured form, they identify the course (or program) strengths and weaknesses using a cooperative brainstorming technique called "roundtable" where ºÚÁÏÉçÇøs rapidly pass around a sheet of paper, adding ideas as they say them aloud. The groups then rank the top three strengths and the top three weaknesses. These data are recorded onto a single template, group by group, and then analyzed by a person skilled in trend analysis, usually the faculty developer. Common themes are coded with the same color across teams, thus emphasizing the common strengths or issues. For example, if four teams mention "poor textbook" or anything similar (e.g., "textbook sucks"), a reader will see red in all the team ratings, if that is the color selected for "poor textbook."

Who has access to the data: Instructor

Delivery method: Online

Description: The ISQ can be used to provide instructors with immediate and specific feedback concerning their teaching. The ISQ conceptualizes teaching in terms of the seven ISQ dimensions based on Feldman's (2007) categories of teaching behavior. Each dimension was measured by two indicative items and two contra-indicative items with a 7-point Likert scale response format (response options ranging from strongly disagree to strongly agree). The contra-indicative items are recoded prior to analyses.

The seven ISQ dimensions are defined as follows:

1. Structure: the extent to which the subject matter is handled systematically and in an orderly way
2. Explication: the extent to which the instructor explains the subject matter, especially the more complex topics
3. Stimulation: the extent to which the instructor interests ºÚÁÏÉçÇøs in the subject matter
4. Validation: the extent to which the instructor stresses the benefits and the relevance of the subject matter for educational goals or future occupation
5. Instruction: the extent to which the instructor provides instructions about how to study the subject matter
6. Comprehension: the extent to which the instructor creates opportunities for questions and remarks regarding the subject matter
7. Activation: the extent to which the instructor encourages ºÚÁÏÉçÇøs to actively think about the subject matter

These statements are rated on a scale of Strongly Disagree, Disagree, Somewhat Disagree, Do Not Agree Nor Disagree, Somewhat Agree, Agree, and Strongly Agree.

For more clarity on this evaluation process, please visit this 

Knol MH, Dolan CV, Mellenbergh GJ, van der Maas HLJ (2016) Measuring the Quality of University Lectures: Development and Validation of the Instructional Skills Questionnaire (ISQ). PLoS ONE 11(2): e0149163. doi:10.1371/journal.pone.0149163

Who has access to the data: Instructor

Delivery method: Instructor reflection

Description: The instructor completes the Teaching Practices Inventory (TPI) and writes a reflection. Students are not directly involved in this process. These questions are specifically geared toward the science and mathematic lecture-based subjects. It takes about 10-15 minutes to complete. The acronym for this method is representative of Carl Wieman Science Education Initiative at the University of British Columbia.

Who has access to the data: Instructor

Delivery method: In Person

Description: Instructors will observe their classroom in multiple 2-minute increments throughout the session. These observations focus largely on ºÚÁÏÉçÇø behavior rather than teacher quality. There are 25 different codes that can be used to describe each observation installment that can then be turned into quantitative data for further and easier assessment. These codes represent themes such as listening, individual thinking, clicker, discussion, worksheet group work, other group work, answer, ºÚÁÏÉçÇø, whole class discussion, predicting, ºÚÁÏÉçÇø present, test/quiz, waiting, other, lecturing, writing, follow up, pose, moving/guiding, one-on-one, demo+, and admin. There is a recommended 1.5 hour training session to learn how to properly implement this method as an evaluation method.

Who has access to the data: Instructor

Delivery method: Paper

Description: This observation method is advocated for use in classrooms that are considered 'reformed' in regards to being ºÚÁÏÉçÇø-centered and activity-based rather than teacher-centered. The RTOP looks at 5 main items in a reformed classroom: lesson design and implementation, propositional knowledge, procedural knowledge, ºÚÁÏÉçÇø-ºÚÁÏÉçÇø interaction, and ºÚÁÏÉçÇø teacher interaction. Each of these items, or subscales, are graded on a scale of 0-4. This procedure was originally created for mathematics and science based courses.

Example Section:

• Instructional strategies and activities respected ºÚÁÏÉçÇøs' prior knowledge and the preconceptions inherent therein.
• The lesson was designed to engage ºÚÁÏÉçÇøs as members of a learning community.
• In this lesson, ºÚÁÏÉçÇø exploration preceded formal presentation.
• This lesson encouraged ºÚÁÏÉçÇøs to seek and value alternative modes of investigation or of problem solving.
• The focus and direction of the lesson was often determined by ideas originating with ºÚÁÏÉçÇøs.

Students rate using the following scale: 0 - Never Occurred, 1, 2, 3, 4 - Very Descriptive.

Who has access to the data: Instructor and Observers

Delivery method: Paper

Description: This method is used to examine the dynamics that occur between ºÚÁÏÉçÇøs, instructors, and technologies within the classroom. This method gives descriptions of teaching rather than giving the judgment of quality of teaching. The TDOP has 6 main areas that it focuses on: instructional practices, ºÚÁÏÉçÇø-teacher dialogue, instructional technology, potential ºÚÁÏÉçÇø cognitive engagement,* pedagogical strategies,* and ºÚÁÏÉçÇøs' time on task.* The 3 listed above with an asterisk (*) are considered optional to those using this evaluation tool. To use this method, one needs to select who is going to be observing the class (usually more than 1 person), select which of the 6 you want them to focus on, use the codes on the template (a sample is given below), participate in a training to read/use results, conduct observations, then analyze and interpret data.

Who has access to the data: Instructor

Delivery method: Classroom observation or critique of classroom artifacts

Description: Peer review is often identified with peer observations, but it is more broadly a method of assessing any aspect of the class for the instructor under review. This typically includes peer observations of teaching, and other evidence such as syllabi, assignments, ºÚÁÏÉçÇø work, and exams. Your peer may use their own background knowledge of teaching to evaluate these items/events, or it may be beneficial to use the benchmarks provided a professional organization in your field. If you are interested in learning more or would like to request help looking for these professional benchmarks, please contact your department chair.

It is also worth noting a common distinction between two very different forms of peer review: formative and summative. Formative evaluation typically is oriented solely toward the improvement of teaching and is part of instructional mentorship and development. Summative evaluation, in contrast, is that done to inform personnel decisions. To improve the freedom and exploration of individual instructors, formative reviews may be shielded from scrutiny for a period of years until such time that there needs to be accountability to standards of excellence for personnel decisions. At this point in time, summative evaluations are more common since they are tied to decisions related to reappointment, promotion, or tenure (Bernstein et al. 2000). Because the more consequential nature of summative evaluations tends to diminish the formative value of the peer review process, it is important to maintain a clear distinction between these types of evaluation and be transparent with those under review.

Visit the Peer Review of Teaching page on the Provost's website.

• What have you learned in this course that you find particularly interesting or compelling?
• What's helpful in this course to your learning?
• What suggestions do you have for change?
• How is the course going for you?
• What would help make it a better learning experience for you?
• On a scale of 1-7, with 1 being low and 7 being high, how is the course going for you? Why did you choose this number?
• Do you usually understand what is expected of you in preparing for and participating in this class? If not, please explain why not.
• What aspects of this course and your instructor's teaching help you learn best?
• What specific advice would you give to help your instructor improve your learning in this course?
• What other ideas would you suggest to improve this course (e.g., changes in course structure, assignments, or exams)?
• What are the most important things you have learned so far in this class?
• What don't you think you understand well enough yet?
• What would you like to see more of between now and the end of the semester?
• What do you think we could cut down on?
• What do you need to do in terms of understanding the material between now and the end of the semester?
• What is your overall evaluation of the instructor?
• Which aspects of this course/instructor are learning to valuable learning experiences?
• What aspects of the course/instructor need to be improved to increase the value of the learning experience?
• Please write any additional comments or suggestions.

Students rate from strongly agree through strongly disagree - Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree.

1. I find the format of this class (lecture, discussion, problem-solving) helpful to the way that I learn.
2. I feel that this class format engages my interest.
3. I feel comfortable speaking in this class.
4. I learn better when the instructor summarizes key ideas from a class session.
5. I find the comments on exams or other written work helpful to my understanding of the class content.
6. I find that this class stimulates my interest in reading about this subject outside of class.
7. I feel comfortable approaching the instructor with questions or comments.
8. I think that I would learn better if a different format were used for this class (suggested below).
9. The instructor holds the ºÚÁÏÉçÇøs to high academic standards
10. The instructor effectively challenges me to think and to learn.
11. The instructor is well prepared.
12. Examinations and/or other graded components cover course concepts in a challenging manner.
13. The instructor shows enthusiasm for the subject.
14. I feel free to ask questions and to make comments in class.
15. The instructor deals with questions and comments effectively.
16. The instructor is generally available during office hours.

• I take responsibility for helping make this course a positive learning environment.
• I attend class.
• I prepare for class.
• When I attend class, I am actively engaged.
• I stay up-to-date in the course work.
• I seek help when I need it.
• What steps could you take to improve your own learning in this course?
• How many hours per week, outside of regularly scheduled class meetings, do you spend on this class?
  • 1-2, 2-4, 4-6, 6-8, more than 8
• How much of the reading that has been assigned so far have you completed?
  • 100%, 90%, 75%, 50%, less than 50%