American Society for Engineering Education                                                                Pacific Southwest Section

Published in Spring 2003 Conference Proceedings



Using On-Line Quizzes Outside the Classroom to Increase Student Engagement Inside the Classroom



Eileen M. Cashman, Elizabeth A. Eschenbach

Environmental Resources Engineering, Humboldt State University




This paper describes the implementation of a web-based teaching tool at Humboldt State University (HSU) to teach Introduction to Environmental Resources Engineering to a large class of introductory environmental engineering and environmental science students.  The course instructors are adapting an approach called Just-in-Time Teaching (JiTT), to increase interaction between the students and the faculty in the classroom. In this adaptation, students complete on-line quizzes that are due by electronic submission before class begins.  The instructor reviews the responses “just in time” for class to assess the students’ acquired knowledge, as well as misconceptions, and so, better utilize active learning strategies in the classroom.  This work is part of a Course, Curriculum and Laboratory Improvement (CCLI) grant from NSF.


This paper provides examples that demonstrate how the instructors are adapting this web-based tool at HSU and discusses the strengths and drawbacks of the approach.  The instructors have observed that students come to class more engaged and prepared to discuss course material using JiTT.  Feedback from the students after one semester indicated that 92% thought the JiTT approach improved their learning. 


Course Description

Engineering 115:  Introduction to Environmental Resources Engineering is a required introductory course for both Environmental Resources Engineering students and Environmental Science students at HSU.  The course has 60-70 students enrolled each semester.  The course has two 50-minute recitation periods where all students meet together and a three-hour laboratory period with 24 students in each lab section.  Three lab sections are offered.


The course includes critical analyses of problems from both engineering and science perspectives through case studies in air resources, geotechnical resources, water resources and energy resources.  The course integrates lecture, discussion, student projects, computer labs, wet labs and outdoor field labs in the context of environmental engineering and science students working together on resource issues.


Students are introduced to a number of problem solving and research skills, including using the Internet, web page design, word processing, data collection using field instruments, and data analysis using spreadsheet modeling and Geographical Information Systems.  Students are expected to take this course their first or second semester on campus and the only requirement is a co-requisite of college algebra.


Just-in-Time Teaching - JiTT

JiTT blends web-based preparatory assignments with classroom learning.  The JiTT strategy has been pioneered by physics faculty and has been used by faculty at more than 50 institutions across the country (Novak et al. 1999).  There is documented increased attendance and decreased attrition with the implementation of JiTT (Novak et al. 1997, Rozycki 1999).


Immediate feedback is a critical component of JiTT.  Students complete web-based preparatory assignments before class.  Students know that their comments and questions will be incorporated into the content delivered in the classroom.  Students come to class motivated to learn what they have not been able to teach themselves.  Thus, instructors spend time on common misconceptions and stumbling blocks and do not spend time on material students have shown they have easily grasped.


Web-based Preparatory Assignment - On-line Quizzes

On-line quizzes are an important component of JiTT.  At HSU, once a week, students complete on-line quizzes that are due by electronic submission (via BlackBoard) one hour before class.  The instructor reviews the responses “just in time” for class and adjusts and organizes lessons based on those student responses.  The students come to class prepared and already engaged in the material.  The instructor has already assessed the students’ acquired knowledge, as well as misconceptions, and so can better utilize active learning strategies in the classroom.


The quiz generally consists of two to three short-answer questions and a feedback box.  There is usually a calculation question that requires that students make some assumptions, and a conceptual question that requires that students process a specific concept and synthesize it in a clear concise answer.  At HSU, the feedback box is heavily utilized and provides additional feedback on topics the instructor may not have covered in the quiz questions, but are still of concern to students.


Quiz answers are graded on the quality of the thought and the detail in the information provided.  Students can receive full-credit for a quiz answer that is incorrect, if they have articulated their assumptions and thought processes in a way that makes it clear to the instructor where the student’s error occurred.  Students receive no credit for an answer that is correct if they do not demonstrate how they arrived at their answer.  In this sense, the term “quiz” is a misnomer.  At other institutions, such as the Air Force Academy, these web-based assignments have been termed “Pre-Flights” (Novak 1999).


The instructor must download and review quiz answers during the hour prior to class.  Instructor preparation involves reading through the quiz answers and summarizing the main ideas submitted by the students.  The instructor must also be prepared to modify the planned lecture material or learning activities based on the answers (and questions) submitted, if necessary.  One hour prior to class has proven to be an adequate amount of time for an introductory course of 70 students.  Two examples provided below describe modifications made to lecture and recitation period activities based on student feedback.


On-line Quiz Examples

Two example on-line quiz questions are presented below.  Summaries of student answers are presented along with an analysis of student responses and a discussion of how the course content was adjusted based on student responses.  The first question illustrates a conceptual question and the second question a calculation question. 


The questions were posed as part of a unit on indoor and outdoor air resources.  The class discussion and course readings were materials related to indoor and outdoor air pollutants.  In the lab, students were in the middle of a ventilation rate analysis of rooms on campus using CO2 as a tracer gas.  Students had collected CO2 data in a classroom on campus and were just beginning to conduct a spreadsheet analysis of their data to calculate a ventilation rate.   Students were aware of the air emissions problems associated with fireplaces, wood-burning stoves and fossil fuel combustion.  Energy resources and technology had not yet been covered.


Question 1:  The authors of your text suggest electric heating as a means to reduce air emissions.  Do you think this is a good suggestion?  Explain your answer.


Summary of Student Responses







Adjusting Course Content

In reviewing these answers, the instructor was able to better gauge at what point the students were ready to begin discussing the management of air resources.  From the answers it was clear that many students in the class had thought about alternative energy technologies and were able to think outside the air resource unit and make connections to other resource issues.   Some of the students were ready to discuss issues of social justice and equity in resource use and how the use of technology can exacerbate or resolve those issues.  As an instructor, it became clearer where to begin the conversation in class. 


From these answers, the instructor quickly developed a portion of the lecture addressing the issue of high quality and low quality energy and second law efficiencies.  The student feedback showed that they had a deeper understanding of the connections between air and energy resources and the associated technologies than the instructor had anticipated.  The instructor was able to skip introductory material and build on the some of the more clearly articulated student ideas.  The instructor pointed out some of the more complex and intriguing ideas that students were already thinking about, but maybe had not articulated or realized by introducing some very general concepts of thermodynamics in this portion of the class.


The connections made between air resources and energy resources also resulted in the instructor rethinking the sequencing of the course units.  In the original course syllabus, the next unit in the course is geotechnical resources (this is largely dictated by the sequencing of the chapters in the course textbook).  This information has been noted and will be used to possibly reorder the sequence of units for next semester. In this respect, the use of JiTT has provided feedback that will be useful for curriculum development at a larger scale (course planning). 


The recitation session following the quiz focused on helping students to better articulate their ideas about the connections between air resources and energy resources and begin to critically think about ways to approach and manage these issues.


Question 2:  In your reading of the Scientific American article, the authors reported on exposure to chloroform while in a steamy shower.  How much (grams) chloroform do you inhale in your average shower?  Based on OSHA’s recommendations of 50 ppm over an 8-hour day/ 40-hour work week, is this a concern to you?


The first part of this question requires students to complete several tasks.  They must read the article and extract an average exposure from the graphics presented by the authors.  The average exposure concentrations of chloroform are presented only in graphical form and the graphs are on a log scale.  The units provided in the article are in mg/m3.


Once they estimate the concentration in shower steam, they must make a series of estimates, such as: 1) how long are they are in a shower, 2) how much air do they breath while in the shower, 3) what is the volume of their lungs, and 4) how many breaths do they take per minute?


The final part of this question requires students to realize that the recommended standard is reported in ppm and they have a reported exposure in mg/m3.  The student must convert between the two units using the Ideal Gas Law (IGL).  The students have been referred to reading that summarizes the IGL and there is discussion of this conversion in the text and a table of conversion factors available from common air pollutants (chloroform is not one of them). The students had already participated in a discussion of units of air pollutants including conversions and how to interpret the concentration unit of ppm.


Summary of Student Responses







Adjusting Course Content

After reviewing this information, the instructor decided that more time should be spent in class on basic definitions for the terms used in the calculations.  Students needed to talk about the appropriate units for volume.  The class needed to be reminded of how to read a logarithmic graphic and of the definition of molecular weight.  Without this information, the instructor may have focused more on the mechanics of solving the IGL equation when in fact most of the students were not having trouble with the algebraic manipulations.  


In the recitation period immediately following this quiz, the instructor summarized the main areas of misconceptions brought out by the student answers and discussed each one.  The issues of how to read the graphics and the definition of molecular weight was addressed quickly as these were concepts about which most students just needed reminding.  The question of how to estimate lung size was addressed by presenting several of the student answers to the class.  Students were impressed (and amused) by their classmates’ creativity and by the variety of ways their colleagues estimated lung capacity.


Finally, it was clear that the entire calculation of using the IGL to convert between mg/m3 and ppm should be reviewed in class.  The students guided the instructor through the calculation and were able to ask very specific questions based on where they had hit roadblocks in completing this calculation on their own.  The students were clearly engaged in this calculation and not simply copying what the instructor wrote on the board.


This quiz question lead to further class discussion of the following questions:


These questions provided material for several future recitation sessions and the students were engaged in the process. 


Benefits and Drawbacks of JiTT

The following section discusses general observations from the instructors with regard to the benefits and drawbacks of the JiTT approach.  There are also some comments on further components the instructors would like to implement in the context of this course.
















The implementation of JiTT quizzes in the introductory survey course in Environmental Resources Engineering has been successful in terms of engaging the students via the Blackboard delivered online quizzes. The instructors perceive a better understanding of the students’ misconceptions and have an increased sense of interaction in a class of 70 students.


This technique continues to be implemented at HSU.  Improvements will come about as more detailed and quantitative assessment results become available and other faculty members implement the course.



1. Novak, Gregor, Evelyn Patterson  (1997).  World Wide Web Technology as a New Teaching and Learning Environment. International Journal of Modern Physics, Vol 8, No. 1: 19-39.


2.  Novak, Gregor, Andrew Gavrin, Wolfgang Christian, Evelyn Patterson  (1999).  Just-In-Time-Teaching: Blending Active Learning with Web Technology. ISBN 0-13-085034-9, 1/e. Prentice Hall.


3.  Rozycki, William  (1999).  “Just In Time Teaching.”  Research and Creative Activity.  Office of the University Graduate School at Indiana University. April.







Biographical Information:


Eileen M. Cashman is an assistant professor in the Environmental Resources Engineering Department at Humboldt State University.  She teaches introductory engineering courses and upper division courses in fluid mechanics, water quality and river hydraulics. 


Elizabeth A. Eschenbach is an associate professor in the Environmental Resources Engineering Department at Humboldt State University.  She teaches introductory engineering courses and upper division courses in probability, environmental impact assessment and water resources. 


Dr. Cashman and Dr. Eschenbach are Co-PIs on an NSF funded grant for Course Curriculum and Laboratory Improvement grant (CCLI) award 0127139 and the work described above has been supported by this grant.  Both authors are members of ASEE and Faculty-21 in Project Kaleidoscope.