The effect of post-laboratory discussions on student understanding and content retention in entry-level biology courses

An important part of the science process is sharing and discussing experimental results and approacheswith peers. Validation of knowledge by experiment replication (each student performing the sameexperiment, then sharing their results) and the opportunity to evaluate the experimental approach applied by others is important according to Domin (2009), which could be achieved through post-lab discussions in the online classroom. Hammond and Moring (2007) listed post-lab discussions as an important component in an active, inquiry-driven approach to science laboratory education, and peer-review introduces students to the nature of science according to Trautman (2009). However, such post-lab peer interactions have not yet been systematically tested in the onlineclassroom until now.The Effect of Post-Laboratory Discussions on Student Understanding and Content Retention in Entry- Level Biology Courses April Boulton, Ph.D. and GroTorsethaugen, Ph.D. University of Maryland University College Undergraduate Studies - Natural Sciences Introduction An important part of the science process is sharing and discussing experimental results and approacheswith peers. Validation of knowledge by experiment replication (each student performing the sameexperiment, then sharing their results) and the opportunity to evaluate the experimental approach applied by others is important according to Domin (2009), which could be achieved through post-lab discussions in the online classroom. Hammond and Moring (2007) listed post-lab discussions as an important component in an active, inquiry-driven approach to science laboratory education, and peer-review introduces students to the nature of science according to Trautman (2009). However, such post-lab peer interactions have not yet been systematically tested in the onlineclassroom until now. Domin (2009) suggested that “open-inquiry instruction” is more suitable for older or more experienced students in regards to the delivery of science content. Many of UMUC’s students are older and bring a significant amount of life experiences into the classroom so we believe the addition of more “open inquiry” instruction could be beneficial to our students. In addition, several recent studies address the effect of pre-lab activities on student understanding of concepts and achievement of course outcomes (Jones and Edwards, 2010; Gregory and Di Trapani, 2012), but we were unable to locate studies that directly measure the effectiveness of post-lab discussions. We hypothesized that entry-level biology students, who participated in post-lab discussions, would perform significantly better on a standardized assessment than their peers, who did not participate in such post-lab dialogues. Experimental Design During the fall 2012 and 2013 semesters we taught 8 BIOL 103 sections (4 each) with a total of 201 enrolled students, as counted at the end of the semesters. In four of the sections (2 sections/instructor) we used the standard BIOL 103 syllabus template, which does not employ post-lab discussions (control group, n=90 students). For the other four sections (2 sections/instructor) we used the standard BIOL 103 syllabus template with the added weekly task of participating in a post-lab discussion (treatment group, n=111 students). See Appendix 1 for post-lab discussion activities and questions. We examined significant effects between the following variables: differences across sections in mean performance on the Week 1 assessment (prior to any biology content delivered); differences in mean performance on Week 1 and 8 assessments between treatment (hereafter abbrev. tx) and control sections; and detectable differences in student performance as a function of instructor (i.e., experimenter error or effect). During Week 1, all students, both control and tx, completed the timed, standardized assessment to gauge baseline knowledge. In Week 8, after all post-lab discussions had been completed, students took the same timed, standardized assessment (given in Week 1) to gauge knowledge retention. The standardized assessment consisted of 25 multiple choice questions (Appendix 2), which students received 30 minutes to complete through the Fast Test web-site. Additionally, both groups of students completed a brief survey in Week 8 where they were asked to rate statements about the effectiveness of the lab activities and post-lab discussions on a scale from 1- 5, from strongly disagree (1) to strongly agree (5); the tx group answered a few additional questions that pertained to the post-lab discussions (Appendix 3). Results and Discussion Our main hypothesis was not supported by the data in that student performance on the standardized assessment did not significantly differ between control and tx groups (Table 1). Even though there was a slight improvement on the standardized assessment in Week 8 for both control and tx groups, the average score in Week 8 was very low (i.e., 59% for control; 61% for tx). Such overall poor performance on the standardized assessment may indicate that it was too challenging for the average BIOL 103 student or that 30 minutes was not enough time to accurately answer the questions. In regards to the opinion survey administered at the semester’s end, there were no significant differences between tx controlsections in the average ratings for any of the statements that were included in both tx and control sections (Table 2). Four statements about the post lab discussions were only included in the survey given to the tx sections where students had participated in the post lab discussion (Table 3). The majority of students answered that they agreed (4) or strongly agreed (5) that the post lab discussions had “helped them better understand the scientific method” (75%) and “the biological concepts covered in the text book and course modules” (74%). Additionally, 74% of students agreed or strongly agreed that they had “enjoyed discussing their results and comparing experimental approaches with classmates in the post-lab discussions”. Only 16% of students agreed or strongly agreed that the post lab discussions were” just more work and did not benefit their understanding of the lab activities and the corresponding biological concepts” (Table 2). This sentiment was supported by the fact that both tx and control groups showed an equal (19%) attrition rate in course enrollment during the semester (Table 1), indicating that students in the tx group were not more likely to drop BIOL 103 due to the increased workload associated with the post-lab discussions. So although the tx group (post-lab discussions) did not perform significantly better on the standardized assessment than their peers in control sections, they still believed that participation in the post-lab discussions was beneficial. Based on the number of students enrolled in the course at the end of the semester, on average, 60% of students participated in the post-lab discussions in the four tx sections. Participation ranged from 42% to 79% in individual post-lab discussions. The relatively low participation rate may have contributed to the lack of a tx effect as measured by the standardized assessment. BIOL 103 is a 4 credit course that is time consuming and challenging for most students to complete in just eight weeks. This may have decreased the effectiveness of the post-lab discussions because a relatively large portion of the students either did not participate at all or just invested minimal time and effort in the discussions. This preliminary analysis did not support our hypothesis. However, future analyses of the data will focus on analyzing the individual performance of students in tx sections who did diligently participate in the post-lab discussions versus those who did not. Given the large standard deviations across treatments (Table 1), we expect that students who participated maximally in the post-lab discussions will show the hypothesized tx effect when compared to those students who did not participate. References Domin DS (2009) “Considering Laboratory Instruction through Kuhn’s View of the Nature of Science”J. Chem. Educ., 2009, 86 (3), p 274 Gregory SJ, Di Trapani G (2012) “A Blended Learning Approach to Laboratory Preparation,” International Journal of Innovation in Science and Mathematics Education, 20 (1), p 56 Hammond CN, Mohrig JR, Colby DA (2007) “On the Successful Use of Inquiry-Driven Experiments inthe Organic Chemistry Laboratory,” J. Chem. Educ., 2007, 84 (6), p 992 Jones SM, Edwards A (2010) “Online Pre-laboratory Exercises Enhance Student Preparedness for FirstYear Biology Practical Classes,” International Journal of Innovation in Science and MathematicsEducation, 18 (2), p 1 Kirschner and Meester (1988) “The laboratory in higher science education: Problems, premises, andobjectives.” Higher Education 17(1), p81 Trautman, N (2009) “Designing Peer Review for Pedagogical Success. What can we learn from professional science?” Journal of College Science Teaching, 38 (4), p14 Table 1: Mean score (± standard deviation) on the pre- and post-assessment across control and treatment groups. Control Treatment Mean score on standardized assessment for the pre-test 12.0 ± 3.4 (n=97) 11.9 ± 3.1 (n=107) Mean score on standardized assessment for the post-test 14.8 ± 4.2 (n=69) 15.3 ± 4.0 (n=81) Table 2: Students ratings of statements in Control and Treatment sections on a scale from 1 (strongly disagree) to 5 (strongly agree). Statement Control (n=67) Treatment (n=65) The lab activities were beneficial in improving my understanding of the scientific method. 4.5 4.5 The lab activities were beneficial in helping me understand the biological concepts covered in the text book and course modules 4.5 4.5 I felt well prepared for the lab activities 4.0 4.0 The lab manual was easy to understand and follow 3.8 4.0 The instructor’s weekly feedback on the lab assignments (under the Study Groups) was useful and helped me understand the lab activities better. 4.6 4.7 A group discussion with my peersbefore the lab activities were performed would have helped me understand the lab activities better. 3.6 3.6 A more involved group discussion with my peersafter the lab activities were performed would have helped me understand the lab activities better. 3.4 Table 3: Student opinion on the benefits of study group participation. Number and percentage of students that rated each statement 1, 2, 3, 4, 5 Statement Average Rating 1 2 3 4 5 4-5 combined The post-lab discussions helped me better understand the process of the scientific method. (n=65) 4.1 1 (1.5%) 5 (7.7%) 10 (15.4%) 18 (27.7%) 31 (47.7%) 75.4% The post-lab discussions helped me better understand the biological concepts covered in the text book and course modules. (n=65) 4.1 1 (1.5%) 3 (4.6%) 13 (20%) 18 (27.7%) 30 (46.2%) 73.9% I enjoyed discussing my results and comparing experimental approaches with classmates in the post-lab discussions. n=64) 4.1 2 (3.1%) 3 (4.7%) 12 (18.8%) 17 (26.6%) 30 (46.9%) 73.5% The post-lab discussions were just more work and did not benefit my understanding of the lab activities and the corresponding biological concepts. (n=65) 2.1 25 (42.4%) 16 (27.1%) 8 (13.6%) 8 (12.3%) 2 (3.4%) 14.7% Appendix 1 POST-LAB QUESTIONS Administered to Treatment Group Only Lab 1: The Scientific Method 1. Check out one of these resources to learn more about how oxygen levels affect aquatic life and how scientists study this topic: • http://www.teachoceanscience.net/teaching_resources/education_modules/dead_zones/g et_started/ • http://www.chesapeakebay.net/discover/bayecosystem/dissolvedoxygen 2. Share your hypothesis and proposed experiment from questions 2 and 3 in Lab 1. Compare your hypothesis and experimental approach to those of your classmates and to what you learned from the Ocean Science and/or Chesapeake Bay web-site(s). Based on this comparison, would you change your own hypothesis and/or experimental approach? If yes, explain how. If no, explain why you still believe your hypothesis and approach is good. 3. How does this Study Group activity mimic the scientific process? What do scientists do after they have completed an experiment? Note: When commenting on at least one classmate’s response, provide suggestions for how their hypothesis and/or experiment could be improved. Lab 2: The Chemistry of Life 1. List one activity you performed in the lab that was experimental, in nature. Be sure to state the IV and DV 2. List one activity that was observational (not a true experiment). Justify both of your answers above. 3. Post your results from the pH activity. Compare your results to a peer’s pH results for similar compounds—what differences and similarities strike you? 4. Post your results from the calculations in #3 (Experiment 2). Create an analogy to share with your peers on why the ratio of surface area:volume affects diffusion potential? Lab 3: Cell Structure and Function In the "Feedback" note posted above, review the table from Experiment 2: Osmosis Experiment. 1. Did you get the results you predicted in the osmosis experiment? Explain. 2. Compare your results to the table posted. If different, which factors may have influenced your results? Then answer one of the following: 1. Explain the difference between osmosis and diffusion, include examples 2. Explain osmosis with an example from your everyday life, include which direction water molecules are moving and which solutions are hypertonic, hypotonic and/or isotonic in your example 3. Did the lab activity help you understand osmosis, hypertonic, hypotonic and isotonic solutions better? Why or why not? What made you understand and remember which solutions are hypertonic and which are hypotonic? Please share if you found good tutorials or web-sites. Lab 4: Enzymes In Experiment 2 question #5, the lab manual asked you to “Design an experiment to determine the optimal temperature for enzyme function, complete with controls. Where would you find the enzymes for this experiment? What substrate would you use?” Here are some additional resources to assist you in your understanding of the problem: http://www.youtube.com/watch?v=D2j2KGwJXJc http://staff.jccc.net/pdecell/metabolism/enzymes/enzymes.html#physical **Share your full experiment here. Be sure to state your hypothesis, control(s), independent and dependent variables in your description. Reply to at least one peer’s submission with special attention to how the experimental design relates to optimal temperature ranges for enzyme function. Lab 6: Taxonomy Most of you developed really good classification systems for question 3 in Experiment 1 where you had to pick an additional 10 household items. A few of you had trouble and ended up creating independent categories for your items, which showed no relationship among them. 1. Share your creation in this conference (pasting your picture or figure directly into the textbox is preferable over attaching the document). 2. Explain the rules you created to classify the objects. 3. Why might using the function or behavior of animals be more problematic than physical traits for defining classification rules? Support your answer. 4. Reply to at least one classmate, commenting on their classification system and adding suggestions for improvement. Lab 7: Ecology of Organisms On average, the class data indicated that the seeds sprouted ~90-100% of the time under neutral conditions (plain water) and only ~ 0-20% of the time in the other two conditions (baking soda vs. vinegar). 1. State a good conclusion based on the above class average. 2. Did you get the same conclusion as the class overall? Compare your own results with the class average data: germination percentage for each day, plus length of germinated seedlings on day 7. 3. What are some of the factors that many have influenced the difference in results 4. Which results do you believe are more scientifically valid and accurate—your own or the class average? Explain why. 5. Discuss why germination rates matter in nature. Appendix 2 BIOL 103 Standardized Assessment Directions: Pre-test Version To help me gauge the strengths of our class, which vary from semester to semester, please make your best attempt at answering the 25 questions in the pre-lab quiz below. You will automatically earn all 10 pts, if it is obvious that you made an earnest attempt. If I see evidence of random answer selection, then points will be deducted as necessary. Remember: you’re being graded on your honest attempt at all 25 questions, not just the number of correct answers. You have exactly 30 minutes to complete it once you begin the quiz. Once you start, you may not stop at any time, nor return to it at a later date. You may not consult the internet or any person while taking this quiz as it is supposed to reflect your knowledge of various areas in the field of biology. In the unlikely event that you lose power while taking the quiz (don’t start it during a thunder storm!), you can use your personalized “Crash Code” to get back into the quiz. You will be provided with this Crash Code before you start the quiz, so remember to write it down! Once power returns and you log back in you will still have the remaining time left to complete the test (if you were 10 minutes into the test, you will have 20 minutes left to complete the test). If this does happen you will need to send me an e-mail to explain what happened as I will see that you logged in twice. Finally, you cannot progress to the next question, until the current question has been answered. This also means that you will not be permitted to return to any one question at the end of the quiz. Thus, you will not be able to skip questions and return to them later, so do your best the first time you view a question. Post-testVersion Each student took a similar version of this quiz at the semester’s start. Now that you have worked through the bulk of material in BIOL 103, I would, again, like to test your understanding of the laboratory concepts we covered throughout the semester. This time, you can earn up to 25 pts on this quiz, which is based entirely on your correct answers. Be sure to try your best for each question, and do not leave any answers blank. You have exactly 30 minutes to complete it once you begin the quiz. Once you start, you may not stop at any time, nor return to it at a later date. You may not consult the internet or any person while taking this quiz. Plagiarism will result in a failing grade. In the unlikely event that you lose power while taking the quiz (don’t start it during a thunder storm!), you can use your personalized “Crash Code” to get back into the quiz. You will be provided with this Crash Code before you start the quiz, so remember to write it down! Once power returns and you log back in you will still have the remaining time left to complete the test (if you were 10 minutes into the test, you will have 20 minutes left to complete the test). If this does happen you will need to send me an e-mail to explain what happened as I will see that you logged in twice. Finally, you cannot progress to the next question, until the current question has been answered. This also means that you will not be permitted to return to any one question at the end of the quiz. Thus, you will not be able to skip questions and return to them later, so do your best the first time you view a question. Lab 1: The Scientific Method A scientist is observing that a pond located downstream from a farm is filled with green algae. This pond has been previous years been clear without any visible signs up algae growth. The scientist asks the farmer if he has made any changes in his agricultural practices this year and the farmer tell her about this new great fertilizer called RapidGrowth that he has started using. The scientist is wondering if the new fertilizer might be the cause of the algae growth in the pond. She designs an experiment to find an answer to this question. Which of the following would be the best hypothesis for his experiment? a) Does the RapidGrowth fertilizer leaking from the farm field cause an increase in algae growth in the pond? b) The RapidGrowth fertilizer is bad for the pond c) The RapidGrowth fertilizer causes an increase in green algae growth in the pond d) Fertilizers will lead to an increase in algae growth in ponds because of the increase in nutrients running of the field Correct answer: C A biology student has designed an experiment to study how different oxygen levels affect fish. Average oxygen levels measured in natural bodies of water are 5 mg/L. He sets up five aquariums with the following oxygen levels: 0 mg/L, 3 mg/L, 5 mg/L, 7 mg/L and 10 mg/L. He puts 20 fish of the same species in each aquarium and counts the number of fish in the tank every week over a 2 month period. Which oxygen tank is the control in this experiment? a) 0 mg/L b) 3 mg/L c) 5 mg/L d) 10 mg/L Correct answer: C A biology student has designed an experiment to study how different oxygen levels affect fish. Average oxygen levels measured in natural bodies of water are 5 mg/L. He sets up five aquariums with the following oxygen levels: 0 mg/L, 3 mg/L, 5 mg/L, 7 mg/L and 10 mg/L. He puts 20 fish of the same species in each aquarium and counts the number of fish in the tank every week over a 2 month period. What is the dependent variable in this experiment? a) Oxygen levels b) 20 fish of the same species c) 2 months d) Number of fish counted every week Correct answer: D Which of the following statements are *NOT* true? a) Scientists share information about their research projects through publications, meetings, web-sites and personal communication b) Science depends on observation and measurements that can be verified by others c) The details of the experimental design deve