EFFECTS OF CLASSROOM CELL PHONE USE ON EXPECTED AND ACTUAL LEARNING

EFFECTS OF CLASSROOM CELL PHONEUSE ON EXPECTED AND ACTUAL LEARNINGARNOLD D . FROESECHRISTINA N . CARPENTERDENYSE A . INMANJESSICA R . SCHOOLEYREBECCA B . BARNESPAUL W . BRECHTJASMIN D . CHACONSterling CollegeStudies of driving indicate that the conversational aspects ofusing cell phones generate high risks from divided attention.Prior surveys document high rates at which students carryphones to and use them during class. Some experiments havedemonstrated that cell phones distract students from leaming.The present studies combined survey and experimental methodsto determine student expectations about and actual performanceunder cell phone use conditions. On the survey, students estimated the number of questions they could answer out of 10 whentexting and when not texting. For the experiment, we used arepeated measures design with simulated classroom presentations and measured performance on a 10-item quiz. Studentsexpected to lose close to 30% on a quiz and actually did loseclose to 30% when texting. We discuss implications of ourmethodology and our findings for improving student leaming.Studies of drivers using cell phonesreveal that the cognitive distraction of conversations significantiy increases accidentrisk. The National Safety Council (2010)published a literature review explainingwhy cognitive load from cell phones produces inattention blindness for drivers.messages, and manual instead of verbal"talking" as they reply. If conversationalcognitive load increases accident risk fordrivers, the same cognitive load shouldincrease errors on tests of lesson materialpresented while students are texting.Strayer and Johnston (2001) showed thatSurvey Researchlistening to music or even to a recordedResearchers have explored the disbook did not produce high accident risks, tracting effects of cell phones in classroomsas did conversing on cell phones.using surveys. Many students admit toThese findings are important for con- using cell phones for social networkingsidering the potential effects of classroom purposes in the classroom (Bayer, Klein,texting on students’ ability to leam pre- & Rubinstein, 2009; Besser, 2007;sented material. Texting is conversational, Kennedy & Smith, 2010; Rubinkam,though it involves visual instead of audi- 2010). Some studies documented perceptory "listening" as students read incoming tions of distraction from phone ringing323324 7 College Student Journal(Campbell, 2006) and from texting or sending instant messages during a class or studysession (Besser, 2007; Kennedy & Smith,2010; Levine, Waite, & Bowman, 2007).These studies employed survey responsesto evaluate effects.The typical measurement scales forsuch reports are quandtatively weak. Forexample. Besser (2007) and Kennedy andSmith (2010) measured student percepdons of the effects of cell phone use onclass performance using statements withwhich respondents either agreed or disagreed. Besser’s statement was abouttexting drawing attendon away from class,and Kennedy and Smith’s statement wasabout these acdvities helping class performance. These nominal measurementsdo not provide informadon about the quantity of expected information loss. Otherresearchers (Campbell, 2006; Levin, Waite,& Bowman, 2007) have expanded the number of response options. For example,Campbell (2006) used a 5-point Likertscale ranging from strongly agree tostrongly disagree to evaluate student atdtudes about the disruptive effects of ringingphones. Although these scales increaseresponse variability, there is no clear reladonship between level of agreement witha statement such as "when a mobile phonerings during class, it is a serious distraction" and any quandty of informadon loss.The absence of clarity about the expected size of the effect presents addidonalinterpredve problems. Some researchershave found a difference between expressedattitudes about phone risks and actualbehavior. An American Automobile Association Foundation for Traffic Safety(2008) survey showed that drivers viewedcell phone use as a serious safety risk. Nevertheless, 46% of those claiming that suchuse was an "extremely serious risk" sdllreported using their phones while drivingwithin 30 days prior to the interview.Kennedy and Smith (2010) reported similar discrepancies in student behavior.Although students generally "agreed" thatcell phones disrupted classroom leaming,they persisted in using their cell phones inthe classroom. Levels of agreement do notclearly indicate the size of the expectedeffect. If respondents agree that risk isincreased, but perceive that the risk is low,they may feel jusdfied in ignoring the risk.Experimental ResearchSome researchers have employed experimental techniques to assess actual effectsof cell phone activity on classroom-related performance. Bowman, Levine, Waite,and Gendron (2009) and Fox, Rosen, andCrawford (2009) compared comprehension scores for students who were or werenot sending instant messages during a nonclass reading task. Neither study revealeddifferences in comprehension, but completing the reading took significandy longerfor those engaged in instant messaging.These results do not generahze to a lecture or discussion-based classroomenvironment where students do not control the dming of informadon.Other researchers have experimentallyexplored distracdon from a cell phone ringing in a classroom. In two studies,researchers compared classroom scores formaterial when no phone was ringing toscores when a phone was ringing (End,Worthman, Mathews, & Wetterau, 2010;Shelton, Elliott, Eaves, & Exner, 2009). InEffects of Cell Phone Use on Learning… / 325both studies, performance deteriorated significantly for material presented during theringing condition. Performance decrementsranged from 25-40% during ringingperiods. These two studies addressed distraction effects for bystanders and left openthe question of distraction for texting performers.Ellis, Daniels, and Jauregui (2010) mostdirectly assessed the effects of texting onperformers in a real classroom context.Students in the experimental condition sentthree text messages to the instmctor during the lecture. The control grouppresumably had tumed their phones off.Experimental students scored significantly lower than control students did on a popquiz at the end of class. Although thisexperiment comes directly from a classroom setfing, sending a text message to ateacher who does not respond is likely notas distracting as a conversational textingdialogue.PurposeThe above studies begin to explore howtexting changes classroom leaming. However, their limitations suggested thefollowing research strategies. First, wedesigned both a survey to assess how muchinformation students thought they wouldlose if they were texting, and a corresponding experiment to explore the actualloss of information. Second, we generated a survey response scale that had strongernumerical properties than dichotomous orLikert-scale response options. Third, oursurvey response scale had numerical properties that matched those of ourexperimental outcome variable. This matchallowed us to compare quantity estimatesof expected quiz score changes with experimental performance scores. Finally, wedesigned an experiment that approximated both the classroom environment andstudents’ texting experiences. Hearing acell phone ringing in a class distracts leamers from lesson content. However, ifincreased cognitive load explains leamingdeficits from texting distraction, the mostinvasive distraction should occur forstudents actively engaged in texting conversations during a class. Implementingthese developments permitted us to compare expected and actual effects ofnon-class-related texting on classroomleaming. We expected that students wouldbe aware of leaming decrements producedby texting, and that their actual performance would confirm that expectation.Study 1This study employed a self-report survey to assess students’ cell phone activityin classes and their expectations of theeffects of such activity on leaming outcomes. Unlike previous studies usingself-report measures, we created a measure of anticipated leaming deficits fromtexting based on measurements commonto classroom settings.MethodParticipants. We collected surveysfrom 693 students at seven colleges anduniversities across the United States during October through December, 2009.Seven teachers at these schools administered the surveys in their classes duringclass time. Participants’ average age was20.5 years. Ninety-nine percent owned cellphones. They had owned cell phones an326 / College Student JournalTable 1Verbal and Quantitative Comparison of Self-Described TextingHow WouldYou DescribeYourself as aText User?TotalHow Often Do You Text in a Day?0 -25 2 6 – 5 0 51 -75 76-100 100+timestimestimes times times0000Emergency-only5111753Minimal1446238784Moderate13976547021Avid154117100148163average of 5.4 years and used texting functions an average of 4.1 years.Instrument. Our survey requesteddemographic information from students(summarized above), and informationabout frequency of carrying their phonesand texting frequency in various dailyactivity contexts. Participants also estimated their expected learning performanceif they texted during class. Our metric forperformance was the question, "If you werelistening to some information, and someone asked you 10 factual questions aboutthat information, estimate the number ofquestions you might be able to correctlyanswer?" Participants answered that question for two conditions—if they were andwere not texting while they listened to theinformation.Procedure. Instructors read an introductory script to their classes that providedinstructions and the informed consentoption of not completing the survey. Surveys were confidential, and studentscompleted them during a 6-minute timelimit.Total563254360682ResultsMore than half (52.8%) of our respondents described themselves as "avid users"and 90% described themselves as moderate or avid users. These verbal categoriescorresponded with reported number oftexts sent per day, r^ (682) = .612,p < .01,as shown in Table 1.Most students carry their phones toclass. Seventy-five percent reported carrying phones to class "always," and another16.4% said "most ofthe time." These carrying frequencies were lower than whenstudents performed daily errands (87%reported "always"), but higher than whenin leisure activity (72% reported "always"),at work (61% reported "always"), orattending church (46% reported "always").Students predicted scoring significantly better if not texting (M = 8.93, SD =1.68) than if texting (M= 6.01, SD = 2.25),i(676) = 31.31,/? < .01, effect size (t/ ^/Ñ)= 1.20. Low-frequency users expectedgreater decrements from texting (M=4.16,SD = 2.77) than did moderate (M = 3.01,SD = 2.24) or higher-frequency users (M= 2.61, SD = 2.41), F(2, 672) = 12.14, pEffects of Cell Phone Use on Learning… / 327< .01, effect size (rf) = .035. A Tukey posthoc test indicated that the low-frequencyusers differed significanüy from both higher-frequency users.DiscussionThese data confirm prior reports of theubiquity of cell phones in the classroom(Bayer, Klein, & Rubinstein, 2009; Besser, 2007; Kennedy & Smith, 2010;Rubinkam, 2010). They add contextualinformadon to classroom frequency data,indicating that the classroom presentsfewer inhibidons to phone use than dochurch and work setdngs.More importandy, these data present astrong metric for expected leaming effectsof phone use in the classroom. Researcherscan directly compare expected point losses on a 10-item quiz to actual performancefrom a classroom experiment.Study 2We designed a simulated classroom inwhich we manipulated student texdng. Otirgoal was to establish actual effects of texdng on quiz performance, and comparethis performance with expectadons derivedfrom the survey in Study 1.MethodParticipants. We randomly selected 82names from a complete college student list,and 40 of these students (21 men and 19women) agreed to pardcipate. We believethis procedure produced a much bettersample of students than the typical General Psychology student sample receivingcourse credit for participadon. Our samplederived from random selecdon, and par-dcipants received no incendves for participation beyond being involved in andreceiving informadon about the results ofthe project.Materials. Pardcipants brought theirpersonal cell phones to a classroom thatcontained a computer, a projector andscreen, and sound connecdvity. Studentshad access to pencils and blank paper sothey could take notes. Another room acrossthe hallway was available for break periods between sessions and forco-experimenters who texted participantsduring tesdng.We prepared two lessons for participants. Each lesson provided author andcontent information about the books,"Young Men and Fire" by NormanMaclean (1992), and "Let the Great WorldSpin" by Colum McCann (2009). No pardcipant indicated any prior knowledge ofeither book. Each presentadon consistedof a prerecorded narrative and accompanying, self-timed, PowerPoint presentadonthat lasted about 6 minutes. The presentations simulated classroom teaching. Foreach presentadon, we prepared a 10-itemmultiple-choice quiz. We pretested thequizzes with people who had not read thebooks and modified them so that pretestscores were close to chance levels.Procedure. We tested all participantstwice—once while texdng and once whilenot texdng. We counterbalanced all storyand condition orders, and each storyappeared an equal number of dmes in eachorder condidon. We tested texdng and nontexting pardcipants simultaneously in smallgroups depending on when pardcipantscould attend. Texdng and non-texdng par-328 / College Student Journalticipants sat on different sides of the roomto reduce distraction. Co-experimenterssat in the room across the hall.We told all participants that they wouldwatch an informational presentation; theycould take notes if they desired; and theyshould try to retain the presented information for a quiz following a 5-minutebreak. During the break, participants hadaccess to refreshments. They were told notto discuss the content of the presentation.We identified the texting condition foreach participant before each presentation.The texting participants set their phoneson vibrate, and were free to respond immediately to any texts that arrived. Thenon-texting participants turned off thevibrate function, placed their phones outof sight and did not use their phones during the presentation. Following the firstquiz, the groups switched conditions forthe second presentation.The co-experimenters confirmed phonefunctionality with participants before theexperiment began. Following confirmation, the experimenter signaledco-experimenters to begin texting the participants. When all texting participantsreceived their first message, the experimenter started the PowerPoint presentation.Co-experimenters exchanged messages asquickly as possible with assigned participants throughout the presentation. Weprepared a list of texting topics involvinggeneral introductory information, butallowed texting content to develop spontaneously throughout the interactions.ResultsQuiz scores were significantly lowerwhen students texted (M = 6.02, SD =2.224) than when they did not text (M =8.25, SD = 1.597), i(39) = 5.34, p < .01,effect size (t/ ^//V) = .84. The difference inscores represented a 27% decline duringtexting from the non-texting performance.Neither the story during which they texted,nor the order of texting and non-texting,produced different results.For a convenience sample of 15 students, we recorded the time participantsactually spent reading or texting on theirphone during the texting phase. Participants spent an average of 2.69 minutesengaged in texting during the presentation.The range of texting times was from 1.5 to4.25 minutes. Time engaged in texting wasnegatively, though not significantly, correlated with quiz score in the texting phase,076DiscussionOur data support a prior report (Ellis,Daniels, & Jauregui, 2010) of deleteriouseffects of texting on classroom leaming..Score reductions for texting conditionswere greater in our experiment than in theprior experiment. Our methodologicaladdition of conversational texting mayaccount for our greater score reductions.Although the correlation between texting time and texting score was notsignificant, the direction and size of thecorrelation leave open possibilities thatlevel of engagement in texting is a factorin losing classroom information.Our method presents a strong tool forevaluating the effects of texting on leam-Effects of Cell Phone Use on Learning… 7 329ing. The counterbalanced, repeated-measures design controlled subject and ordervariables. The pre-recorded presentadonsequated lesson materials for all pardcipants across tesdng sessions. Nevertheless,due to phone connectivity differences, pardcipants spent widely differing amountsof dme actually engaged in texdng. Weexpect that methodological refinementscould demonstrate even greater loss ofinformadon than we found.General DiscussionOur research successfully implemented a survey measure of students’expectadons about the effects of texdngon leaming that was comparable to typical classroom measures—predicted quizscores. The measure is quantitativelystrong—a rado measurement scale—andeasy for respondents to understand. Thedata confirmed that self-report measurescan provide informadon that is verified inexperimental outcome studies. Oneremaining limitadon is that students mayfail to account for chance performance levels associated with multiple-choicequesdons. With four response altemadves,that chance level—25%—represents nosignificant leaming. It is likely that thosestudents who predicted scores lower thanchance did not understand this baselineminimum.The texdng manipuladon in the simulated classroom environment more closelyapproximated texting during real class sessions than previous experiments. Studentsin the texdng condidon responded to messages from their own friends as well asfrom co-experimenters. The messagesengaged pardcipants in conversadon, a procedure that the driving studies (NadonalSafety Council, 2010; Strayer & Johnston,2001) suggested as a source of distractionand one that was missing from the Ellis,Daniels, and Jaurgui (2010) study. Thisengagement likely accounted for moreinformadon loss in our study than Ellis,Daniels, and Jaurgui (2010) found. Furthermore, the conversations occurredsimultaneously with the lesson presentation, unlike the studies reported byBowman, Levine, Waite, and Gendron(2009) and Fox, Rosen, and Crawford(2009). The differences in informadon lossthat we obtained, in contrast to Bowman,Levine, Waite, and Gendron (2009) andFox, Rosen, and Crawford (2009) supportthe idea that cognitive load increases wheninformadon presentadon conflicts with texting communications. One remainingdifference between our experimental setting and a real classroom is that somestudents commented about how differentit was to freely text during a classroompresentation.Our data confirm that students expecttexdng to dismpt their classroom leaming,and that texting does dismpt leaming. Thereal score declines (27%) approximatedthe expected dechnes (33%). The somewhat higher expected declines could haveoccurred as students failed to account forthe 25% chance baseline and from texdngrequirements that did not occupy all ofthelesson dme. The corresponding declinesfor self-report and experimental measurements suggest that students are aware thatusing cell phones for personal communication in class compromises classroom330 / College Student Journalleaming. Thus, our data support the valueof self-reports of the effects of using cellphones on leaming, at least as presentedwith the measurement tools we used.Survey participants varied considerablyin their score predictions under texting conditions. Some participants expected nodetrimental effects of texting. Similarly,experimental participants varied considerably in their quiz scores under textingconditions. Some texting participantsanswered all questions correctly. We donot know if each participant’s expectedand actual performance measures were correlated because different participantscompleted the survey and the experiment.These data could reflect the same kind ofdiscrepancies reported by the AmericanAutomobile Association Foundation forTraffic Safety (2008) between participants’expectations of safety risks for others butfalse immunity from risk for self. Furtherresearch could solicit information lossexpectations from experimental participants to determine whether students canaccurately predict their own distractibilityStanovich (2009) summarized twoaspects of rationality—epistemic andinstmmental. Epistemic rationality existswhen a person’s view ofthe way the worldworks matches the way it actually works.The correspondence of average expectedand actual losses in our studies suggests adegree of epistemic rationality. Participantsreally do know what happens when students text. Instrumental rationality isevident when a person sets a goal and follows appropriate steps to achieve that goal.Our data suggest deficits in instmmentalrationality for students who pay to becomeeducated, yet choose to engage in counterproductive behaviors.Given that students generally expecttexting to dismpt their leaming, researcherscan reasonably ask why students riskpotential failure to maintain social contact? Wei and Wang (2010) recentlyexplored two models of student motivation for classroom texting. They predictedthat instmctor immediacy—making eyecontact, calling students by name, talkingwith students outside of class, among otherbehaviors—could enhance students’ motivation to learn and thus reduce texting.Altematively, students’ habits and gratifications they receivefi-omthe activity couldmaintain texting. Their data confirmed thatimmediacy enhanced motivation to leam,but that motivation did not correlate withtexting rates. They concluded that thehabits and gratifications model better fitstheir data. These results raise questionsabout how phone carrying habits and phonechecking impulses relate to instmctionalvariables. Students may benefit from knowing whether carrying their phones to classincreases their impulses to check for messages. Likewise, teachers may want toknow if interruptions to lesson flowincrease students’ urges to check theirphones. These possibilities present fertileground for future research.Finally, faculty variations in handlingtexting events in classrooms may affectstudent behaviors in ways that alter leaming. Further research could exploredifferences between faculty and studentsin perceptions of the effects of texting aswell as of techniques for handling unwanted texting in class. Knowing suchperceptions and the effectiveness of inter-Effects of Cell Phone Use on Learning… / 331venfion techniques in the context of thedemonstrated effects of texting couldimprove classroom environments andenhance student learning.Bowman, L. L., Levine, L. E., Waite, B. M., &Gendron, M. (2009). Can students really multitask? An experimental study of instantmessaging while reading. Computers & Education, 54, 927 – 931 .doi : 10.1016/j .compedu 2009.09.024.Author NoteDenyse A. Inman is now at the Schoolof Behavioral Sciences, California BaptistUniversity. Christina N. Carpenter is nowat Bailey, Colorado. Jasmin D. Chacon isnow at the Department of Psychology, Gallaudet University.We thank Brian Allen, AndreaMehringer, and Jeffi-ey Ropp for assistancein conducting the research, coding data,and discussing our ideas.Address correspondence concerningthis article to Arnold D. Froese, Psychology Department, Sterling College, Sterling,KS 67579. E-mail: an-oese46@gmail.comCampbell, S. W. (2006) Perceptions of mobilephones in college classrooms: Ringing, cheating, and classroom policies. CommunicationEducation, 55, 280 – 294. doi: 10.1080/03634520600748573.ReferencesAmerican Automobile Association Foundation forTraffic Safety. (2008). 2008 Traffic Safety Culture Index. Washington, DC: AAA Foundationfor Traffic Safety. Downloaded fromhttp://www.aaafoundation.ore/pdf/CellPhone.’iandDrivinpRe…