Exploring the Impact of the Autobiographical Story on Learning English as a Foreign Language: Case Study in the Primary Classroom
The empirical evidence in this dissertation has been drawn from a case study involving a group of fourth grade primary students (aged 9 to 10) who took part in two storytelling activities held on different lessons; one being a listening activity, and the other being a listening and reading activity.
The first activity consisted in listening to an autobiographical story on my favourite hobby told by myself using several A3 size colour images as visual support. The second activity consisted in listening to an ELEANITZ story on a CD recording, for the first time, while students read it on individual copies of the storybook. The storybook (Elorza, Fano, & Tejada, 2005) combines text and colour illustrations in all pages.
The main methodological limitations of this study, namely the size of the sample, the use of convenience sampling, the differences in the conditions for the storytelling of both stories, and the absence of counterbalancing measures relate to aspects explained in section 1.1, which outlines the context of the study.
The participants were a class in the fourth grade of primary (children aged 9 to 10) at Astigarragako Herri Eskola (Gipuzkoa, Spain). There were 26 students in the class, 13 girls and 13 boys. Some students were ill on the days when the two stories were told. As a result, 22 students took part in the autobiographical storytelling (11 girls and 11 boys), and 21 took part in the ELEANITZ storytelling (10 girls and 11 boys). Overall, 17 took part in both activities, 5 took part only in the first one, and 4 took part only in the second one.
This group of students was selected through convenience sampling among the several groups of students whose EFL lessons I attended regularly during my 12-week school placement. An additional criterion taken into account for selecting the sample was the potential interest towards autobiographical stories. During this school placement, I took part in EFL lessons for two EYFS grades and three primary grades. The primary grades involved children aged 6 to 7, 7 to 8 and 9 to 10 respectively. The latter showed greater interest by far in personal stories concerning the EFL teacher and myself, and were therefore selected for the sample.
Regarding the level of proficiency in English, the sample had an additional degree of diversity beyond what would be expected in any Basque school, where some students start taking extra English lessons outside school as early as the beginning of primary. The EFL teacher reported that there were great disparities in the proficiency level within the group, as three or four students barely met the most basic objectives, and were a great distance away from those classmates who took extra English lessons outside school.
There are additional factors that need to be taken into account concerning diversity within the sample. Firstly, one of the students, who joined the group on the previous academic year, was in the Language Reinforcement Programme, as her L1 was Spanish and she had had no previous contact with either Basque or English before. Secondly, another student had dyslexia and received support from a Special Educational Needs teacher at school. Lastly, another student had joined the group two academic years before, showed signs of deep disengagement at school in all subjects, and was under study to assess if support needed to be given to her.
The activities were completed during two consecutive lessons held one week apart, as the students were away on a school trip for three days in between. The general procedure followed in both activities was the task-feedback circle proposed by Scrinever (1994).
Students sat at their desks, as they usually do in the English lessons. Desks were arranged in six groups of four to five students.
Both lessons were recorded on video, using two video cameras in order to record the faces of the students and myself, who was standing opposite to them. The size and spatial arrangement of the classroom did not make it possible to record all students with the video cameras, which were in a fixed position.
In the first lesson, I introduced briefly the activities of the two consecutive lessons and explained the general procedure that would be applied in both. Prior to that, their general classroom teacher had explained to them the purpose of the research, as outlined in section 3.5.
Both lessons followed the same pre-task phase, composed of the following steps:
Both the vocabulary preview and the question preview were included among the prelistening activities, as they have been reported to improve listening and reading comprehension (Elkhafaifi, 2005; Zeleke, 2013).
After the pre-task phase, in the first lesson I told the autobiographical story. In the second lesson, each student was given one copy of the storybook, and listened to the CD recording of the ELEANITZ story, while reading it in silence individually.
After the storytelling, I followed the same procedure in both lessons: I read each question in the questionnaire aloud, leaving time for the participants to answer it. Those who needed it were given extra time to answer the questions.
The second lesson ended with a short post-test survey, to assess participants’ attitudes towards the two stories, which can be found in Appendix D.
During the pre-task discussions, the students understood readily what the local questions requested, and their doubts concerned mainly vocabulary that was new to them. On the contrary, I experienced great difficulty in explaining the meaning of global questions four and seven of the questionnaire about the autobiographical story. That is the reason why question four, which dealt with summarising the story, was reformulated in the questionnaire for the ELEANITZ story, in order to make it easier to understand for the participants.
The storytelling of the autobiographical story lasted 11 minutes, and the ELEANITZ story’s CD recording lasted 5 minutes and 40 seconds.
Finally, there were some differences in the setting between the two lessons, as there was considerable noise coming from the corridor while the CD recording of the ELEANITZ story was played during the second lesson. In addition, the volume and sound quality of the CD player in that lesson were noticeably poorer than my live voice telling the autobiographical story in the first lesson.
3.3 The stories
The story taken from the ELEANITZ course materials is titled “Where’s the magician?” (Elorza et al., 2005). In summary, the story tells the adventures of six friends when they go to see a show by the Tilly Troupe in the local theatre. These six children are the characters in the stories of the “Story Projects” material that students go through during the third and fourth grades of primary. The friends arrange to go to the show and, once inside, the master of ceremonies requests volunteers to perform in the show, because the Tilly Troupe has not arrived yet. Two of the friends do magic tricks, other two tell jokes dressed up as clowns, and the last two sing and dance. Their performances are a great success.
Although the stories in ELEANITZ are usually introduced to students through collective dramatisation, this particular story is typically presented reading the storybook in silence while listening to the text being read on a CD recording (Gipuzkoako Ikastolen Elkartea, 2009).
The autobiographical story was titled “Hiking in Japan.” Careful thought was given to aspects linked with authenticity of the text due to its relationship with student motivation (Guariento & Morley, 2001), as well as to the importance of truthfulness in autobiographical stories (Leggo, 2007). As a result, it was decided to tell a truthful story, although it was adapted to the participants and the context. I wanted to share a personal story as a language learner and my struggles along the way, of the sort that Kazuyoshi (2002) reports being successful at creating a collaborative learning environment. Unfortunately, the school policy stated that EFL teachers should pretend to be native English speakers. Thus, I chose a personal story about my holidays in Japan, knowing just a few words of Japanese, hoping that it would echo the first experiences of the participants with English.
In summary, the autobiographical story tells how my husband and I bought food and prepared our rucksacks to spend a week hiking in the Japanese Alps. It explains our experience sleeping in our tent and in a mountain hut, our difficulties understanding maps and signs, and our experience making two Japanese friends, even though we spoke no Japanese and they spoke no English.
The semi-scripted text can be found in Appendix B. The autobiographical story was shaped following criteria to make input comprehensible (see Appendix C). However, this semi-scripted text was not read aloud, nor memorised, as recommended by Buck (2001).
The entire text of the ELEANITZ story and the semi-scripted text of the autobiographical story were subjected to the analysis of readability shown in Table 2, using the readability test performed by Microsoft Word 2010.
The Flesch Reading Ease test rates text on a 100-point scale. The higher the score, the easier it is to understand the text. The formula for the Flesch Reading Ease score is:
The Flesch-Kincaid Grade Level test rates text on a U.S. school grade level. The formula for the Flesch-Kincaid Grade Level score is:
Where, ASL is the Average Sentence Length (the number of words divided by the number of sentences), and ASW is the Average of Syllables per Word (the number of syllables divided by the number of words).
Most standard texts have a Flesch Reading Ease score of 60-70. According to Zamanian & Heydari (2012), Flesch Reading Ease scores of 70-80 are interpreted as being fairly easy and suitable for seventh graders (ages 12 to 13), while scores of 80-90 are interpreted as being easy and suitable for sixth graders (ages 11 to 12).
I am aware that readability is a wide field of research and that alternatives to traditional readability indices have been proposed (e.g. Crossley, Allen, & McNamara, 2011; Crossley, Greenfield, & McNamara, 2008; Kouame, 2010), but it is beyond the scope of this study to analyse these issues in depth.
The information on readability is brought here only to illustrate that, as a starting point, the text of the autobiographical story was more challenging than the text of the ELEANITZ story.
3.4 Instruments for data collection
For methodological triangulation purposes, three data collection instruments were used in order to answer the three research questions outlined in section 1.2. The first instrument for collecting data in the case study was the questionnaire, which contained eight questions. Questions one to seven formed the comprehension questionnaire, which is the data collection tool linked with the first research question. One comprehension questionnaire was designed and passed for each story in order to test general language proficiency related to the level of comprehension.
The other two data collection tools relate to the second research question. Student engagement was assessed using a behavioural observation instrument, which was filled while watching the video recordings of the two lessons. It is worth noticing that technical and spatial limitations in the EFL classroom had prevented from recording all the students who were present in both lessons.
A short post-survey test, run after the questionnaire of the ELEANITZ story, was used to study the perception and attitudes of the participants regarding both stories. This information was supplemented with the eighth question included in the questionnaires, which allowed assessing aspects linked to motivation.
Finally, regarding the third research question, three open-ended questions (questions 7 and 8 in the questionnaires, and question B in the post-test survey) allowed to test whether the participants used English or not in their responses.
The following subsections explain each instrument in detail.
3.4.1 The questionnaires
As it has been mentioned, the first seven questions of the questionnaire (see Appendix D) formed the comprehension questionnaire. The questionnaire had an eighth question, which gave a chance to make connections between the story and the listener’s personal experience. The purpose of this last question was to assess aspects related to motivation, as it has been reported that content which is interesting and relevant for students plays an important role as an intrinsic factor, which improves levels of motivation (Nicholson, 2013). A story that one can relate to is expected to be interesting and relevant in this sense. This question complemented other motivational aspects analysed through the behavioural observation instrument explained in subsection 3.4.2.
Focusing on the comprehension questionnaire, there is a wide range of variables that affect comprehension. Besides the factors that influence comprehension and take place at the time of telling a story that have been analysed in section 2.6, there are also important aspects linked with the design of a questionnaire that can have an effect on comprehension (Buck, 2001; Ellis & Brewster, 2002; Guariento & Morley, 2001; Révész & Brunfaut, 2013; Rubin, 1994; Vandergrift, 2007). Appendix C explains the criteria that have guided the design of the comprehension questionnaires in this case study.
The main purpose of the comprehension questionnaires was to test general language proficiency. The first three questions were local questions that referred to exact details of the story, and the following four questions were global questions, which required synthesising information and making inferences. One of the global questions was related to understanding gist, one required integrating information from different passages of the story, another one asked the listener to draw a practical conclusion out of the story, and the seventh question required making an inference on the meaning of the story. Local and global questions were included for the purpose of assessing listening comprehension in a global way, as proposed by Buck (2001).
Questions 1 to 6 were traditional multiple-choice questions with four options, while questions 7 and 8 were open-ended questions. Traditional multiple-choice questions were chosen despite the fact that designing them is difficult (Buck, 2001), because this response format has been reported to help listeners understand oral texts and perform better in the test, especially when target language proficiency is low (Hsiao-fang, 2004).
Each of the multiple-choice questions had one or two distractors among the four possible answers that referred to events or details mentioned in the story, although they did not answer the question accurately. In the autobiographical story those details were sometimes introduced in the images used as visual aid, rather than in the text itself. In addition to the correct answer, one or two distractors among the options referred to aspects that were not part of the story. For instance, in the third question of the questionnaire on the autobiographical story, the correct answer was 2000 m, but the story mentioned that the mountain range itself has an altitude of 3000 m. The other two possible answers, 1000 m and 4000 m, were not mentioned in the story at all. The distractors that referred to aspects mentioned elsewhere in the story were introduced in order to detect partial or incomplete understanding of the story.
It must be remembered that the purpose of the case study regarding the first research question was to analyse comprehension, and not articulation ability in the target language. Thus, students were allowed to answer the open-ended questions in L1, in the same fashion as other studies have done (Elkhafaifi, 2005; Hsiao-fang, 2004; G.-P. Park, 2004; Révész & Brunfaut, 2013; Woodall, 2010). Additional evidence supporting this decision can be found in studies that have not allowed the use of L1 in open-ended questions and suggest that articulation ability may have caused test bias (McKendry & Murphy, 2011). Besides, allowing the students to use their L1 in the open-ended questions gave the opportunity to test the use of English in those questions, the object of the third research question.
Finally, the comprehension questionnaires were closely aligned with the official Basque curriculum contents for the second cycle of primary, which encompasses the third and fourth grades of primary, regarding EFL, as well as Basque and Spanish language (Gobierno Vasco. Departamento de Educación, Universidad e Investigación, 2010).
3.4.2 The behavioural observation instrument
Connected with the second research question, a behavioural observation instrument was created, in order to assess student engagement during the two lessons that had been recorded. The increasing widespread of the use of computers in classrooms has recently allowed for detailed quantitative measuring of student behaviour (e.g. Bulger, Mayer, Almeroth, & Blau, 2008). Unfortunately, this was not our case, and the analysis of student engagement took into account observable behavioural measures, such as on-task and off-task behaviour frequency, adapting the behavioural observation checklists created by Ciampa (2012), Morgan (2008) and Turner et al. (2014).
On-task and off-task behaviours were recorded manually on the behavioural observation instrument using momentary time-sampling (sweeps) at seven-minute intervals during the video recording of each session. Every seven minutes along the recording, each student was briefly observed and his or her on-task or off-task behaviour was recorded on the behavioural observation instrument.
This technique was chosen for the sake of time and effort, bearing in mind that the results of the behavioural observation instrument were going to be partial, as only some students were recorded, and therefore the information drawn from it would be complementary to the data collected with the other tools. On the other hand, the fact that I told the autobiographical story myself prevented the use of other techniques based on field notes.
3.4.3 The post-test survey
The post-test survey (see Appendix D) was also connected to the second research question and consisted of two questions, one multiple-choice and one open-ended. It intended to measure the perception and attitudes of the participants regarding both stories. It was kept very simple, bearing in mind that the participants would answer it after the questionnaire on the ELEANITZ story. Like in the open-ended questions in the questionnaires, participants were allowed to answer the open-ended question in L1, hence allowing for the collection of data related to the third research question.
As it has been pointed out in subsection 3.4.1, the post-survey test was complemented with the eighth question in the questionnaires. Following the criteria applied in similar studies (Taguchi, 2006), measures of motivation and attitude based on self-reporting were avoided. Therefore, participants were asked if the story reminded them of anything that had happened to them (question 8 of the questionnaire), which story they liked best (question A of the post-test survey), as well as which story they would choose if their general classroom teacher asked them to summarise one of the stories, and why (question B of the post-test survey).
3.5 Ethical issues
Although no specific guidelines regarding ethical issues for research by undergraduate students were communicated by the university, I have considered ethical aspects in my research design.
This study does not involve discussion of sensitive topics. No harmful or negative consequences could be expected for the participants or me during the activities involved.
In order to ensure that the students understood the activities proposed to them and the context of the study, I asked their general classroom teacher to explain them in Basque, based on some notes I prepared beforehand.
In summary, students were aware that they were free to decide whether to take part in the activities or not (listening to the ELEANITZ story was compulsory, though), that the results would not be taken into account in the assessment of the English unit, and that I would give them feedback on the findings of the study.
4. Data analysis and results
This chapter analyses the data collected in the study in order to draw the key results. The chapter is divided in sections, named after the instruments for data collection, which summarise the findings derived from the data collection tools, and highlight the key results obtained.
4.1 The comprehension questionnaires
Questions 1 to 7 of the questionnaires formed the comprehension questionnaires, which relate to the first research question, and are analysed in this section. The results of the eighth question are discussed in sections 4.3 and 4.4, as that question deals primarily with motivational and attitudinal aspects related to the second research question.
The answers to the comprehension questionnaires were evaluated using a scoring rubric. The rubric was designed following widely accepted criteria and guidelines and was based on several models (Allen & Tanner, 2006; Andrade, 2005; Arter, 2000; Birky, 2012; Boston, 2002; Cheyney, 2010; De La Paz, 2009; Hegler, 2003; Livingston, 2012; Mertler, 2001; Moskal, 2000b, 2003; Whittaker, Salend, & Duhaney, 2001; Yoshina & Harada, 2007). Appendix E contains more details on the process of creation of the rubric, and the rubric itself.
The number of participants in the first lesson, which dealt with the autobiographical story, was different from the number of participants in the second lesson, where the ELEANITZ story was used, as shown in Table 3. This had effects over the total maximum possible score for each participant, as will be discussed along this section.
Table F-11 in Appendix F shows the scores that each participant obtained in the comprehension questionnaires of both stories.
Scores were awarded as follows: 2 points for a full response; 1 point for a partial response (the answer referred to an event or detail which was part of the story, but did not answer the question accurately or answered it partially); and 0 points for incorrect responses (the answer referred to an event or detail not mentioned in the story) or lack of response.
The quantitative data derived from the comprehension questionnaires was analysed using simple descriptive statistics, such as frequencies, means, medians, standard deviations and percentages.
Table 4 summarises the overall achievement of the participants in the comprehension questionnaires (N = 26). For instance, the first value shows that two participants (7.7% over the 26) gave full responses in 80 to 100% of the questions in both questionnaires.
Therefore, 46.2% of the participants gave full responses in 40 to 60% of the questions of both questionnaires, 57.7% gave partial responses in 20 to 40% of the questions, and 57.7% of the participants did not respond or gave an incorrect answer in less than 20% of the questions.
Discarding the scores of the nine participants who, having taken part only in one lesson, had answered just one questionnaire would have simplified considerably the data analysis. Yet, that would have meant leaving aside the scores of 35% of the participants. Therefore, it was decided to analyse all data, and calculate weighted values where applicable.
The maximum possible score for participants who answered both questionnaires, as well as for those who only took one, are summarised in Table 5.
The scores for each of the two comprehension questionnaires are best viewed in the form of percentages over the maximum possible score. Table 6 shows that overall participants obtained higher comprehension scores for the autobiographical story than for the ELEANITZ story, with a difference of 3.59 points on the mean (%) and 8.32 on the median (%).
On the other hand, local questions scored higher than global questions, with a difference of 19.63 points on the mean (%) and 15.71 points on the median (%). However, there were noticeable differences between both stories: in the autobiographical story, local questions scored much higher than global questions, having a difference of 34.28 points on the mean (%) and 35.42 points on the median (%). In the ELEANITZ story, local questions and global questions showed a very different pattern, with a difference of 3.37 points in favour of local questions on the mean (%) and 12.50 points in favour of global questions on the median (%).
These contradictory results are in line with previous studies, which have proven the complex nature of the interaction of question types with background and linguistic knowledge (G.-P. Park, 2004),
Finally, a paired samples t-test was conducted for the participants who took both comprehension questionnaires (n = 17), based on the scores for the autobiographical story (M = 9.88, SD = 2.26) and the ELEANITZ story (M = 8.71, SD = 2.73), t(16) = 1.51, two-tail p = .15. The results showed no statistically significant differences between the scores of the participants in the two comprehension questionnaires.
Since p value depends on sample size, the sample size independent effect size was also calculated for the scores of the 17 participants who answered both comprehension questionnaires; Cohen’s d = 0.48, and dunbiased = 0.46. The latter (sometimes called Hedge’s g) was calculated because it is more suitable than Cohen’s d when sample size is small (Fritz, Morris, & Richler, 2012). Hattie (2009) has established that in educational contexts Cohen’s d > 0.40 should be interpreted as the method or intervention researched having desired effects.
4.2 The behavioural observation instrument
The behavioural observation instrument was related to the second research question, which dealt with differences in motivation and engagement. The results of this instrument were supplemented with the questions analysed in section 4.3.
The video recordings of the two lessons had different lengths because the lesson around the ELEANITZ story took longer than the lesson on the autobiographical story. Therefore, students were observed seven times during the lesson on the autobiographical story, and eight times during the lesson on the ELEANITZ story. The last observation of each lesson was discarded because it fell within an interval of the video recording where the activity itself had already finished, and students were handing in their questionnaires and tidying up their desks.
It was difficult to observe on-task and off-task behaviour during the storytelling of the ELEANITZ story, since the participants were looking down on their storybooks, but it was not possible to observe if they were actually reading and following the story. A conservative approach was adopted, and behaviour was considered off-task only when it was obvious.
On the other hand, even though the number of participants observed in both lesson is the same, they correspond to different individuals, as 10 participants were observed in the two lessons, 4 only in the first lesson, and 4 only in the second lesson.
All off-task behaviour observed was categorised as stalling (looking out the window, flicking through papers, playing with the pencil case). Only one participant showed off-task behaviour in the sixth sweep during the lesson on the autobiographical story, while the participants answered the questionnaire as I read one of the questions aloud. This was actually the student who showed signs of deep disengagement at school in all subjects, and who only took part in the first lesson.
During the lesson on the ELEANITZ story, four participants showed off-task behaviour in the third sweep, while participants predicted the contents of the story based on the cover of the storybook during the pre-task phase of the activity, and one participant did the same in the third and sixth sweeps. The sixth sweep of the second lesson corresponds to the moment when the participants were listening to the ELEANITZ story being told on the CD recording.
Among the participants who showed off-task behaviour, three took part only in one lesson – including the participant with off-task behaviour in the first lesson – and three took part in both lessons.
4.3 The last question of the questionnaires and the post-test survey
The eighth question in the questionnaires, together with the two questions of the post-test survey, served to analyse engagement, motivation and attitude towards the two stories, which relate to the second research question. Therefore, this information complements the observation of student behaviour explained in section 4.2.
Table 8 shows that when asked if the autobiographical story reminded them of anything that had happened to them in the past (question eight of the questionnaires), 41% of the participants did so affirmatively. On the contrary, the ELEANITZ story triggered no recalls of autobiographical stories in the participants.
The contents of the autobiographical stories written by the participants in response to the eighth question of the questionnaire reveal the elements of the autobiographical story that they related to mostly. Five participants mentioned particular mountains nearby (Santiagomendi, Adarra, Onddi), and two mentioned hiking.
Most significantly, four participants narrated stories of incidents (three in the mountains, one elsewhere), which echoed the part of the autobiographical story on the man who had to be rescued by a helicopter. It is worth mentioning that 16 of the 22 participants had summarised the autobiographical story in question 4 of the comprehension questionnaire selecting the option that stated that the story was about a man who had a heart attack and had to be rescued in a helicopter.
Given the little time that the participants had to answer the questionnaire, their stories ranged from little more details than the name of the mountain where their personal story took place, to elaborate narratives such as “One day I went hiking with my cousins and uncles. While we were in the mountains, we saw a chapel and went inside, and we were locked. My cousin made a call on the mobile phone and in the end; a man came to rescue us.”
As regards the post-test survey, Table 9 shows that the autobiographical story was liked best, and the participants would prefer it to the ELEANITZ story if they had to choose one to summarise. In particular, 71% of the participants liked the autobiographical story more, and 47% would prefer to summarise it.
Seven participants explained why they liked the autobiographical story more than the ELEANITZ story. Two of them showed emotional connections with mountain landscapes. The remaining five claimed that the autobiographical story was more interesting, more fun and/or nicer.
In contrast, among the three participants who liked the ELEANITZ story better, two justified their choice because it was easier, while the third added to that the fact that there was dance and magic in the story. Interestingly, one of these students obtained the highest score in the sum of both comprehension questionnaires shown in Figure 2, another student was also in the top third of that list, and the third student right in the middle.
Figure 2. Score for each participant in the comprehension questionnaires
Finally, considering that only 6 out of the 17 participants who answered the two questionnaires answered the eighth question in both, the statistical significance of the results was not tested. The same decision was taken for the post-survey questions.
4.4 Presence of English in the open-ended questions
Questions 7 and 8 in the questionnaires and question B in the post-test survey were open-ended questions, allowing for the collection of data related to the third research question on the use of English in the responses. The participants were given the choice to answer those questions in L1, and they were not encouraged in any way to use English. Therefore, it was a surprise to find that some participants used English when they answered those questions, as Table 10 shows.
However, the participants only included explanations in English in the questionnaire on the autobiographical story. Among the three participants who used some English in their explanations in that questionnaire, two only took part in that lesson. The only participant who used some English in the post-test survey attended just the second lesson.
It is interesting to notice that two of the participants who used some English in the questionnaire actually obtained the two lowest scores in the sum of both comprehension questionnaires shown in Figure 2. The third participant was also among the bottom third of the list, while the participant who used some English in the post-test survey obtained the second best score.
Thus, mainly low achievers in the comprehension questionnaires used English in their explanations of the open-ended questions. These results are consistent with previous studies that explored the relationship between listening test scores and motivation orientations, finding that “a high degree of motivation does not appear to be a reliable predictor of proficiency in L2 listening comprehension” (Vandergrift, 2005, p. 79).Continued on Next Page »