PART ONE: RESEARCH DESIGN

1. A title for your study

A mixed methods study of student perceptions of the qualities of highly effective traditional Catholic secondary school teachers.

2. A description of the research context/background/problem leading to your study (three to five sentences approximately)

It is a well-established and researched fact that individually and collectively, teachers play a pivotal role in facilitating student success (Duncan, Gurria, & Leeuwen, 2011; Hattie, 2009; Reddy, Fabiano, & Jimerson, 2013). Moreover, there is much evidence to suggest that the level of teacher quality is linked with the level of student performance (Darling-Hammond & Youngs, 2002; Goe & Stickler, 2008; Hattie, 2003; Leigh, 2010). Although recent research on teacher quality has identified general traits (Canter, 2014; Coles, Owens, Serrano, Slavec, & Evans, 2015; Cooper, Hirn, & Scott, 2015; Spilt, Hughes, Wu, & Kwok, 2012), Jimerson & Haddock (2015) advocate “further research regarding teacher effectiveness” (p488). Consequently, this study proposes looking into teacher effectiveness within the sphere of traditional Catholicism, an area which is gradually burgeoning (Society of Saint Pius X, 2016; Wikipedia, 2017a) and hitherto unexplored. Additionally, as Sutcliffe (2011) suggests, “it is important to examine high school students’ perceptions of teacher quality” (p24).

3. Describe your study aims

The aim of this mixed methods exploratory sequential study is to explore and examine student perceptions of traditional Catholic secondary school teacher effectiveness in order to identify and eventually come up with a list of ten top characteristics for this group. Findings will be used to inform future teacher education programmes which have a specific emphasis on training Traditional Catholic teachers. The nature of the exploratory sequential study is such that it will be divided into two stages (Onwuegbuzie & Collins, 2007). The aim of the first stage of the study is to collect qualitative data from select samples of students in order to then identify general themes/characteristics using an inductive approach to data analysis (Schulz, 2012). The aim of the second stage of this study is to further identify a top ten list of characteristics via a quantitative survey which will ask students to choose from a list of key characteristics and then ask them to rank the top ten. The results would then be published.

4. What are your qualitative research question (s)?
   1. What elements set a highly effective traditional Catholic secondary school teacher apart from others?
      1. What characteristics do they have?
      2. What teaching methods do they use?
      3. What qualities/virtues do they possess?
      4. What behaviours do they exhibit?
      5. What spiritual qualities do they have?
   2. How do highly effective Catholic secondary school teachers promote learning and engagement?
5. What are your quantitative research question (s)?
   3. Of all the characteristics identified, which are considered the most appealing?
6. What are your mixed methods research question (s)?
   4. Which characteristics of highly effective traditional catholic teachers are considered most appealing and to what extent?
7. Describe the qualitative data that you will collect and how you will analyse it. What methods will you use? Discuss some of the key aspects of the data that you wish to collect (e.g. Interview question, what you will look for in observations etc.). How many participants and how will you chose them (sampling) (approximately 300 words).

I intend taking a focus group approach to the collection of qualitative data. Focus groups allow participants to interact with one another, share and compare and often generate new insights beyond what individual interviews can do (Carey, 2016; Pedersen et al., 2016). Furthermore, given the fact that participants will be secondary age students, I thought they would feel more comfortable with each other rather than in an individual interview alone with an adult. At the same time focus group interviews give the researcher an opportunity to hear the language of the participants and explore the topic in more depth (Pedersen et al., 2016).

In regard to the number of participants I intend using what Onwuegbuzie & Collins (2007) call cluster sampling. That is select focus groups consisting of 10 students from each of 10 different traditional Catholic schools so that qualitative data is collected from a total of 100 participants. It was thought advisable to select clusters of senior age students, that is, students in their last year of school only, for two reasons. Firstly, older students have years of experience behind them and are able to reflect on those experiences, and secondly, older students are more mature.

In regard to key aspects of the data, it is hoped that semi-structured questions such as “reflecting on your own personal learning experiences, what examples of effective catholic teaching have you seen?” and “what makes a traditional catholic teacher great?” would elicit responses from the students which the researcher could then explore in more depth. Data would be collected via a voice recorder then from the recording a conversation analysis would be undertaken. This involves constructing a transcript of the interviews and then analysing the data using an inductive approach, that is finding themes/characteristics and recurring themes/characteristics which have emerged from the data (Schulz, 2012; Wikipedia, 2017b). These popular themes could then be used to develop a quantitative instrument for the next stage of the research project.

A few factors would also need to be considered during data collection and analysis. For example, the schools, parents and students would have to give informed and voluntary consent and anonymity and confidentiality with regard to names of teachers and students, including the names of the schools would have to be respected (Tolich & Davidson, 1999).

8. Describe the quantitative data that you will collect and how you will analyse it. What methods will you use? Discuss some of the key variables that you will collect data on. How many participants and how will you chose them (sampling) (approximately 300 words).

I intend collecting quantitative data from a survey which would be sent to the same students who had participated in the first stage of this research project, that is to say, those senior age secondary school students who had participated in the focus groups. The survey would consist of a list of variables, that is, the results of the qualitative conversation analysis and the key characteristics of highly effective catholic school teachers. To cite an example, one such variable might be the proposition that highly effective catholic secondary school teachers prepare well for their lessons. Others might be that highly effective catholic secondary school teachers have positive attitudes or that they are masters in their subject areas or that they include a spiritual dimension in their teaching. I consider it advisable to add one other variable and that is school location so the researcher can identify how many surveys from each school were returned. Other nominal demographic data such as gender, age etc. were considered unnecessary for this particular research project because the focus of the project is on a list of top ten characteristics of effective catholic secondary school teachers and not on gender or age differences.

In regard to the administration of the survey itself, students would be invited to read the list of variables and choose a top ten. Students would be asked to rank the characteristic they felt most defined an effective catholic secondary school teacher as number one and so on and so forth until they reach number ten. Results would be entered into a Microsoft Excel spreadsheet and the top ten key characteristics would be determined by working out which characteristics had the 10 lowest total scores.

9. Drawing on the work of Creswell & Plano Clark (2011), draw a diagram of your mixed methods study.  

PART TWO: CRITICAL REVIEW

1. Using Onwuegbuzie and Poth’s meta themes, conduct a critical review of the following mixed methods article (500-750 words):

Wyant, J.D., Jones, E.M. & Bulgar, S.M. (2015). A mixed methods analysis of a single-course strategy to integrate technology into PETE. Journal of Teaching in Physical Education, 34, 131-151.

A critical review has been conducted on Wyant, Jones, & Bulger’s (2015) above mentioned article using Onwuegbuzie and Poth’s (2016) meta-themes. Unless otherwise noted, all references are citations from this article.

META-THEME #1 – WARRANTEDNESS

Many terms were introduced and defined however I assess that there were too many. In the introduction alone readers were introduced to the terms PETE, NASPE, ISTE, CBAM, Stages of Adoption, External Barriers, First-Order Barriers and TPACK. Whole articles have been devoted to explaining TPACK! Additionally, what are “occupational socialization researchers”?

The reference list was comprehensive however there were quite a few errors. For example “throug technology” instead of “through technology”; the use of capitals in some of the titles “Teachers’ Use of Educational Technology in U.S. Public Schools: 2009 (NCES 2010-040)“, US States missing “Eugene:” instead of “Eugene, OR”, the words “Author Retrieved” instead of just “Retrieved” and many recent journal articles were missing DOI numbers. In regard to the latter, was this the fault of the authors or were they simply unable to be found?

Citations were inconsistent within the article, for example, three authors were consolidated to (Hall et al., 1979) and (Rochanasmita et al., 2009) while others with four were written out in full (Allan, Erickson, Brookhouse & Johnson, 2010) and (Russell, Bebell, O’Dwyer & O’Connor, 2003).

META-THEME #2 – JUSTIFICATION

The article appears to be underdeveloped. Although the authors used the TPACK model as a theoretical framework to explain the first theme, nowhere in the article do the authors explain this. They refer readers to the TPACK model in the introduction but that is all. The same could be said for First-Order Barriers which is used as a framework for the second theme.

The authors advance qualitative/quantitative research/mixed methods research. They used a “mixed methods design” and “quantitative and qualitative data collection procedures” consisting of “closed-ended survey instruments” and “weekly journal entries and end of course semi-structured interviews“.

The purpose of the study was cleared stated in the purpose statement paragraph – “The purpose of this study was to examine the influence of a domain-specific technology course…”

META-THEME #3 – WRITING QUALITY

Although the authors made good use of headings and sub-headings some sections seemed out of order. For example “Research Design” followed “Methods” when it really should be the other way around, participants were mentioned twice and the introduction seemed too long. Furthermore the authors did not follow the structure recommended by Onwuegbuzie and Poth (2016) who advocate the following framework – literature review, theoretical framework, rationale, purpose statement, research questions, hypothesis, and educational significance.

The authors used a number of appropriate transitional words such as “Accordingly“, “More specifically“, “For example“, “Typically“, “Further” etc. However, the abstract repeats the purpose statement.

META-THEME #4 – TRANSPARENCY

Information relating to sample size was mentioned specifically and clearly. Readers were directed to the fact that “participants included 12 pre-service teachers” and “were recruited from a sample of 34 (male=24, female=10)“. However, information regarding the “two participant sampling techniques” could have been made clearer. For example, the authors could have clearly stated that the first participant sampling technique used was……, the second participant sampling technique was….etc.

All tables and figures were referred to, for example, “see Figure 1“, “see Figure 2“, “see Table 1” etc. However, in my opinion they were not explained sufficiently. To cite just one example on this point, Figure 1 was a visual model of the mixed methods design, however, the authors only referred readers to this visual diagram under Data Collection Procedures and Analysis and not under Research Design where it seems more appropriate to put it. Furthermore, readers were referred to the model as a timeline – “a timeline was created ….. (see Figure 1)”, yet the caption clearly indicated that it was a “Visual Model of Mixed Methods Research Design“.

Directions for future research were provided in the final concluding paragraph – “What this study further highlights is the need for scholars to devote greater attention to the research and dissemination of technology-related projects”.

META-THEME #5 – INTEGRATION

Figure 1, which consisted of a visual model of the mixed methods research design, was clear and visually appealing. However as said above, it should have been referred to and explained in detail under the heading “Research Design“.

Measures were taken to ensure validity/legitimation. For example, “the use of experienced social science researchers (plural) to review and legitimize research procedures“, and “interview transcriptions and journal entries were also distributed to participants to confirm the data accurately captured their feelings“.

META-THEME #6 – PHILOSOPHICAL LENS

The authors refer to research philosophies such as “occupational socialization” and “constructivist-based learning“, however, both could have been presented in a better way. For example, in regard to the former, the authors refer the readers to “occupational socialization researchers“, yet they do not clearly define this it but simply refer readers to it at the very end of the paragraph “As with occupational socialization, research on learning…”. Shouldn’t philosophies be clearly defined first so readers understand?

REFERENCES

Canter, L. (2014). Classroom management for academics success. Bloomington, IN: Solution Tree Press.

Carey, M. A. (2016). Focus groups-what is the same, what is new, what is next? Qualitative Health Research, 26(6), 731-733. doi.org/10.1177/1049732316636848

Coles, E. K., Owens, J. S., Serrano, V. J., Slavec, J., & Evans, S. W. (2015). From consultation to student outcomes: The role of teacher knowledge, skills, and beliefs in increasing integrity in classroom management strategies. School Mental Health, 7(1), 34-48. doi.org/10.1007/s12310-015-9143-2

Cooper, J. T., Hirn, R. G., & Scott, T. M. (2015). Teacher as change agent: Considering instructional practice to prevent student failure. Preventing School Failure: Alternative Education for Children and Youth, 59(1), 1-4. doi.org/10.1080/1045988X.2014.919135

Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed). Los Angeles, CA: SAGE Publications.

Darling-Hammond, L., & Youngs, P. (2002). Defining ‘highly qualified teachers’: What does ‘scientifically-based research actually tell us? Educational Researcher, 31(9), 13-25. doi.org/10.3102/0013189X031009013

Duncan, A., Gurria, A., & Leeuwen, F. (2011). Uncommon wisdom on teaching. Retrieved from www.huffingpost.com/arne-duncan/uncommon-wisdom-on-teachi_b_836541.html

Goe, L., & Stickler, L. M. (2008). Teacher quality and student achievement: Making the most of recent research. National Comprehensive Center for Teacher Quality. Retrieved from http://files.eric.ed.gov/fulltext/ED520769.pdf

Hattie, J. (2003). Teachers make a difference: What is the research evidence? Presented at the Paper presented at the Australian Council for Educational Research Annual Conference on Building Teacher Quality, Melbourne, Australia. Retrieved from http://research.acer.edu.au/cgi/viewcontent.cgi?article=1003&context=research_conference_2003

Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. London, England: Routledge.

Jimerson, S. R., & Haddock, A. D. (2015). Understanding the importance of teachers in facilitating student success: Contemporary science, practice, and policy. School Psychology Quarterly, 30(4), 488-493. doi.org/10.1037/spq0000134

Leigh, A. (2010). Estimating teacher effectiveness from two-year changes in students’ test scores. Economics of Education Review, 29(3), 480-488. doi.org/10.1016/j.econedurev.2009.10.010

Onwuegbuzie, A. J., & Collins, K. M. . (2007). A typology of mixed methods sampling designs in social science research. The Qualitative Report, 12(2), 281-316.

Pedersen, B., Delmar, C., Falkmer, U., & Grønkjaer, M. (2016). Bridging the gap between interviewer and interviewee: Developing an interview guide for individual interviews by means of a focus group. Scandinavian Journal of Caring Sciences, 30(3), 631-638. doi.org/10.1111/scs.12280

Reddy, L. A., Fabiano, G. A., & Jimerson, S. R. (2013). Assessment of general education teachers’ Tier 1 classroom practices: Contemporary science, practice, and policy. School Psychology Quarterly, 28(4), 273-276. doi.org/10.1037/spq0000047

Schulz, J. (2012). Analysing your Interviews [Southampton Education School] [Video File]. Retrieved from https://www.youtube.com/watch?v=59GsjhPolPs

Society of Saint Pius X. (2016). General statistics about the SSPX. Retrieved from http://sspx.org/en/general-statistics-about-sspx

Spilt, J. L., Hughes, J. N., Wu, J.-Y., & Kwok, O.-M. (2012). Dynamics of teacher-student relationships: Stability and change across elementary school and the influence on children’s academic success: Teacher-student relationship trajectories. Child Development, 83(4), 1180-1195. doi.org/10.1111/j.1467-8624.2012.01761.x

Sutcliffe, C., P. (2011). Secondary students’ perceptions of teacher quality. Electronic Theses & Dissertations. 391. Retrieved from www.digitalcommons.georgiasouthern.edu/etd/391

Tolich, M., & Davidson, C. (1999). Starting fieldwork: An introduction to qualitative research in New Zealand. Auckland, N.Z: New York, NY: Oxford University Press.

Wikipedia. (2017a). Conversation analysis. In Wikipedia, the free encyclopaedia. Retrieved from https://en.wikipedia.org/wiki/Conversation_analysis

Wikipedia. (2017b). Traditionalist Catholic. In Wikipedia, the free encyclopaedia. Retrieved from https://en.wikipedia.org/wiki/Traditionalist_Catholic

Wyant, J. D., Jones, E. M., & Bulger, S. M. (2015). A mixed methods analysis of a single-course strategy to integrate technology into PETE. Journal of Teaching in Physical Education, 34(1), 131-151. doi.org/10.1123/jtpe.2013-0114