Measuring the Frequency of Critical Thinking in a Second Language Academic Discussion Course

by Samuel Reid1 & Travis West2
1,2Foreign Language Education and Research Center, Rikkyo University, Japan.


Reid, S., West, T. (2021). Measuring the Frequency of Critical Thinking in a Second Language Academic Discussion Course. In D. Koyama (Ed.), Development of Innovative Pedagogical Practices for a Modern Learning Experience (pp 237-268). Jagadhri, HR: CSMFL Publications.


As technology and globalization increase the chances of exposure to information, learners’ Critical Thinking (CT) and researchers’ ability to measure it will play an important role in developing modern educational experiences. This is particularly the case for English language learners who wish to enter tertiary education in English-speaking countries (Liaw, 2007; Wagner, 2010). Emphasis on such skills is increasingly a facet of language education in Japanese contexts. This can be seen in changes implemented by the Japan Ministry of Education, Culture, Sports, Science, and Technology which have encouraged a focus on CT in English language courses during recent years (MEXT, 2011). However, it can be difficult for second language (L2) learners to exhibit CT in an L2 (Bali, 2015; Luk & Lin, 2015). Measuring CT in learner output has also proven difficult, which can be an obstacle to further integrating CT in L2 pedagogy. Studies exploring ways of measuring CT in an L2 have largely focused on written work (e.g., Davidson & Dunham, 1997; Floyd, 2011; Stapleton, 2001), while analysis of CT in spoken L2 discourse has seen little attention. As a result, little advice can be found on practical steps for teachers to help learners display CT when speaking in an L2.

This chapter describes a study of arguments made during group discussions in an L2 English Discussion course at a Japanese university. A corpus of spontaneous spoken discourse recorded during class was analyzed to measure the frequency of CT displayed in an academic setting where CT was not an explicit focus of the course. Arguments in the corpus were identified using Ramage et al.’s (2016) model of argument criteria, and a categorization system was developed in which discourse was classified as displaying either objective reasoning or subjective reasoning. Participants were found to have used approximately 72% objective and 28% subjective reasoning. However, further analysis revealed an important qualitative difference in arguments identified as incorporating objective reasoning. The results of the study suggest two areas that may help teachers promote an increase in student usage of CT: the importance of question prompts in orienting learners towards CT in their answers, and a specific focus on the role of pronoun usage in taking a subjective or objective stance.

Keywords: Higher-order thinking skills, pronoun usage, corpus linguistics, argumentation

This is a part of: Development of Innovative Pedagogical Practices for a Modern Learning Experience (Ed. Dennis Koyama, Ph.D.)

© CSMFL Publications & its authors.


  1. Adolphs, S., & Knight, D. (2010). Building a spoken corpus: What are the basics? In A. O’Keeffe & M. McCarthy (Eds.), The Routledge handbook of corpus linguistics (pp. 38–52). Routledge.
  2. Alagozlu, N. (2007). Critical thinking and voice in EFL writing. The Asian EFL Journal Quarterly, 9(3), 118–136.
  3. Anthony, L. (2018). AntConc (Version 3.5.7) [Computer Software]. Waseda University.
  4. Bailin, S. (1995). Is critical thinking biased? Clarifications and implications. Science & Educational Theory, 45(2), 191–197.
  5. Bali, M. (2015). Critical Thinking through a multicultural lens: Cultural challenges of teaching critical thinking.In M. Davies & R. Barnett (Eds.), The Palgrave Handbook of Critical Thinking in Higher Education (pp. 317–334). Macmillan.
  6. Barnawi, O. Z. (2011). Finding a place for critical thinking and self-voice in college English as a foreign language. Writing Classrooms, 4(2), 190–197.
  7. Benesch, S. (1999). Thinking critically, thinking dialogically. TESOL Quarterly, 33(3), 573–580.
  8. Buck, J. (2017). Developing the Rikkyo English discussion corpus. In G. Brooks (Ed.), The 2016 PanSIG Journal (pp. 25–30).JALT.
  9. Chirgwin, S. K., & Huijser, H. (2015). Cultural variance, critical thinking, and indigenous knowledges: Exploring a both-ways approach. In M. Davies & R. Barnett (Eds.), The Palgrave handbook of critical thinking in higher education. (pp. 335–350). Macmillan.
  10. Crammond, J. G. (1998). The uses and complexity of argument structures in expert and student persuasive writing. Written Communication, 15, 230–268.
  11. Crosthwaite, P., Boynton, A., & Cole, S. (2017). Exploring rater conceptions of academic stance and engagement during group tutorial discussion assessment. Journal of English for Academic Purposes, 28, 1–13.
  12. Davidson, B. W., & Dunham, R. A. (1997). Assessing EFL student progress in critical thinking with the Ennis-Weir Critical Thinking Essay Test. JALT Journal, 19(1), 43–57.
  13. Davies, W. M. (2006). An ‘infusion’ approach to critical thinking: Moore on the critical thinking debate. Higher Education Research & Development, 25(2), 179–193.
  14. Davies, M., & Barnett, R. (2015). The Palgrave handbook of critical thinking in higher education. Macmillan.
  15. Ennis, R. H. (2011). The nature of critical thinking: An outline of critical thinking dispositions and abilities.
  16. Facione, P. A. (1990). Critical thinking: A Statement of expert consensus for purposes of educational assessment and instruction.
  17. Fisher, A. (2011) Critical thinking – An introduction. (2nd ed.). Cambridge University Press.
  18. Floyd, C. B. (2011). Critical thinking in a second language. Higher Education Research & Development, 30(3), 289–302.
  19. Gunter, K. K. (2011). Braiding and rhetorical power players: Transforming academic writing through rhetorical dialectic. Journal of Basic Writing, 30(1), 64–98.
  20. Ikuenobe, P. (2001). Teaching and assessing critical thinking abilities as outcomes in an informal logic course. Teaching in Higher Education, 6(1), 19–32.
  21. Liaw, M. (2007). Content-based reading and writing for critical thinking skills in an EFL context. English Teaching & Learning, 31(2), 45–87.
  22. Luk, J., & Lin, A. (2015). Voices without words: Doing critical literate talk in English as a second language. TESOL Quarterly, 49(1), 67-91.
  23. McEnery, T., Xiao, R., & Tono, Y. (2006). Corpus-based language studies: An advanced resource book. Routledge.
  24. Mehta, S. R., & Al-Mahrooqi, R. (2015). Can thinking be taught? Linking critical thinking and writing in an EFL context. RELC Journal, 46(1), 23–36.
  25. MEXT. (2011). Koutougakkou gakushuushidouyouryou [The course of study for foreign languages for senior high school], provisional English version. Retrieved from
  26. Moon, J. (2008). Critical thinking: An exploration of theory and practice. Routledge.
  27. Moore, T. J. (2011). Critical thinking and language: The challenge of generic skills and disciplinary discourses. Continuum.
  28. Morikawa, S., Harrington, L., & Shiina, K. (2012). Critical thinking in a Japanese university current affairs course. Papers on Languages and Culture, (6), 117–113.
  29. Nunn, R., & Caroline Brandt, C. (2016) A phenomenological approach to teaching reflective writing. English Scholarship Beyond Borders, 2(1), 130–151.
  30. Opitz, T. A. (2016). The efficacy of drilling backchannels in a CLT context: A corpus analysis of Japanese ESL learners. New Directions in Teaching and Learning English Discussion, 4(1), 200–210.
  31. Opitz, T. A. (2017). Tracking lexical development in the EDC classroom: RED corpus study #2. New Directions in Teaching and Learning English Discussion, 5(1), 208–214.
  32. Paul, R., & Elder, L. (1996). Using intellectual standards to assess student reasoning. Retrieved September 6, 2006, from TRK12-using-intellectual-standards.shtml
  33. Paul, R., & Elder, L. (2008). Defining critical thinking.
  34. Possin, K. (2014). Critique of the Watson-Glaser Critical Thinking Appraisal Test: The more you know, the lower your score. Informal Logic, 34(4), 393–416.
  35. Ramage, J. D., Bean, J. C., & Johnson, J. (2016). Writing arguments: A rhetoric with readings (10thed.). Pearson.
  36. Rear, D. (2012). The dilemma of critical thinking: Conformism and non-conformism in Japanese education policy. In T. Isles & P. Matanle (Eds.), Researching twenty-first century Japan: New perspectives for the electronic age (pp. 119–137). Lexington Books.
  37. Shea, D. P. (2018). Trying to teach dialogically: The good, the bad, and the misguided. Language Teaching Research, 23(6), 1–18.
  38. Sinclair, J. (2005). Corpus and text-basic principles. In M. Wynne (Ed.), Developing linguistic corpora: A guide to good practice (pp. 1–16). Oxbow Books.
  39. Stapleton, P. (2001). Assessing critical thinking in the writing of Japanese university students: Insights about assumptions and content familiarity. Written Communication, 18(4), 506–548.
  40. Stroupe, R. R. (2006). Integrating critical thinking throughout ESL curricula. TESL Reporter, 39(2), 42–61.
  41. Toulmin, S. (2003). The uses of argument. Cambridge University Press.
  42. Tsuruta, Y. (2013). The knowledge society and the internationalization of Japanese higher education. Asia Pacific Journal of Education, 33(2), 140–155.
  43. Wagner, T. (2010). The global achievement gap: Why even our best schools don’t teach the new survival skills our children need – And what we can do about it. Basic Books.

[email protected]

Follow us @