Abstract
The automatic generation of multiple-choice item (MI) has attracted amounts of attention. However, only a limited number of existing research address automatic MI generation for prepositions, and even fewer consider learners’ need in the generation process. In this paper, we propose an approach to generate preposition MIs suitable for non-native English learners of different language proficiency. First we select sentences with similar difficulty level to that of a given textbook as stems by using the sentence difficulty model we constructed. Then, we use the Word2vec model to retrieve a preposition list of distractor candidates where three of them are chosen as distractors. To validate the effectiveness of our approach, we produce four tests of preposition MIs at different difficulty levels and conduct a series of experiments regarding evaluations of stem difficulty, distractor plausibility and reliability. The experimental results show that our approach can generate preposition MIs targeting learners at different levels. The results of distractor plausibility and reliability also point to the validity of our approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chodorow, M., Tetreault, J.R., Han, N.R.: Detection of grammatical errors involving prepositions. In: Proceedings of the Fourth ACL-SIGSEM Workshop on Prepositions, pp. 25–30. Association for Computational Linguistics (2007)
Izumi, E., Uchimoto, K., Saiga, T., Supnithi, T., Isahara, H.: Automatic error detection in the Japanese learners’ English spoken data. In: Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, vol. 2, pp. 145–148. Association for Computational Linguistics (2003)
Dahlmeier, D., Ng, H.T., Schultz, T.: Joint learning of preposition senses and semantic roles of prepositional phrases. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, vol. 1, pp. 450–458. Association for Computational Linguistics (2009)
Lee, J., Yeung, C.Y., Zeldes, A., Reznicek, M., Lüdeling, A., Webster, J.: Cityu corpus of essay drafts of English language learners: a corpus of textual revision in second language writing. Lang. Resour. Eval. 49(3), 659–683 (2015)
Alderson, J.C.: Do corpora have a role in language assessment? In: Using Corpora for Language Research, pp. 248–259 (1996)
Heaton, J.B.: Writing English Language Tests: A Practical Guide for Teachers of English as a Second or Foreign Language. Longman Publishing Group, Harlow (1975)
Hoshino, A., Nakagawa, H.: A real-time multiple-choice question generation for language testing: a preliminary study. In: Proceedings of the Second Workshop on Building Educational Applications Using NLP, pp. 17–20. Association for Computational Linguistics (2005)
Smith, S., Avinesh, P.V.S., Kilgarriff, A.: Gap-fill tests for language learners: corpus-driven item generation. In: Proceedings of ICON-2010: 8th International Conference on Natural Language Processing, pp. 1–6. Macmillan Publishers (2010)
Sumita, E., Sugaya, F., Yamamoto, S.: Measuring non-native speakers’ proficiency of English by using a test with automatically-generated fill-in-the-blank questions. In: Proceedings of the Second Workshop on Building Educational Applications Using NLP, pp. 61–68. Association for Computational Linguistics (2005)
Lee, J., Sturgeon, D., Luo, M.: A CALL system for learning preposition usage. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pp. 984–993 (2016)
Pino, J., Eskenazi, M.: Semi-automatic generation of cloze question distractors effect of students’ L1. In: International Workshop on Speech and Language Technology in Education (2009)
Brown, J.C., Frishkoff, G.A., Eskenazi, M.: Automatic question generation for vocabulary assessment. In: Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, pp. 819–826. Association for Computational Linguistics (2005)
Lin, Y.C., Sung, L.C., Chen, M.C.: An automatic multiple-choice question generation scheme for English adjective understanding. In: Workshop on Modeling, Management and Generation of Problems/Questions in eLearning, the 15th International Conference on Computers in Education (ICCE 2007), pp. 137–142 (2007)
Mitkov, R., Ha, L.A.: Computer-aided generation of multiple-choice tests. In: Proceedings of the HLT-NAACL 03 Workshop on Building Educational Applications Using Natural Language Processing, vol. 2, pp. 17–22. Association for Computational Linguistics (2003)
Coniam, D.: A preliminary inquiry into using corpus word frequency data in the automatic generation of English language cloze tests. Calico J. 14(2–4), 15–33 (1997)
Agarwal, M., Mannem, P.: Automatic gap-fill question generation from text books. In: The Workshop on Innovative Use of NLP for Building Educational Applications, pp. 56–64. Association for Computational Linguistics (2012)
Lee, J., Seneff, S.: Automatic generation of cloze items for prepositions. In: Conference of the International Speech Communication Association, INTERSPEECH 2007, Antwerp, Belgium, pp. 2173–2176 (2007)
Sakaguchi, K., Arase, Y., Komachi, M.: Discriminative approach to fill-in-the-blank quiz generation for language learners. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pp. 238–242 (2013)
Chall, J.S., Dale, E.: Readability Revisited: The New Dale-Chall Readability Formula. Brookline Books, Northampton (1995)
Lennon, C., Burdick, H.: The lexile framework as an approach for reading measurement and success (2004). Electronic Publication on http://www.lexile.com
Pitler, E., Nenkova, A.: Revisiting readability: a unified framework for predicting text quality. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 186–195. Association for Computational Linguistics (2008)
Tanaka-Ishii, K., Tezuka, S., Terada, H.: Sorting by readability. Comput. Linguist. 36(2), 203–227 (2010)
Heilman, M., Collins-Thompson, K., Eskenazi, M.: An analysis of statistical models and features for reading difficulty prediction. In: Proceedings of the Third Workshop on Innovative Use of NLP for Building Educational Applications, pp. 71–79. Association for Computational Linguistics (2008)
François, T., Miltsakaki, E.: Do NLP and machine learning improve traditional readability formulas? In: The Workshop on Predicting and Improving Text Readability for Target Reader Populations, pp. 49–57. Association for Computational Linguistics (2012)
Perera, K.: The assessment of linguistic difficulty in reading material. Educ. Rev. 32(2), 151–161 (1980)
Troia, G.A. (ed.): Instruction and Assessment for Struggling Writers: Evidence-Based Practices. Guilford Press, New York (2011)
Lin, D.: On the structural complexity of natural language sentences. In: Proceedings of the 16th Conference on Computational Linguistics, vol. 2, pp. 729–733. Association for Computational Linguistics (1996)
Liu, H.: Dependency distance as a metric of language comprehension difficulty. J. Cogn. Sci. 9(2), 159–191 (2008)
Lu, Q., Xu, C., Liu, H.: Can chunking reduce syntactic complexity of natural lan-guages? Complexity 21(S2), 33–41 (2016)
Mishra, A., Bhattacharyya, P., Carl, M.: Automatically predicting sentence translation difficulty. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, vol. 2, pp. 346–351 (2013)
Specia, L., Shah, K., Souza, J.G., Cohn, T.: QuEst-A translation quality estimation framework. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 79–84 (2013)
Koehn, P., et al.: Open source toolkit for statistical machine translation: factored translation models and confusion network decoding. In: Final Report of the 2006 JHU Summer Workshop (2006)
Landis, J.R., Koch, G.G.: The measurement of observer agreement for categorical data. Biometrics 33(1), 159–174 (1977)
Acknowledgments
This paper is supported by the National Science Foundation of China (No.61462045).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix: Sample Items of Preposition Test Paper for Each Textbook
Appendix: Sample Items of Preposition Test Paper for Each Textbook
Rights and permissions
Copyright information
© 2018 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Xiao, W., Wang, M., Zhang, C., Tan, Y., Chen, Z. (2018). Automatic Generation of Multiple-Choice Items for Prepositions Based on Word2vec. In: Zhou, Q., Miao, Q., Wang, H., Xie, W., Wang, Y., Lu, Z. (eds) Data Science. ICPCSEE 2018. Communications in Computer and Information Science, vol 902. Springer, Singapore. https://doi.org/10.1007/978-981-13-2206-8_8
Download citation
DOI: https://doi.org/10.1007/978-981-13-2206-8_8
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-2205-1
Online ISBN: 978-981-13-2206-8
eBook Packages: Computer ScienceComputer Science (R0)