Papers from 2018 by Taraka Rama
Proceedings of NAACL 2018, 2018
We evaluate the performance of state-of-the-art algorithms for automatic cognate detection by com... more We evaluate the performance of state-of-the-art algorithms for automatic cognate detection by comparing how useful automatically inferred cognates are for the task of phyloge-netic inference compared to classical manually annotated cognate sets. Our findings suggest that phylogenies inferred from automated cog-nate sets come close to phylogenies inferred from expert-annotated ones, although on average , the latter are still superior. We conclude that future work on phylogenetic reconstruction can profit much from automatic cog-nate detection. Especially where scholars are merely interested in exploring the bigger picture of a language family's phylogeny, algorithms for automatic cognate detection are a useful complement for current research on language phylogenies.
Papers by Taraka Rama
Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial)
Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial)
This paper presents a computational analysis of Gondi dialects spoken in central India. We presen... more This paper presents a computational analysis of Gondi dialects spoken in central India. We present a digitized data set of the dialect area, and analyze the data using different techniques from dialectome-try, deep learning, and computational biology. We show that the methods largely agree with each other and with the earlier non-computational analyses of the language group.
Journal of Language Evolution
Age estimation of language families is an important task in historical linguistics. In our study ... more Age estimation of language families is an important task in historical linguistics. In our study we present an approach that utilizes information about the diversity across sound inventories of language families for the task of age estimation. Our approach involves three steps: (1) the construction of a phoneme network, which is a bipartite network structure that represents language families and its phoneme inventories in network-theoretic terms, (2) the reconstruction of such a real-world data network in form of a preferential attachment synthetic process, and (3) the detection of the optimal preferential attachment noise parameter, for which the synthetic network is the best approximation of the real-world data network. Our statistical analysis reveals that the optimal noise parameter appears to be a good predictor for the age of a language family.
Supertagging is an approach originally developed by to improve the parsing efficiency. In the beg... more Supertagging is an approach originally developed by to improve the parsing efficiency. In the beginning, the scholars used small training datasets and somewhat naïve smoothing techniques to learn the probability distributions of supertags. Since its inception, the applicability of Supertags has been explored for TAG (tree-adjoining grammar) formalism as well as other related yet, different formalisms such as CCG. This article will try to summarize the various chapters, relevant to statistical parsing, from the most recent edited book volume (Bangalore and Joshi, 2010). The chapters were selected so as to blend the learning of supertags, its integration into full-scale parsing, and in semantic parsing.
Crosslingual Information Retrieval (CLIR) usually requires query translation and, due to named en... more Crosslingual Information Retrieval (CLIR) usually requires query translation and, due to named entities in the case of IR, query translation requires a good transliteration system when writing systems differ. Transliteration can be seen as a problem of generation or alignment. For IR, since we can extract a word list from the corpus being searched, it should be seen as an alignment problem. The shift from generation to alignment can lead to higher transliteration accuracies and significant improvements in the CLIR results. We were able to achieve an increase (over generation) in the CLIR Mean Average Precision by 22.66% and 29.08% for English to Hindi and English to Marathi, respectively. RÉSUMÉ. La recherche d'information interlingue implique la traduction des requêtes. En raison du grand nombre d'entités nommées dans les requêtes, des systèmes de translittération efficaces doivent être mis en oeuvre quand les systèmes d'écriture diffèrent. Comme l'extraction de liste de mots cibles à partir des corpus interrogés est possible, nous préférons assimiler la translittération à un problème d'alignement plutôt qu'à un problème de génération. Ce choix conduit à de meilleures translittérations et à des améliorations importantes des réponses aux requêtes. Nous avons ainsi amélioré la précision moyenne de notre système de 22,66 % de l'anglais vers l'hindi et de 29,08 % de l'anglais vers le marathi.
In this paper, we describe the problem of cognate identification and its relation to phylogenetic... more In this paper, we describe the problem of cognate identification and its relation to phylogenetic inference. We introduce subsequence based features for discriminating cognates from noncognates. We show that subsequence based features perform better than the state-of-the-art string similarity measures for the purpose of cognate identification. We use the cognate judgments for the purpose of phylogenetic inference and observe that these classifiers infer a tree which is close to the gold standard tree. The contribution of this paper is the use of subsequence features for cognate identification and to employ the cognate judgments for phylogenetic inference.
Journal of Quantitative Linguistics, Dec 17, 2013
In this paper, we apply an information theoretic measure, self-entropy of phoneme n-gram distribu... more In this paper, we apply an information theoretic measure, self-entropy of phoneme n-gram distributions, for quantifying the amount of phonological variation in words for the same concepts across languages, thereby investigating the stability of concepts in a standardized concept list – based on the 100-item Swadesh list – specifically designed for automated language classification. Our findings are consistent with those of the ASJP project (Automated Similarity Judgment Program; Holman et al. 2008a). The correlation of our ranking with that of ASJP is statistically highly significant. Our ranking also largely agrees with two other reduced concept lists proposed in the literature. Our results suggest that n-gram analysis works at least as well as other measures for investigating the relation of phonological similarity to geographical spread, automatic language classification, and typological similarity, while being computationally considerably cheaper than the most widespread method (normalized Levenshtein distance), very important when processing large quantities of language data.
Proceedings of Human Language Technologies the 2009 Annual Conference of the North American Chapter of the Association For Computational Linguistics Companion Volume Student Research Workshop and Doctoral Consortium, 2009
Letter-to-phoneme conversion plays an important role in several applications. It can be a difficu... more Letter-to-phoneme conversion plays an important role in several applications. It can be a difficult task because the mapping from letters to phonemes can be many-to-many. We present a language independent letter-to-phoneme conversion approach which is based on the popular phrase based Statistical Machine Translation techniques. The results of our experiments clearly demonstrate that such techniques can be used effectively for letter-tophoneme conversion. Our results show an overall improvement of 5.8% over the baseline and are comparable to the state of the art. We also propose a measure to estimate the difficulty level of L2P task for a language.
The recent availability of typological databases such as World Atlas of Language Structures (WALS... more The recent availability of typological databases such as World Atlas of Language Structures (WALS) has spurred investigations regarding their utility for language classification, the stability of typological features in genetic linguistics and typological universals across the language families of the world. Existing work on building NLP resources such as parallel corpora, treebanks for under-resourced languages has a lot to gain by taking into consideration insights about inter-language relationships. Since Yarowsky et al. , there have been a number of attempts to create resources for resource-poor languages by projecting information from resource-rich languages using comparable corpora. An important intuition in such work is that syntactic information can be transferred with higher accuracy between languages if they are similar. In this paper, we compare typological distances derived from fifteen vector similarity measures with family internal classifications and also lexical divergence. These results are only a first step towards the use of WALS database in the projection of NLP resources for typologically or genetically similar, yet resource-poor languages.
Ever since the first Quantitative Investigations in Theoretical Linguistics conference in 2002 in... more Ever since the first Quantitative Investigations in Theoretical Linguistics conference in 2002 in Osnabrück, QITL conferences have taken up a special position in the landscape of linguistics conferences. Not just a special position, we believe, but a position that is to be cherished. Not only is it one of the rare forums where researchers from all subdisciplines of linguistics interested in quantitative linguistic methodology can meet and share insights, it also is a place where proponents of quantitative linguistics and empirical methodology from various theoretical linguistic backgrounds compare and discuss their findings. Most of all, QITL rightfully advocates an approach to quantitative research in which the relation between quantitative methods and theoretical insights is made very explicit.
In this paper we explore various parameter settings of the state-of-art Statistical Machine Trans... more In this paper we explore various parameter settings of the state-of-art Statistical Machine Translation system to improve the quality of the translation for a `distant' language pair like English-Hindi. We proposed new techniques for efficient reordering. A slight improvement over the baseline is reported using these techniques. We also show that a simple pre-processing step can improve the quality of the translation significantly.
In this article, we investigate the properties of phoneme N-grams across half of the world's lang... more In this article, we investigate the properties of phoneme N-grams across half of the world's languages. We investigate if the sizes of three different N-gram distributions of the world's language families obey a power law. Further, the N-gram distributions of language families parallel the sizes of the families, which seem to obey a power law distribution. The correlation between N-gram distributions and language family sizes improves with increasing values of N. We applied statistical tests, originally given by physicists, to test the hypothesis of power law fit to twelve different datasets. The study also raises some new questions about the use of N-gram distributions in linguistic research, which we answer by running a statistical test.
Since the 1950s, linguists have been using short lists (40–200 items) of basic vocabulary as the ... more Since the 1950s, linguists have been using short lists (40–200 items) of basic vocabulary as the central component in a methodology which is claimed to make it possible to automatically calculate genetic relationships among languages. In the last few years these methods have experienced something of a revival, in that more languages are involved, different distance measures are systematically compared and evaluated, and methods from computational biology are used for calculating language family trees. In this paper, we explore how this methodology can be extended in another direction, by using larger word lists automatically extracted from a parallel corpus using word alignment software. We present prelimi- nary results from using the Europarl parallel corpus in this way for estimating the distances between some languages in the Indo-European language family.
In this paper, we use corpus-based measures for constructing phylogenetic trees and try to addres... more In this paper, we use corpus-based measures for constructing phylogenetic trees and try to address some questions about the validity of doing this and applicability to linguistic areas as against language families. We experiment with four corpus based distance measures for constructing phylogenetic trees. Three of these measures were earlier tried for estimating language distances. We use a fourth measure based on phonetic and orthographic feature n-grams. We compare the trees obtained using these measures and present our observations.
Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2015
In this paper, we describe the problem of cognate identification in NLP. We introduce the idea of... more In this paper, we describe the problem of cognate identification in NLP. We introduce the idea of gap-weighted subsequences for discriminating cognates from non-cognates. We also propose a scheme to integrate phonetic features into the feature vectors for cognate identification. We show that subsequence based features perform better than state-of-the-art classifier for the purpose of cognate identification. The contribution of this paper is the use of subsequence features for cognate identification.
Sequences in Language and Text, 2015
Like many other research fields, linguistics is entering the age of big data. We are now at a poi... more Like many other research fields, linguistics is entering the age of big data. We are now at a point where it is possible to see how new research questions can be formulated -and old research questions addressed from a new angle or established results verified -on the basis of exhaustive collections of data, rather than small, carefully selected samples. For example, South Asia is often mentioned in the literature as a classic example of a linguistic area, but there is no systematic, empirical study substantiating this claim. Examination of genealogical and areal relationships among South Asian languages requires a large-scale quantitative and qualitative comparative study, encompassing more than one language family. Further, such a study cannot be conducted manually, but needs to draw on extensive digitized language resources and state-of-the-art computational tools. We present some preliminary results of our large-scale investigation of the genealogical and areal relationships among the languages of this region, based on the linguistic descriptions available in the 19 tomes of Grierson's monumental Linguistic Survey of India (1903India ( -1927, which is currently being digitized with the aim of turning the linguistic information in the LSI into a digital language resource suitable for a broad array of linguistic investigations.
Uploads
Papers from 2018 by Taraka Rama
Papers by Taraka Rama