Scribd Logo
Upload

Welcome to Scribd!

  • 63 views

    Sample Program Using Python 3

    Uploaded by

    Joven Ramos
    This document contains 3 code samples demonstrating natural language processing techniques using the NLTK library in Python. The first sample performs morphological analysis using stemming to extract word roots. The second sample performs lexical analysis using tokenization and lemmatization. The third sample generates parse trees using a context-free grammar to represent syntactic structure.

    Copyright:

    © All Rights Reserved
    Download as DOCX, PDF, TXT or read online from Scribd

    Sample Program Using Python 3

    Uploaded by

    Joven Ramos
    63 views5 pages
    This document contains 3 code samples demonstrating natural language processing techniques using the NLTK library in Python. The first sample performs morphological analysis using stemming to extract word roots. The second sample performs lexical analysis using tokenization and lemmatization. The third sample generates parse trees using a context-free grammar to represent syntactic structure.

    Copyright

    © © All Rights Reserved

    Available Formats

    DOCX, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document contains 3 code samples demonstrating natural language processing techniques using the NLTK library in Python. The first sample performs morphological analysis using stemming to extract word roots. The second sample performs lexical analysis using tokenization and lemmatization. The third sample generates parse trees using a context-free grammar to represent syntactic structure.

    Copyright:

    © All Rights Reserved
    Download as DOCX, PDF, TXT or read online from Scribd
    Download as docx, pdf, or txt
    63 views5 pages

    Sample Program Using Python 3

    Uploaded by

    Joven Ramos
    This document contains 3 code samples demonstrating natural language processing techniques using the NLTK library in Python. The first sample performs morphological analysis using stemming to extract word roots. The second sample performs lexical analysis using tokenization and lemmatization. The third sample generates parse trees using a context-free grammar to represent syntactic structure.

    Copyright:

    © All Rights Reserved
    Download as DOCX, PDF, TXT or read online from Scribd
    Download as docx, pdf, or txt
    You are on page 1of 5

    Sample Program using Python 3.

    Sample 1: Morphological Analysis using NLTK Library


    You can find the code on this link:
    https://github.com/jalajthanaki/NLPython/blob/master/ch3/3_1_wordsteam.py

    # This script gives you an idea how stemming has been placed by using NLTK library.
    # It is part of morphological analysis

    from nltk.stem import PorterStemmer

    word = "unexpected"
    text = "disagreement"
    text1 = "disagree"
    text2 = "agreement"
    text3 = "quirkiness"
    text4 = "historical"
    text5 = "canonical"
    text6 = "happiness"
    text7 = "unkind"
    text8 = "dogs"
    text9 = "expected"

    def stemmer_porter():
    port = PorterStemmer()
    print("\nDerivational Morphemes")
    print (" ".join([port.stem(i) for i in text6.split()]))
    print (" ".join([port.stem(i) for i in text7.split()]))
    print ("\nInflectional Morphemes")
    print (" ".join([port.stem(i) for i in text8.split()]))
    print (" ".join([port.stem(i) for i in text9.split()]))
    print ("\nSome examples")
    print (" ".join([port.stem(i) for i in word.split()]))
    print (" ".join([port.stem(i) for i in text.split()]))
    print (" ".join([port.stem(i) for i in text1.split()]))
    print (" ".join([port.stem(i) for i in text2.split()]))
    print (" ".join([port.stem(i) for i in text3.split()]))
    print (" ".join([port.stem(i) for i in text4.split()]))
    print (" ".join([port.stem(i) for i in text5.split()]))

    if __name__ == "__main__":
    stemmer_porter()

    Sample 2: Lexical Analysis using Tokenization and Lemmatization of NLTK


    Library
    You can find the code on this link:
    https://github.com/jalajthanaki/NLPython/tree/master/ch3

    # This script gives you an idea how tokenization and lemmatization has been placed by using NLTK.
    # It is part of lexical analysis
    from nltk.tokenize import word_tokenize
    from nltk.stem.wordnet import WordNetLemmatizer

    def wordtokenization():
    content = """Stemming is funnier than a bummer says the sushi loving computer scientist.
    She really wants to buy cars. She told me angrily. It is better for you.
    Man is walking. We are meeting tomorrow. You really don't know..!"""
    print (word_tokenize(content))
    def wordlemmatization():
    wordlemma = WordNetLemmatizer()
    print (wordlemma.lemmatize('cars'))
    print (wordlemma.lemmatize('walking',pos='v'))
    print (wordlemma.lemmatize('meeting',pos='n'))
    print (wordlemma.lemmatize('meeting',pos='v'))
    print (wordlemma.lemmatize('better',pos='a'))
    print (wordlemma.lemmatize('is',pos='v'))
    print (wordlemma.lemmatize('funnier',pos='a'))
    print (wordlemma.lemmatize('expected',pos='v'))
    print (wordlemma.lemmatize('fantasized',pos='v'))

    if __name__ =="__main__":
    wordtokenization()
    print ("\n")
    print ("----------Word Lemmatization----------")
    wordlemmatization()

    Sample 3: Syntactical Analysis using Parse Tree Algorithm of NLTK Library


    You can find the code on this link:
    https://github.com/jalajthanaki/NLPython/blob/master/ch3/3_3_parsingtree.py

    # This script is for generating parsing tree by using NLTK.


    # NLTK gives us tree representation of stanford parser.
    import nltk
    from nltk import CFG
    from nltk.tree import *
    from collections import defaultdict
    # Part 1: Define a grammar and generate parse result using NLTK
    def definegrammar_pasrereult():
    Grammar = nltk.CFG.fromstring("""
    S -> NP VP
    PP -> P NP
    NP -> Det N | Det N PP | 'I'
    VP -> V NP | VP PP
    Det -> 'an' | 'my'
    N -> 'elephant' | 'pajamas'
    V -> 'shot'
    P -> 'in'
    """)
    sent = "I shot an elephant".split()
    parser = nltk.ChartParser(Grammar)
    trees = parser.parse(sent)
    for tree in trees:
    print(tree)

    # Part 2: Draw the parse tree


    def draw_parser_tree():
    dp1 = Tree('dp', [Tree('d', ['the']), Tree('np', ['dog'])])
    dp2 = Tree('dp', [Tree('d', ['the']), Tree('np', ['cat'])])
    vp = Tree('vp', [Tree('v', ['chased']), dp2])
    tree = Tree('s', [dp1, vp])
    print(tree)
    print(tree.pformat_latex_qtree())
    tree.pretty_print()

    if __name__ == "__main__":
    print("\n--------Parsing result as per defined grammar-------")
    definegrammar_pasrereult()
    print("\n--------Drawing Parse Tree-------")
    draw_parser_tree()

    You might also like