Natural Language Processing

Overview

Language allows you to analyze text to collect data from content such as ideas, entities, key phrases, subgroups, feelings, emotions, relationships, textual norms, using the understanding of the natural language, and to identify specific entities and relationships in unstructured text.

Beginner's Guide


Computers are great at working with structured data like spreadsheets and database tables. But we humans usually communicate in words, not in tables. That’s unfortunate for computers. A lot of information in the world is unstructured — raw text in English or another human language. How can we get a computer to understand the unstructured text and extract data from it?

NLP is short for natural language processing while NLU is the shorthand for natural language understanding. Similarly named, the concepts both deal with the relationship between natural language (as in, what we as humans speak, not what computers understand) and artificial intelligence.

They share a common goal of making sense of concepts represented in unstructured data, like language, as opposed to structured data like statistics, actions, etc. To that end, NLP and NLU are opposites of a lot of other data mining techniques. But that’s where the comparisons stop: NLU and NLP aren’t the same things.

Quickstart

The following 5 services/APIs are provided by Language as part of Release 1.0.0.
  • POS Tagger : Part Of Speech tagging for a piece of text.

  • Sentiment Analysis (English) : Analyses the sentiment of the text and returns if it falls under the positive, neutral, or negative sentiment.

  • NER Tagger : Named Entity Recognition Tagging for a piece of text.

  • Dependency Parser : Dependency Parser represents the Grammatical Structure of the Sentence.

  • Tags : Tags Extractor for extracting keywords from a paragraph.

APIs

All URIs below are relative to https://prod-kong.dltk.ai

POS TaggerPOST/core/nlp/pos/
Sentiment AnalysisPOST/core/nlp/sentiment/
NER TaggerPOST/core/nlp/ner/
Dependency ParserPOST/core/nlp/dependency-parser/
TagsPOST/core/nlp/tags/

POS Tagger

Description

This API would enable you to do a part of speech tagging of the text.
URI
POST  /core/nlp/pos
Headers
api-key Your App’s API Key
Attributes
textText Sentence (String)
Request Example:
{
  "text": "He did a terrific job at the construction site"
}
Response:
{
  "text": "He did a terrific job at the construction site.",
  "result": {
    "the": "DT",
    "a": "DT",
    "site": "NN",
    "at": "IN",
    "terrific": "JJ",
    "construction": "NN",
    "job": "NN",
    "He": "PRP",
    "did": "VBD"
  }
}

Sentiment Analysis

Description
This API would enable you to do a sentiment analysis and classify if the text is positive, neutral or negative.
URI
POST /core/nlp/sentiment
Headers
api-key Your App’s API Key
Attributes
textText Sentence (String)
Request Example:
{
  "text": "He has been always cheerful ever since he moved to Hyderabad."
}
Response:
{
    "scores": {
        "neu": 0.741,
        "pos": 0.259,
        "compound": 0.5423,
        "neg": 0
    },
    "text": "He has been always cheerful ever since he moved to Hyderabady",
    "emotion": "POSITIVE",
    "polarity": 3
}

Named Entity Recognition(NER) Tagger

Description
This API would enable you to do a named entity recognition tagging of the text. This API also finds nearest the match for the person detected and gives information such as occupation, nationality, etc.
URI
POST /core/nlp/ner
Headers
api-key Your App’s API Key
Attributes
textText Sentence (String)
Request Example:
{
  "text": "Madan Puri bought 300 shares of Acme Corp. in 2006."
}
Response:
{
    "result": {
        "300": "CARDINAL",
        "2006": "DATE",
        "Madan Puri": "PERSON",
        "Acme Corp.": "ORG"
    },
    "persons": [
        {
            "Link": "https://en.wikipedia.org/wiki/Madan_Puri",
            "Name": "Madan Puri",
            "Nationality": "Indian",
            "Occupation": "Actor"
        }
    ],
    "text": "Madan Puri bought 300 shares of Acme Corp. in 2006."
}

Dependency Parser

Description
This API would enable you to do dependency-parser.

URI
POST /core/nlp/dependency-parser/

Headers
api-key Your App’s API Key
Attributes
textText Sentence (String)
Request Example:
{
  "text": "He is a cheerful guy."
}
Response:
{
    "cheerful": {
        "children": [],
        "headPOS": "NOUN",
        "dep": "amod",
        "headText": "leader"
    },
    "is": {
        "children": [
            "He",
            "leader",
            "."
        ],
        "headPOS": "VERB",
        "dep": "ROOT",
        "headText": "is"
    },
    "He": {
        "children": [],
        "headPOS": "VERB",
        "dep": "nsubj",
        "headText": "is"
    },
    ".": {
        "children": [],
        "headPOS": "VERB",
        "dep": "punct",
        "headText": "is"
    },
    "a": {
        "children": [],
        "headPOS": "NOUN",
        "dep": "det",
        "headText": "leader"
    },
    "leader": {
        "children": [
            "a",
            "cheerful"
        ],
        "headPOS": "VERB",
        "dep": "attr",
        "headText": "is"
    }
}

Tags Extractor

Description
This API would enable you to get tags from a paragraph.
URI
POST /core/nlp/tags/
Headers
api-key Your App’s API Key
Attributes
textText Sentence (String)
Request Example:
{
"text": "TweetDeck is a social media dashboard application for management of Twitter accounts. Originally an independent app, TweetDeck was subsequently acquired by Twitter Inc. and integrated into
                Twitter's interface."
}
Response:
{
    "tags": [
        "tweetdeck",
        "twitter"
    ],
    "text": "TweetDeck is a social media dashboard application for management of Twitter accounts. Originally an independent app, TweetDeck was subsequently acquired by Twitter Inc. and integrated
                into Twitter's interface."
}

Annotations

NER Supports below entities
Type Description
PERSON People, including fictional.
NORP Nationalities or religious or political groups.
LANGUAGE Any named language.
QUANTITY Measurements, as of weight or distance.
ORDINAL “First”, “second”, etc.
CARDINAL Numerals that do not fall under another type.
MONEY Monetary values, including unit.
PERCENT Percentage, including “%”.
TIME Times smaller than a day.
DATE Absolute or relative dates or periods.
EVENT Named hurricanes, battles, wars, sports events, etc.
WORK_OF_ART Titles of books, songs, etc.
LAW Named documents made into laws.
LOC Non-GPE locations, mountain ranges, bodies of water.
PRODUCT Objects, vehicles, foods, etc. (not services).
FAC Buildings, airports, highways, bridges, etc.
ORG Companies, agencies, institutions, etc.
GPE Countries, cities, states, etc.

Annotations

POS TAGGER supports below entities:
POS Description
PUNCT Punctuations
CD Cardinal Number
SYM Symbol
DT Determiner
NN, NNS Noun
NNP, NNPS Proper Noun
PRP Pronoun
UH Interjection
JJ, JJR, JJS, WDT Adjective, Comparative and Superlative
ADP AdPosition like in, during
VB, VBG, VBN, VBP, VBZ Verb
RB, RBR, RBS Adverb
AUX Auxiliary
CC Conjunction
CCONJ Coordinating Conjunction
IN Subordinating Conjunction
 

SDK

Installation

DLTK requires Python version greater than 3.5. One can install DLTK SDK using following command:
pip install dltk_ai

Creating Client

Create DLTK client to perform a different task.
client = dltk_ai.DltkAiClient('Your API Key')
To use these services, one needs to register to dltk website and create a project. Copy your API key to use different APIs.

Sentiment Analysis

Use to get the sentiment of the given text.
sentiment_analysis_response = client.sentiment_analysis("The stay was great.")
sentiment_analysis_response
To use these services, one needs to register to dltk website and create a project. Copy your API key to use different APIs.

POS Tagger

Use to get the Part Of Speech of the given text.
pos_tagging_response = client.pos_tagger("The stay was great.")
pos_tagging_response

Dependency Parser

Use to get the dependency parser of the given text.
dependency_parser_response = client.dependency_parser('He is a cheerful guy.')
dependency_parser_response

NER Tagger

Use to get the ner tagger of the given text.
ner_tagger_response = client.ner_tagger('Madan Puri bought 300 shares of Acme Corp. in 2006.')
ner_tagger_response

Tag Extraction

Use to extract tags from given paragraph.
tagger_response = client.tags('TweetDeck is a social media dashboard application for management of
                                Twitter accounts. Originally an independent app, TweetDeck was
                                 subsequently acquired by Twitter Inc. and integrated intoTwitter's
                                 interface..')
tagger_response

Sarcasm Analysis

Check if the given sentence is sarcastic in nature.
sarcasm_analysis_response = client.sarcasm_analysis("I am not insulting you, I am describing you")
sarcasm_analysis_response

Abuse Analysis

Check if the given sentence is abusive in nature.
abuse_analysis_response = client.abuse_analysis("You need to shut the fuck up")
abuse_analysis_response

Release Notes

Following are the release notes as part of Release 1.0.0

  • POS Tagger tags supported are DT, NN, IN, JJ, PRP, VBD etc.

  • NER Tagger tags supported are PER, ORG, LOC, DATE, GPE, PRODUCT, LANGUAGE, TIME, MONEY, QUANTITY etc.

  • Sentiment Analysis is supported only in English.

  • Tags Extractor works on paragraphs and minimum frequency of any word to be detected as keyword must be at least two.

  • The Info of the person detected in NER is the nearest match.

Login

Register