KR 2019 homework part 1: overall task: erinevus redaktsioonide vahel

Allikas: Lambda
Mine navigeerimisribale Mine otsikasti
← Vanem muudatusUuem muudatus →
Resümee puudub
14. rida: 14. rida:


You are free to use NLP tools and API-s and datasets, but your program should drive them from the beginning to end (ie your program takes input files, calls tools, modifies and prints output). It is better to use the tools than not to use them!
You are free to use NLP tools and API-s and datasets, but your program should drive them from the beginning to end (ie your program takes input files, calls tools, modifies and prints output). It is better to use the tools than not to use them!
== Popular toolkits for NLP ==
The recommendation is to take the first (spacy) or the second (nltk) toolkit.
* [https://spacy.io Spacy toolkit for Python]
* [http://www.nltk.org/ NLTK: the main Python toolkit], see also [http://pixelmonkey.org/pub/nlp-training/ this tutorial]
* [https://research.googleblog.com/2016/05/announcing-syntaxnet-worlds-most.html Google SyntaxNet] (see [https://github.com/tensorflow/models/tree/master/syntaxnet in github])
* [http://nlp.stanford.edu/software/corenlp.shtml CoreNLP: the main Stanford NLP tool] in the context of a larger set of [http://nlp.stanford.edu/software/ Stanford NLP toolkits] like [http://nlp.stanford.edu/software/CRF-NER.shtml stanford NER etc]
[http://pythonprogramming.net/named-entity-recognition-nltk-python/ and this NER tutorial]
* [http://www.clips.ua.ac.be/pattern Pattern toolkit for Python]
* [http://opennlp.apache.org/ opennlp]
* [https://pypi.python.org/pypi/PyNLPl/ PyNLP for Python]
* [http://williamjturkel.net/2013/06/30/named-entity-recognition-with-command-line-tools-in-linux/ NER tutorial for Linux] in the context of [http://williamjturkel.net/category/linux-unix/ a larger practical tutorial]
=== Web APIs ===
* [https://cloud.google.com/natural-language/ Google cloud natural language API]
* [http://new.opencalais.com opencalais] (free registration required)

Redaktsioon: 6. veebruar 2019, kell 10:48

Task

The task of the first lab is to build and demonstrate a system which can

  • tokenize the text (input and question)
  • run a NER tool on the text giving wikipedia url-s as output. See webaida. Notice that:
    • some NER-s do not give wiki URL-s, but just types. This is also useful.
    • for the second overall scenario (like wikidata) we do need wiki urls, though.
  • run a parser tool on the text. There are several ways to parse, some more useful for our purposes than others. We will come back to that.

Technology and tools

You are free to use any programming language, but Python is recommended.

You are free to use NLP tools and API-s and datasets, but your program should drive them from the beginning to end (ie your program takes input files, calls tools, modifies and prints output). It is better to use the tools than not to use them!

Popular toolkits for NLP

The recommendation is to take the first (spacy) or the second (nltk) toolkit.

and this NER tutorial

Web APIs

Pärit leheküljelt "http://www.lambda.ee/w/index.php?title=KR_2019_homework_part_1:_overall_task&oldid=15405"