KR 2021 homework part 1
The goal of the lab is to explore and prepare data for the question answering system to be built in the following labs. We will later modify, filter and format this data to make it suitable for simple commonsense reasoning / question answering.
Since the goal really is first exploring and then converting the data, there is no exact specification in the style of "take this and do that".
If you have very little or no experience using SQL, then it is recommended to
- Use sqlite, not Postgresql or any other database: sqlite is easiest to use.
- Run this small example from python: kuidas sqlite kasutada
- Read and try out live examples from the w3schools SQL tutorial: at least from the beginning up to the "delete" chapter and then the first few of the "join" chapters.
- Look for the specific details and options of using sqlite in python from the python 3 sqlite api docs
- Do not worry about sql injection or such security stuff: your code will not be accessible from the web and nobody can attack it using sql injection.
Yago for geodata
As a source for geographical data we use yago (here is their old page). Investigate parts of it and download from the download page (again, the old download page is here). The second important taxonomy source is wordnet: Yago contains some (?) of it, but you can use wordnet directly in your system.
Yago is large. You do not need to incorporate all of Yago in your database: the geographical facts for say, one country, and taxonomies are enough.
Your task is to
- select a sensible sub-part of geographical facts in Yago (like, one country) and store it in the SQL database, using both standard SQL and, where it seems useful, json. Postgresql and Sqlite are the best options. Postgresql due to special capabilities of handling json, sqlite due to simplicity of use.
- select a sensible sub-part of Yago taxonomies and store it in the same SQL database.
- investigate whether your yago taxonomy contains relevant parts of wordnet: if not, incorporate also wordnet into your database.
- perform some sample queries to verify that you can actually find information: try searching for both simple facts and also using taxonomies for searching for more abstract concepts.
Quasimodo for basic commonsense knowledge
As a source for basic commonsense knowledge we explore and possibly use quasimodo. Beware: it does not cover very much and contains a lot of weird statements, like "estonia, has_color, yellow".
In particular, please
- Download and explore the version 1.3 dataset.
- Find out if the dataset contains meaningful amount of information about the geographical places (or additional information about the facts about these places) for the sub-part you chose from Yago.
- If yes, please store the relevant meaningful information part in the SQL database, using both standard SQL and, where it seems useful, json.
Conceptnet for basic commonsense knowledge
Similarly to Quasimodo, explore, and if it makes sense, use conceptnet. This contains a lot more than Quasimodo, but does not have as good metainformation (context, plausibility, etc).
In particular,
- Find out if the dataset contains meaningful amount of information about the geographical places (or additional information about the facts about these places) for the sub-part you chose from Yago.
- If yes, please store a significant amout of relevant meaningful information part in the SQL database, using both standard SQL and, where it seems useful, json. Since Conceptnet is big, do not attempt to store all the seemingly relevant information.