ITV9070 2023
Code: ITV9070 "A sign is anything that can be used to tell a lie." --Umberto Eco “I’m writing a book on magic,” I explain, and I’m asked, “Real magic?” By real magic people mean miracles, thaumaturgical acts, and supernatural powers. “No,” I answer: “Conjuring tricks, not real magic.” Real magic, in other words, refers to the magic that is not real, while the magic that is real, that can actually be done, is not real magic. --Lee Siegel |
Sisukord
NB! This is an archive of the year 2023, not the actual materials!
Time, place, credits
Time: Every Friday 16:00 - 17:30
- Normally we will have student presentations during the lecture time, along with planning following presentations and brief intros by the lecturer. We will follow the schedule
- There will be additional video lectures by the lecturer during some of the weeks to study, in addition to the Friday time above.
Location: IT building room ICT-507AB
Credits: 6
The lectures will be in English, unless everybody understands Estonian, in which case they will be in Estonian.
The presentations in practice sessions can be given in English or Estonian, as you prefer.
The contents of practice sessions are presentations by students and their discussion. For all students, presence in the practice sessions is important and is accounted for in the final grade.
We will probably succeed with saving the videos of lectures: will check out during the first weeks.
The course will finish with a written exam. The materials below marked with a red E will be critical for studying for the exam, while the materials marked with blue L are also used but less important. Materials without such marks will not be used for the exam questions.
In order to be allowed to take the exam, each student should give at least four presentations. Presentations are not graded, but when they are determined not to be satfisfactory, they should be re-done.
Some of the later lecture times will be used as additional practice sessions.
Initial course plan, materials and topics
We consider several topics, starting from general ones and moving to more specific.
The main goal of the lectures is to make it easier to understand the concepts and ideas in the topics, in order to better understand the reading materials.
Intro, semiotics, philosophy
- Source for the first round of presentations: Tähendus, tõde, meetod.. Separate original articles can be found by googling.
Main materials:
- Semiotics for beginners. first half is obligatory. E
- Quine: Two Dogmas of Empiricism. E
- what is analytic philosophy.
- analytic/synthetic. E
- logical positivism.
- Problem of universals nominalism/realism. L
- functionalism. E
- phenomenology.
- Propp.
Useful to read/listen:
- philosophy bites podcasts. Start with this strongly recommended episode.
- Sam Harris with David Chalmers
- L Philosophy of Metzinger: background reading Self models and look, for example: Metzinger video
- Tractatus andTractatus: structured full text. L. Wittgenstein. Read a bit, along with a commentary
- Philosophical investigations: read the first 100.A A small summary of early vs late Wittgenstein
About the alphabet:
- http://en.wikipedia.org/wiki/Egyptian_hieroglyphs
- http://en.wikipedia.org/wiki/Written_Chinese
- comparison ja wiki and unicode
- http://en.wikipedia.org/wiki/Phoenician_alphabet
- http://en.wikipedia.org/wiki/Egyptian_uniliteral_signs
- http://en.wikipedia.org/wiki/Cumae_alphabet
Notes on structuralism, deconstruction and friends:
- structuralism L, dekonstruktivism, dekonstrueerimise kiirõpetus, Derrida section, hermeneutics
- insheeption
- sympathetic magic: wiki, Frazer
Notes on Propp:
Notes on the Vienna circle and logical positivism:
- Main ideas: "First, experience is the only source of knowledge; second, logical analysis performed with the help of symbolic logic is the preferred method for solving philosophical problems."
- taust: analüütiline/sünteetiline, apriori/aposteriori
- classical positivism as the background: more here
- a small lecture about the Vienna cicle
- Vienna circle: good short overview ja a longish paper from the Stanfordi encyclopedia of philosohy and Vienna circle manifesto
- Logical positivism: good overview
- wiki logical positivism
Evolution of brain, intelligence and society
Sociobiology, evolutionary psychology and memetics.
Recording of the lecture: 15 Sept
Main material:
- Evolution: intro lecture.
- Sociobiology in wikipedia. E.
- Evolutionary psychology in wikipedia. E. You may also want to read more details in primer ja conceptual grounds
- Memetics in wikipedia. E
- Universal grammar L theory of Chomsky
Recommended overviews:
- wiki on the theories of origin of language
- overview of Chomsky theories of language evolution
- wiki on horizontal gene transfer
Recommended books:
- The Selfish Gene
- Guns, Germs and Steel
- Sapiens: A Brief History of Humankind
- Homo Deus: A Brief History of Tomorrow
Important papers:
- BRAIN EVOLUTION AND NEUROLINGUISTIC PRECONDITIONS L
- CO-EVOLUTION OF NEOCORTEX SIZE, GROUP SIZE AND LANGUAGE IN HUMANS L
- Precis of Origins of the Modern Mind: Three Stages in the Evolution of Culture and Cognition L
- DNA seen through the eyes of a coder
- Epigenetics vt ka seda artiklit
Additional notes and recommendations, not obligatory:
- natural language and natural selection
- Language as an Adaptation to the Cognitive Niche
- Faculty of Language: What Is It, Who Has It, and How Did It Evolve?
- Agner Fog: cultural selection
- Buss lab publications
- Changizi publications: vt allpool peatükk ARTICLES AND BOOKS ntx Economically organized hierarchies in WordNet and the Oxford English Dictionary, citybrain, colorface, xray vision
- the brain from 25000 feet
- John Eccles: Evolution of consciousness
Brain science
Intro talk video: 29 sept.
For exam:
- https://en.wikipedia.org/wiki/Cognitive_neuroscience L
- https://en.wikibooks.org/wiki/Cognitive_Psychology_and_Cognitive_Neuroscience/Memory L
- https://en.wikipedia.org/wiki/Computational_theory_of_mind E
Good general overviews:
- background reading for further reading or listening about brain
- https://grey.colorado.edu/CompCogNeuro/index.php/CCNBook/Main
- https://en.wikibooks.org/wiki/Cognitive_Psychology_and_Cognitive_Neuroscience
- Neurons are Now More Complex than We Thought
Important podcasts:
- Brain Inspired: a podcast where neuroscience and AI converge. A somewhat newer and somewhat technical podcast. Also based on interviews, but with currently active top researchers about their current work.
- Brain science (see also the episodes page) : an old podcast series, still going strong. Basically interviews with important people in the field, many of them classics. Not very technical (mostly); recommended podcasts in BS, youtube lectures and background for these
- Some of the Lex Fridman podcast episodes are relevant, like Jeff Hawkins, Karl Friston, David Eagleman
General notes:
- https://en.wikipedia.org/wiki/Cognitive_neuroscience
- https://en.wikipedia.org/wiki/Computational_theory_of_mind
- https://en.wikipedia.org/wiki/Global_Workspace_Theory
- https://en.wikipedia.org/wiki/Grandmother_cell
Read also:
Infinity, logic and philosophy of mathematics
The first topic is intro to the philosophy of mathematics, mainly the questions of infinity and unsolvability.
Materials to read:
- English intro: infinity, limits of computation and undecidability E
- in parallel, the same themes supporting text, written by the lecturer as an afterword to the book about Gödel's incompleteness theorems. You may try to use Google translate or some such for the English version: however, it is not a an obligagtory material.
- numbers, infinity and metaphors (in Estonian, very short, metaphorical)
- Ordinals: read from 6.2 Classifying the ordinals
- Russell paradox ja and longer from wikipedia
Videos have heads down:
- Intro lecture: 6. oct 2022, in Estonian.
- Seminar lecture: 6. oct 2022, in Estonian.
More about infinity and computability:
- Salvestus teamsis, 13. oktoober, in Estonian.
Helpful read: cardinality wikipedias. Read more about Gödel's incompleteness theorem from SEP: a part of a larger incompleteness-article.
A longer story about the philosophy of mathematics from SEP.
Nonclassical logics, related to the philosophy of mathematics:
- In Estonian: mitteklassikalised loogikad. Siin on pealeloetud jutuga variant: E downloadi ja vaata normaalses täisekraan-vaatamisreziimis, siis kuuled slaidide juures minu juttu.
- A good technical read from SEP about intuitionist logic.
- Estonian material and videos about modal logics: modaalsed loogikad L ja vaata eelmise aasta teamsi salvestust samast loengust
- Good half-hour presentations about axioms of modal logic and kripke semantics / possible worlds. A more philosophical take available in the series (playlist) from the same author, but long. Additionally useful presentation . SEP has good articles om modal logic and related philosophy, both highly technical.
More on nonclassical logics:
- modal logic in wikipeda
- philosophy around modal logic from SEP
- philosophy around intuitionistic logic from SEP L
Thinking and AI
Detailed arguments and counterarguments for the
Some good core stuff
- For starters Rodney Brooks's small AI series part 1, part 2 , part 3 , part 4 and predictions scorecard 2020
- Importantly, two interesting essays by Rodney Brooks from 2023 What Will Transformers Transform? L and Three Things That LLMs Have Made Us Rethink L
- Listen to Stuart Russell on AI dangers
Living classics from the lex fridman podcast:
- https://lexfridman.com/david-ferrucci/
- https://lexfridman.com/gary-marcus/
- https://lexfridman.com/david-silver/
- https://lexfridman.com/tomaso-poggio/
- https://lexfridman.com/richard-dawkins/
- https://lexfridman.com/noam-chomsky/
- https://lexfridman.com/nick-bostrom/
- https://lexfridman.com/jeff-hawkins/
- https://lexfridman.com/judea-pearl/
Semantics of natural language
We will look into both
- Vector semantics
- Semantics based on world model
Details will be added when the block starts
Dialog, pragmatics, uncertain knowledge
The first subject is dialogue pragmatics. Several related branches (have a brief look, but no need to read deeply unless you become interested):
- Speech acts L sentences doing something, not just conveying information
- Implicature L with Grice's maxims as a central theme
- Relevance theory further development of Grice's maxims
To start, certainly look/listen these short lectures:
- pragmatics and speech acts 11 minutes intro to the whole subject
- Grice's Maxims, Implicature, Presupposition: especially nice 12 minute more detailed lecture about Grice maxims
- Deceiving with words in case you did not listen before, do it now!
In case you just want to read original Grice:
- Logic and conversation: Grice himself about the maxims
- further examples, analyzed quite deeply
Have a look at
- Relevance theory, ie further development: 32 pages by main folks working on the subject
- Jurafsky & Martin chapter 26 Dialogue Systems and Chatbots the practical pre-LLM systems from their extremely good NLP book
Additionally, nice to listen and :
- NLP podcasti episode 89 about pre-LLM dialog systems
- Implicature Calculation, Pragmatics or Syntax, or both: more detailed analysis of implicature
- implicature with experiments
- very detailed with some focus on automatization
- dialogue pragmatics from SEP
- implicature from SEP
The next theme is uncertain information and probabilities: how to understand, write down and make conjectures.
General intro here as a part of this course.
Read this intro: Numeric uncertainty (L as explained next):
- Certainly up the the section "Different ways to encode confidences in logic", also looking at the wikipages pointed to
- Briefly look into the following sections, but do not focus or spend much time on these
Then read the first parts and have brief looks at the SEP articles:
Here are some of our papers on our system, about confidence and reasoning with exceptions.
Extra stuff
- A good Slate article about the background of the OpenAI soap opera.
- A famous killerbot warning video.
Presentations
Schedule
Tarmo, Anton, Rahman, Fuad: scheduled for 13 and 20 oct, see below.
15. Sept
- Eva-Lotta - Putnam: tähendus ja osutus. (Meaning and reference). Look at https://www.uvm.edu/~lderosse/courses/lang/Putnam(1973).pdf or https://danielwharris.com/teaching/364/readings/PutnamMeaningReference.pdf and then continue in this direction https://en.wikipedia.org/wiki/Semantic_externalism ja https://plato.stanford.edu/entries/linguistics/#ThrAppLinTheExtEmeEss
- Tiina - Quine: empirismi kaks dogmat. (Two dogmas of empiricism) Look at https://link.springer.com/chapter/10.1007/978-94-010-1863-0_2 (a paper in this collection) and then continue to https://en.wikipedia.org/wiki/Two_Dogmas_of_Empiricism and https://www.britannica.com/topic/philosophy-of-language/Ordinary-language-philosophy#ref923945
- Deniel - Tarski: semantiline tõekontseptsioon (The Semantic Conception of Truth). Look at https://sites.ualberta.ca/~francisp/Phil426/TarskiTruth1944.pdf and then continue to https://en.wikipedia.org/wiki/Semantic_theory_of_truth and https://plato.stanford.edu/entries/truth/ and https://plato.stanford.edu/entries/tarski-truth/
22. Sept
- Eva - Dummett: Truth. Read full text and continue to discussion and the SEP article about truth
- Siim - evolution of morals (not 15 Sept). First get a brief general background: sociobiology and evolutionary psychology. Then the useful stuff to start the subject with (but search for more!): wikipedia, SEP article and continue to this article and this
- Deniss - Ramsey subjektiivne tõenäosusteooria (Ramsey's subjective theory of probabilities). Start with SEP about Ramsey and SEP article about the issues. Then read Ramsey's paper or the same paper here. Also, search for more context.
29. Sept
- Ian - How do bacteria and simple organisms communicate. Start with this Ted lecture and then search for more materials.
- Jaanus - mirror neurons (not 15, 22 Sept). Recommended to start with the brain science podcast 35 Mirror Neurons and podcast 39 (Discusses the role of mirror neurons in imitation and language). Certainly read about the current views and controversies. Search for more materials: it is a popular topic!
- Veiko - neurons. Start by searching the material yourself. Then, certainly read and use these articles: complexity of neurons, complexity of dendrites, research on subsynaptic learning.
6. Oct
- Villem - Frame problem in AI. Start with wiki and continue to Dennett for philosophy and this for blocks world example. For deeper stuff read SAP article and another classic about frame problem.
- Sadiq - modal logics S4 ja S5 and possible worlds (Kripke semantics). wiki is immediately technical and a bit hard to read. The SAP article with the Kripke addendum is better. A good detailed, yet approachable intro + further details are here. Read also this about epistemic modal logic.
- Zhe - frequentist vs bayesian vs other interpretations of probability. First have a look at the SAP page. Notice the Bayesian approach is a part of the wider "Subjective Interpretation" chapter. This is a good comparison of Bayesian/frequentist approaches.
13. Oct
- Tarmo: Main themes of the philosophy of Dennett + overview of his paper "quining qualia".
- Anton: bicameral mind. Start with this.
- Rahman: Structure of infinity: Ordinals and cardinals. Start with wiki cardinality,wiki cardinals and wiki ordinals, also Ordinals: read from 6.2 Classifying the ordinals
20. Oct
- Fuad: pragmatist philosophy
- Eva-Lotta: Gödels incompleteness theorems and their influence on philosophy. There is an OK overview in Estonian by the lecturer. You can get the main details from SEP.
- Tiina: Overview of the main points, arguments, thought examples and counter-arguments of/for the functionalist school. As always, SEP is a good detailed material. The computational mind thought line is very closely related to functionalism.
27. Oct
- Deniel: Nick Boström's Simulation Argument. Please prepare already for 20. Oct, as a spare.
- Eva: Anthropic principle and the Bolzmann brain
- Siim: the views of Andy Clark about perception, learning etc: start with https://www.samharris.org/podcasts/making-sense-episodes/322-predicting-reality
3. Nov
All of the following are really popular, hence it is easy to google for different papers, presentations and other materials. The symbolic vs non-symbolic is harder, due to an extreme amount of heated discussion.
- Deniss: Donald D. Hoffman and the user interface (MUI) theory of mind. Please prepare already for 27. Oct, as a spare.
- Ian: Kahneman & Tversky: briefly about Prospect theory and more on System 1 and System 2 thinking
- Jaanus: symbolic vs non-symbolic debate. If possible, please connect it also the classic nature-vs-nurture and individual-learning-vs-evolution and plato-vs-aristotle thought lines. It may be also interesting to have a look at this paper.
10. Nov
NB! The following schedule is preliminary: some people are still waiting for the slot, so please email me (again, in case you are not scheduled despite previous emails etc) to schedule you for the 10. Nov, pushing some of the presentations below to the 17. Nov:
- Veiko: Main points of Chomsky's philosophy of language. Main criticisms and support. Notice that it is a prime example of "evolution-first" as opposed to "learning-first" position.
- Villem: Paul Grice: overview of the most important of Grice's claims and theory, including dialog (start with https://en.wikipedia.org/wiki/Implicature and https://en.wikipedia.org/wiki/Speech_act and have a look at https://www.youtube.com/watch?v=CGJGIWqnSFY ).
- Sadiq: Intuitionistic logic + examples of proofs of theorems which are (a) possible classically, but not intuitionistically, (b) easy classically, but hard intuitionistically. Please prepare already for 13. Oct, as a spare. Start with wiki and continue to SEP. Have a look at the curry-howard stuff. Most importantly, google for materials talking about the irrationality(?) of the square root of two and around this.
17 Nov
Normal 30-min talks:
- Zhe: Wolfram's theory of the world, using hypergraphs. A useful podcast and ted talk and some earlier writings
- Tarmo: Judea Pearl on causality. A useful podcast episode and another and yet another and homepage.
- Anton : Karl Friston and the free energy principle . A usefulpodcast episode and the homepage.
24 Nov
Normal 30-min talks:
- Rahman: Yann LeCun on the problems of LLMs, like the need to find new architectures for machine learning and the need to connect LLMs to tools. lecun about his views, recent, another video with slides. An important detailed paper and a larger tool use paper
- Fuad: arguments about the danger / high risk of AI (e.g. Stuart Russell, Geoffrey Hinton, Yoshua Bengio), vs arguments about the usefulness / small risk of AI (e.g. Yann LeCun, Marc Andreessen, Andrew Ng). Best to google these guys presenting their views, since the topic is hugely popular.
- Tiina: vector semantics. Use this chapter from this book as the main material (surely you can reuse slides from here as well. Obviously, searching for additional examples etc would be also nice.
1 Dec
Quick 15-minute talks:
- Moved to 15 dec: Eva-Lotta: panpsychism and Philip Goff. A useful podcast episode. Also, have a quick intro look into the “hard problem of consciousness” of Chalmers.
- Moved to 15 dec: Tiina: Max Tegmark and the mathematical multiverse. A useful podcast episode.
- Deniel: David Deutsch and the probabilistic (or rather, not) nature of scientific theories. A useful podcast. Also the homepage. Here is one complicated paper on constructor theory, as a potential additional deep source.
- Eva: Jeff Hawkins and the main Numenta theories of intelligence. See numenta, some of the talks by Hawkins and, for example, a useful paper from there.
- Siim: Yejin Choi on AI and Common Sense. A suitable podcast episode and the homepage.
- Muaan: Anil Seth on consciousness. A useful podcast episode and a homepage.
8 Dec
Quick 15-minute talks:
- Ian: Different views on controlling AI dangers: Connor Leahy, listen to the episode 158 on https://www.eye-on.ai/podcast-archive and search for more
- Jaanus: David Krakauer about complex systems. See https://davidckrakauer.com/ and listen to https://www.preposterousuniverse.com/podcast/2023/07/10/242-david-krakauer-on-complexity-agency-and-information/
- Veiko: https://en.wikipedia.org/wiki/Symbolic_interactionism
- Villem: Mike Frank: Early Language and Cognition. Interview and perhaps a longer paper 1 and paper 2
- Sadiq: Select one: Either Nick Enfield: Language vs. Reality Interview and perhaps a longer paper or James Woodward on causation Interview and the homepage
- Zhe: philosophy of embodied mathematics, start with https://en.wikipedia.org/wiki/Where_Mathematics_Comes_From and explore further from there
15 Dec
Quick 15-minute talks:
- Tarmo: Antonio Damasio and the origin of important stuff in homeostasis. Interview and a ted talk. See also https://en.wikipedia.org/wiki/Somatic_marker_hypothesis
- Anton: Donald D. Hoffman and the user interface (MUI) theory of mind. Was planned for Deniss on 3. Nov, not presented? Easy to google, highly popular.
- Not present: Rahman: Randy Gallistel on memory interview 1 and maybe also interview 2. Some additional background: here
- Fuad: Confirmation bias
- Eva-Lotta: panpsychism and Philip Goff. A useful podcast episode. Also, have a quick intro look into the “hard problem of consciousness” of Chalmers.
- Tiina: Max Tegmark and the mathematical multiverse. A useful podcast episode.
plus 1 reserve slot to be filled as needed
22 Dec
- Consultation for exam + some spare slots, including:
- Tiina: functionalism
Some themes for selection
Ideas for the first rounds from this book (search originals from the web):
- Putnam: tähendus ja osutus. (Meaning and reference)
- Quine: empirismi kaks dogmat. (Two dogmas of empiricism)
- Dummett: Truth.
- Tarski: semantiline tõekontseptsioon.
- Strawson: tähendus ja tõde
- Ramsey subjektiivne tõenäosusteooria.
Further:
- Nick Boström "are we living in simulation". http://www.simulation-argument.com/ and http://www.nickbostrom.com/
- Paul Grice: "meaning" and overview of other important parts of Grice claims and theory.
- Evolution of morals. alusta siit ja otsi ise edasi.
- Frege: Frege tähendusest ja osutusest.
- Dennett: kvaalide kvainimine (quining qualia).
- Chomsky keeleteooria põhipunktid
- Mirror Neurons mh alustuseks brain science podcast 35 Mirror Neurons ja podcast 39 (Discusses the role of mirror neurons in imitation and language) pluss googelda veidi juurde.
- Frame problem in AI. Start with http://en.wikipedia.org/wiki/Frame_problem and continue to find additional materials and examples.
- How do bacteria and simple organisms communicate. Start with this TED lecture.
- infinity: cardinals and ordinals.
- Overview of pragmatism as a philosophical school: main ideas, development etc.
- example of a proof which is classically easy but intuitionistically hard: irrationality around square root of two.
- Anthropic principle and the Boltzmann brain.
- modal logics S4 ja S5 and possible worlds (Kripke semantics).
- neurons.
- Research and give an overview of the state of the art in machine translation: top systems, their capacity and the main principles of how they work.
- Statistical principles in machine translation a la Google
- Confirmation bias, start with this article
- How do bacteria and simple organisms communicate. Start with this TED lecture
- Evolution of morals. alusta siit
- How do bacteria and simple organisms communicate. Start with this TED lecture
- Frame problem in AI. Start with http://en.wikipedia.org/wiki/Frame_problem and continue to find additional materials and examples.
- Intuitsionistlik loogika: näited teoreemidest, mis kehtivad klassikaliselt, kuid mitte intuitsionistlikult. Näiteid tõestustest, millel lihtne klassikaline, kuid keeruline intuitsionistlik tõestus.
- Ordinaalid ja kardinaalid.
- Neuroimaging, auditory hallucinations, and the bicameral mind.pdf start here and download pdf
- Neurons are more complex than we thought, see here and here for starters.
- Could a Neuroscientist Understand a Microprocessor?
- The Hyperbolic Geometry of DMT Experiences and see this for some background.
- Perception Without Representation
- Brain Computation Is Organized via Power-of-Two-Based Permutation Logic