Readings – Artificial Intelligence Knowledge Representation

Representation Levels

• Rasmussen, J. (1983). Skills, rules, and knowledge: Signals, signs, and symbols, and other distinctions in human performance models, IEEE Trans. SMC, Vol SMC-13, No. 3 (May, 1983), 257-266.

I’ve just started reading this; it is a really good introduction. Lots of the basic concepts about representation levels, from the deep learning (connectionist, or bottom-up) perspective.

• Bengio, Y. (2009), Learning Deep Architectures for AI, Foundations and Trends in Machine Learning, (January, 2009). doi:10.1561/2200000006. pdf.

Representation Levels in Text Analysis and Machine Translation

• Daniel Gildea and Daniel Jurafsky, 2002. Automatic Labeling of Semantic Roles Computational Linguistics 28 (3), 245-288. pdf.
  • Abstract: We present a system for identifying the semantic relationships, or semantic roles, filled by constituents of a sentence within a semantic frame. Given an input sentence and a target word and frame, the system labels constituents with either abstract semantic roles such as AGENT or PATIENT, or more domain-specific semantic roles such as SPEAKER, MESSAGE, and TOPIC. The system is based on statistical classifiers trained on roughly 50,000 sentences that were hand-annotated with semantic roles by the FrameNet semantic labeling project. We then parsed each training sentence into a syntactic tree and extracted various lexical and syntactic features, including the phrase type of each constituent, its grammatical function, and position in the sentence …
  • AJM’s Notes: You can skim / read this lightly, just to get the sense that three different representation levels – statistical, syntactic, and semantic – interact with each other when we do text understanding. Gildea and Jurafsky have done immense work over the past decades, and their work is substantial and respected. This is a good reference article.

Some more useful reads that deal with multiple levels of knowledge representation for linguistic AI:

• Marta R. Costa-jussà and Chris Quirk, Morphological, Syntax and Semantic Knowledge in Statistical Machine Translation
• Dharmendra KANEJIYA, Arun KUMAR†, Surendra PRASAD (Statistical Language Modeling with Performance Benchmarks using Various Levels of Syntactic-Semantic Information, pdf
• This web page some useful thoughts: Knowledge Representation
• Chaitanya Chemudugunta1, America Holloway1, Padhraic Smyth, and Mark
  Steyvers (2008). Modeling Documents by Combining Semantic Concepts with Unsupervised Statistical Learning, Proc. 7th International Semantic Web Conference. pdf.
  • Abstract: Human-defined concepts are fundamental building-blocks in constructing knowledge bases such as ontologies. Statistical learning techniques provide an alternative automated approach to concept definition, driven by data rather than prior knowledge. In this paper we propose a probabilistic modeling framework that combines both human-defined concepts and data-driven topics in a principled manner. The methodology we propose is based on applications of statistical topic models (also known as latent Dirichlet allocation models).

Representation Levels – Ancillary

Old publications that inspired me at the time; potentially useful.

• Hengst, Bernhard (2003). Theory of hierarchical, multilevel, systems, Discovering Hierarchy in Reinforcement Learning, PhD Thesis, School of Computer Science and Engineering, University of New South Wales, Australia, December 2003.
• Mesarovic, M.D., Macko, D., and Takahara, Y. (1999). Theory of hierarchical, multilevel, systems, Mathematics in science and engineering, ed. Bellmann, Richard. New York: Academic Press, 1970.
• Timpf, S. (1999).Abstraction, levels of detail, and hierarchies in map series, Spatial Information Theory – cognitive and computational foundations of geographic information science, Stade, Germany, Freksa, Christian and Mark, David M. (eds.), Springer-Verlag, LNCS 1661, pp.125-140.
• Vicente, Lola & Vellido, Alfredo (2011). Review of Hierarchical Models for Data Clustering and Visualization, Preprint.
• Voorspoels, Wouter, Storms, Gert, & Vanpaemel, Wolf (2011). Representation at different levels in a conceptual hierarchy, Acta Psychologica, 138, 1 (September 2011), Pages 11–18.