Representation in Cognitive Science Representation in Cognitive Science Nicholas Shea 1 3 Great Clarendon Street, Oxford, OX2 6DP, United Kingdom Oxford University Press is a department of the University of Oxford. It furthers the University’s objective of excellence in research, scholarship, and education by publishing worldwide. Oxford is a registered trade mark of Oxford University Press in the UK and in certain other countries © Nicholas Shea 2018 The moral rights of the author have been asserted First Edition published in 2018 Impression: 1 Some rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, for commercial purposes, without the prior permission in writing of Oxford University Press, or as expressly permitted by law, by licence or under terms agreed with the appropriate reprographics rights organization. This is an open access publication, available online and distributed under the terms of a Creative Commons Attribution – Non Commercial – No Derivatives 4.0 International licence (CC BY-NC-ND 4.0), a copy of which is available at http://creativecommons.org/licenses/by-nc-nd/4.0/. Enquiries concerning reproduction outside the scope of this licence should be sent to the Rights Department, Oxford University Press, at the address above Published in the United States of America by Oxford University Press 198 Madison Avenue, New York, NY 10016, United States of America British Library Cataloguing in Publication Data Data available Library of Congress Control Number: 2018939590 ISBN 978–0–19–881288–3 Printed and bound by CPI Group (UK) Ltd, Croydon, CR0 4YY Links to third party websites are provided by Oxford in good faith and for information only. Oxford disclaims any responsibility for the materials contained in any third party website referenced in this work. To Ellie Huffing and puffing with correlation and function might give us a good account of subdoxastic aboutness. . . [but it is unlikely to work for the content of doxastic states.] * Martin Davies (pers. comm.), developed in Davies (2005) * Not that the antecedent is easy. And even for the subpersonal case we may have to puff on a few more ingredients. But I too am optimistic that we can get a good account. This book aims to show how. Preface The book is in three parts: introduction, exposition, and defence. Part I is introductory and light on argument. Chapter 1 is about others’ views. Chapter 2 is the framework for my own view. I don’t rehearse well-known arguments but simply gesture at the literature. The aim is to demarcate the problem and motivate my own approach. Part II changes gear, into more standard philosophical mode. It aims to state my positive view precisely and to test it against a series of case studies from cognitive science. Part III engages more carefully with the existing literature, showing that the account developed in Part II can deal with important arguments made by previous researchers, and arguing that the framework put forward in Part I has been vindicated. There is a paragraph-by-paragraph summary at the end of the book. Readers who want to go straight to a particular issue may find this a useful guide. It replaces the chapter summaries often found at the end of each chapter of a monograph. The bibli- ography lists the pages where I discuss each reference, so acts as a fine-grained index to particular issues. There is also the usual keyword index at the end. Part I 1. Introduction 3 1.1 A Foundational Question 3 1.2 Homing In on the Problem 8 1.3 Existing Approaches 12 1.4 Teleosemantics 15 1.5 Challenges to Teleosemantics 18 2. Framework 25 2.1 Setting Aside Some Harder Cases 25 2.2 What Should Constrain Our Theorizing? 28 2.3 Externalist Explanandum, Externalist Explanans 31 2.4 Representation Without a Homunculus 36 2.5 What Vehicle Realism Buys 37 2.6 Pluralism: Varitel Semantics 41 Part II 3. Functions for Representation 47 3.1 Introduction 47 3.2 A Natural Cluster Underpins a Proprietary Explanatory Role 48 3.3 Robust Outcome Functions 52 3.4 Stabilized Functions: Three Types 56 (a) Consequence etiology in general, and natural selection 56 (b) Persistence of organisms 57 (c) Learning with feedback 59 (d) A ‘very modern history’ theory of functions 62 3.5 Task Functions 64 3.6 How Task Functions Get Explanatory Purchase 67 (a) Illustrated with a toy system 67 (b) Swamp systems 69 3.7 Rival Accounts 72 3.8 Conclusion 74 4. Correlational Information 75 4.1 Introduction 75 (a) Exploitable correlational information 75 (b) Toy example 80 4.2 Unmediated Explanatory Information 83 (a) Explaining task functions 83 (b) Reliance on explanation 88 (c) Evidential test 89 Contents x contents 4.3 Feedforward Hierarchical Processing 91 4.4 Taxonomy of Cases 94 4.5 One Vehicle for Two Purposes 96 4.6 Representations Processed Differently in Different Contexts 97 (a) Analogue magnitude representations 97 (b) PFC representations of choice influenced by colour and motion 100 4.7 One Representation Processed via Two Routes 103 4.8 Feedback and Cycles 106 4.9 Conclusion 110 5. Structural Correspondence 111 5.1 Introduction 111 5.2 The Cognitive Map in the Rat Hippocampus 113 5.3 Preliminary Definitions 116 5.4 Content-Constituting Structural Correspondence 120 (a) Exploitable structural correspondence 120 (b) Unmediated explanatory structural correspondence 123 5.5 Unexploited Structural Correspondence 126 5.6 Two More Cases of UE Structural Correspondence 132 (a) Similarity structure 132 (b) Causal structure 134 5.7 Some Further Issues 137 (a) Exploiting structural correspondence cannot be assimilated to exploiting correlation 137 (b) Approximate instantiation 140 (c) Evidential test for UE structural correspondence 142 5.8 Conclusion 143 Part III 6. Standard Objections 147 6.1 Introduction 147 6.2 Indeterminacy 148 (a) Aspects of the problem 148 (b) Determinacy of task functions 150 (c) Correlations that play an unmediated role in explaining task functions 151 (d) UE structural correspondence 154 (e) Natural properties 155 (f) Different contents for different vehicles 156 (g) The appropriate amount of determinacy 157 (h) Comparison to other theories 158 6.3 Compositionality and Non-Conceptual Representation 162 6.4 Objection to Relying on (Historical) Functions 166 (a) Swampman 166 (b) Comparison to Millikan and Papineau 169 contents xi 6.5 Norms of Representation and of Function 171 (a) Systematic misrepresentation 171 (b) Psychologically proprietary representation 174 6.6 Conclusion 175 7. Descriptive and Directive Representation 177 7.1 Introduction 177 7.2 An Account of the Distinction 179 7.3 Application to Case Studies 183 (a) UE information 183 (b) UE structural correspondence 185 7.4 Comparison to Existing Accounts 188 7.5 Further Sophistication 192 (a) More complex directive systems 192 (b) Another mode of representing 193 7.6 Conclusion 194 8. How Content Explains 197 8.1 Introduction 197 8.2 How Content Explains 198 (a) Explanatory traction in varitel semantics 198 (b) Non-semantic causal description? 200 (c) Doing without talk of representation 204 (d) Other views about the explanatory purchase of content 205 8.3 Causal Efficacy of Semantic Properties 208 8.4 Why Require Exploitable Relations? 209 8.5 Ambit of Varitel Semantics 210 (a) Representation only if content is explanatory? 210 (b) Are any cases excluded? 213 8.6 Development and Content 216 8.7 Miscellaneous Qualifications 218 8.8 How to Find Out What Is Represented 221 8.9 Differences at the Personal Level 222 Paragraph-by-Paragraph Summary 227 Acknowledgements 267 Figure Credits 269 References 271 Index 285 PA RT I 1 Introduction 1.1 A Foundational Question 3 1.2 Homing In on the Problem 8 1.3 Existing Approaches 12 1.4 Teleosemantics 15 1.5 Challenges to Teleosemantics 18 1.1 A Foundational Question The mind holds many mysteries. Thinking used to be one of them. Staring idly out of the window, a chain of thought runs through my mind. Concentrating hard to solve a problem, I reason my way through a series of ideas until I find an answer (if I’m lucky). Having thoughts running through our minds is one of the most obvious aspects of the lived human experience. It seems central to the way we behave, especially in the cases we care most about. But what are thoughts and what is this process we call thinking? That was once as mysterious as the movement of the heavens or the nature of life itself. New technology can fundamentally change our understanding of what is possible and what mysterious. For Descartes, mechanical automata were a revelation. These fairground curiosities moved in ways that looked animate, uncannily like the move- ments of animals and even people. A capacity that had previously been linked inextric- ably to a fundamental life force, or to the soul, could now be seen as purely mechanical. Descartes famously argued that this could only go so far. Mechanism would not explain consciousness, nor the capacity for free will. Nor, he thought, could mechan- ism explain linguistic competence. It was inconceivable that a machine could produce different arrangements of words so as to give an appropriately grammatical answer to questions asked of it. 1 Consciousness and free will remain baffling. But another machine has made what was inconceivable to Descartes an everyday reality to us. Computers produce appropriately arranged strings of words—Google even annoyingly finishes half-typed sentences—in ways that at least respect the meaning of the words they churn out. Until quite recently a ‘computer’ was a person who did calculations. Now we know that calculations can be done mechanically. Babbage, 1 Descartes (1637/1988, p. 44: AT VI 56: CSM I I40), quoted by Stoljar (2001, pp. 405–6). 4 introduction Lovelace, and others in the nineteenth century saw the possibility of general-purpose mechanical computation, but it wasn’t until the valve-based, then transistor-based computers of the twentieth century that it became apparent just how powerful this idea was. 2 This remarkable insight can also answer our question about thinking: the answer is that thinking is the processing of mental representations. We’re familiar with words and symbols as representations, from marks made on a wet clay tablet to texts appear- ing on the latest electronic tablet: they are items with meaning. 3 A written sentence is a representation that takes the form of ink marks on paper: ‘roses are red’. It also has meaning—it is about flowers and their colour. Mental representations are similar: I believe that today is Tuesday, see that there is an apple in the bowl, hope that the sun will come out, and think about an exciting mountain climb. These thoughts are all mental representations. The core is the same as with words and symbols. Mental repre- sentations are physical things with meaning. A train of thought is a series of mental representations. That is the so-called ‘representational theory of mind’. I say the representational theory of mind is ‘an’ answer to our question about thinking, not ‘the’ answer, because not everyone agrees it is a good idea to appeal to mental rep- resentations. Granted, doing physical manipulations on things that have meaning is a great idea. We count on our fingers to add up. We manipulate symbols on the page to arrive at a mathematical proof. The physical stuff being manipulated can take many forms. Babbage’s difference engine uses gears and cogs to do long multiplication (see Figure 1.1). And now our amazingly powerful computers can do this kind of thing at inhuman speed on an astonishing scale. They manipulate voltage levels not fingers and can do a lot more than work out how many eggs will be left after breakfast. But they too work by performing physical manipulations on representations. The only trouble with carrying this over to the case of thinking is that we’re not really sure how mental repre- sentations get their meaning. For myself, I do think that the idea of mental representation is the answer to the mystery of thinking. There is very good reason to believe that thinking is the process- ing of meaningful physical entities, mental representations. That insight is one of the most important discoveries of the twentieth century—it may turn out to be the most important. But I have to admit that the question of meaning is a little problem in the foundations. We’ve done well on the ‘processing’ bit but we’re still a bit iffy about the ‘meaningful’ bit. We know what processing of physical particulars is, and how pro- cessing can respect the meaning of symbols. For example, we can make a machine whose manipulations obey logical rules and so preserve truth. But we don’t yet have a clear idea of how representations could get meanings, when the meaning does not derive from the understanding of an external interpreter. 2 Developments in logic, notably by Frege, were of course an important intermediate step, on which Turing, von Neumann, and others built in designing computing machines. 3 We’ll have to stretch the point for some of my son’s texts. introduction 5 So, the question remains: how do mental states 4 manage to be about things in the external world? That mental representations are about things in the world, although utterly commonplace, is deeply puzzling. How do they get their aboutness ? The phys- ical and biological sciences offer no model of how naturalistically respectable prop- erties could be like that. This is an undoubted lacuna in our understanding, a void hidden away in the foundations of the cognitive sciences. We behave in ways that are suited to our environment. We do so by representing the world and processing those representations in rational ways—at least, there is strong evidence that we do in very many cases. Mental representations represent objects and properties in the world: the 4 I use ‘mental’ broadly to cover all aspects of an agent’s psychology, including unconscious and/or low-level information processing; and ‘state’ loosely, so as to include dynamic states, i.e. events and processes. ‘Mental state’ is a convenient shorthand for entities of all kinds that are psychological and bear content. Figure 1.1 Babbage’s difference engine uses cogs and gears to perform physical manipulations on representations of numbers. It is used to multiply large numbers together. The components are representations of numbers and the physical manipulations make sense in the light of those contents—they multiply the numbers (using the method of differences). 6 introduction shape of a fruit, the movement of an animal, the expression on a face. I work out how much pasta to cook by thinking about how many people there will be for dinner and how much each they will eat. ‘Content’ is a useful shorthand for the objects, properties and conditions that a representation refers to or is about. So, the content of one of my thoughts about dinner is: each person needs 150g of pasta What then is the link between a mental representation and its content? The content of a representation must depend somehow on the way it is produced in response to input, the way it interacts with other representations, and the behaviour that results. How do those processes link a mental representation with the external objects and properties it refers to? How does the thought in my head connect up with quantities of pasta? In short: what determines the content of a mental representation? That is the ‘content question’. Surprisingly, there is no agreed answer. This little foundational worry hasn’t stopped the cognitive sciences getting on and using the idea of mental representation to great effect. Representational explanation is the central resource of scientific psychology. Many kinds of behaviour have been con- vincingly explained in terms of the internal algorithms or heuristics by which they are generated. Ever since the ‘cognitive revolution’ gave the behavioural sciences the idea of mental representation, one phenomenon after another has succumbed to represen- tational explanation, from the trajectories of the limbs when reaching to pick up an object, to parsing the grammar of a sentence. The recent successes of cognitive neuro- science depend on the same insight, while also telling us how representations are realized in the brain, a kind of understanding until recently thought to be fanciful. Figure 1.2 shows a typical example. The details of this experiment need not detain us for now (detailed case studies come in Part II). Just focus on the explanatory scheme. There is a set of interconnected brain areas, plus a computation being performed by those brain areas (sketched in the lower half of panel (a)). Together that tells us how participants in the experiment manage to perform their task (inset). So, although we lack a theory of it, there is little reason to doubt the existence of representational content. We’re in the position of the academic in the cartoon musing, ‘Well it works in practice, Bob, but I’m not sure it’s really gonna work in theory.’ The lack of an answer to the content question does arouse suspicion that mental representation is a dubious concept. Some want to eliminate the notion of representa- tional content from our theorizing entirely, perhaps replacing it with a purely neural account of behavioural mechanisms. If that were right, it would radically revise our conception of ourselves as reason-guided agents since reasons are mental contents. That conception runs deep in the humanities and social sciences, not to mention ordinary life. But even neuroscientists should want to hold onto the idea of represen- tation, because their explanations would be seriously impoverished without it. Even when the causes of behaviour can be picked out in neural terms, our understanding of why that pattern of neural activity produces this kind of behaviour depends crucially on neural activity being about things in the organism’s environment. Figure 1.2 doesn’t just show neural areas, but also how the activity of those areas should be understood as introduction 7 representing things about the stimuli presented to the people doing a task. The content of a neural representation makes an explanatory connection with distal features of the agent’s environment, features that the agent reacts to and then acts on. One aspect of the problem is consciousness. I want to set that aside. Consciousness raises a host of additional difficulties. Furthermore, there are cases of thinking and primary (a) (b) secondary higher prediction sensory input 1 1 R R 2 ITG STG FG V1 sensory input prediction A/B 12 2 1.5 1 0.5 0 10 8 6 4 2 0 match non- match A/B 3 0 –0.5 –1 –1.5 –2 2.5 2 1.5 1 0.5 0 match non- match A/B match non- match A/B match non- match Current Opinion in Neurobiology 50% A B 50% 2 1 1 E E 2 2 match 50% 50% non-match Figure 1.2 A figure that illustrates the explanatory scheme typical of cognitive neuroscience (from Rushworth et al. 2009). There is a computation (sketched in the lower half of panel (a)), implemented in some interacting brain areas, so as to perform a behavioural task (inset). The details are not important for present purposes.