Back Cover Blurb
- This book is a companion volume to "Elman (Jeffrey), Bates (Elizabeth), Johnson (Mark), Karmiloff-Smith (Annette), Parisi (Domenico) & Plunkett (Kim) - Rethinking Innateness" (The MIT Press, 1996), which proposed a new theoretical framework to answer the question "What does it mean to say that a behavior is innate?' The new work provides concrete illustrations — in the form of computer simulations — of properties of connectionist models that are particularly relevant to cognitive development. This enables the reader to pursue in depth some of the practical and empirical issues raised in the first book.
- The authors' larger goal is to demonstrate the usefulness of neural network modeling as a research methodology. The book therefore makes an ideal introductory text for a laboratory-based undergraduate course in connectionist modeling; it can be used with or without the first volume.
- The first three chapters provide overviews of some of the technical aspects of doing simulations and of network dynamics and learning. The remaining chapters explore simulations that focus on various aspects of connectionist models.
- The book comes with a complete software package, including demonstration projects, for running neural network simulations on both Macintosh operating systems and Windows 95. It also contains a series of exercises in the use of the neural network simulator provided with the book. The software is also available to run on a variety of UNIX platforms.
- Kim Plunkett is a University Lecturer in Experimental Psychology and Fellow of St. Hugh's College, Oxford University.
- Jeffrey Elman is Professor of Cognitive Science and Chair of the Department of Cognitive Sciences at the University of California, San Diego.
- Software included with handbook:-
→ Tlearn version 1.0 for Macintosh
→ Tlearn version 1.0 for Windows 95
ContentsSeries foreword – ix
Preface – xi
- Introduction and overview – 1
- The methodology of simulations – 19
- Learning to use the simulator – 31
- Learning internal representations – 75
- Autoassociation – 99
- Generalization – 115
- Translation invariance – 137
- Simple recurrent networks – 151
- Critical points in learning – 163
- Modeling stages in cognitive development – 179
- Learning the English past tense – 203
- The importance of starting small – 233
APPENDIX A: Installation procedures – 259
APPENDIX B: User manual – 261
References – 301
Index – 305
Preface
- Learning to do simulations is like learning many skills. You learn best by doing. Reading about simulations and watching others do them can be useful. But when it comes down to it, there is no substitute for first-hand experience.
- Connectionists are certainly not the first to develop models, nor the first to run simulations of them. But simulations play a particularly important role in connectionism. Why is this?
- Connectionist models are often deceptively simple. After all, they are composed of simple neuron-like processing elements (usually called “nodes”). In fact, the nodes of most connectionist models are vastly simpler than real neurons. There appear to be no elaborate rules nor grammars (which has led some to argue — erroneously in our opinion — that connectionist models “have no rules”). The nodes are simply hooked together in rather straightforward arrangements, presented with some stimulus, and then allowed to interact freely. Periodically, some nodes may output values.
- Put this way, one might wonder why go to the bother of writing a computer program to actually simulate a process whose outcome ought to be entirely predictable in advance. The surprise is that very often the outcome is not what one might have predicted. The dynamics of these simple systems turn out to be quite complex. For various reasons, including the fact that they are nonlinear (see Chapter 4 in the companion volume, "Elman (Jeffrey), Bates (Elizabeth), Johnson (Mark), Karmiloff-Smith (Annette), Parisi (Domenico) & Plunkett (Kim) - Rethinking Innateness"), it is frequently only possible to know how they will perform by running careful experiments and observing their behavior under controlled conditions. (In this respect, they are not unlike humans and other unpredictable biological systems.) Simulations are therefore crucial, and modelers’ expectations are often betrayed.
- The experimental aspect of running simulations is something we want to emphasize in this book. For many of our readers, particularly those who have been trained as developmentalists, the experimental methodology will be familiar. Rather than thinking of doing simulations as a very new and different way of doing research, we encourage our experimentalist readers to regard this as simply doing experiments with a different sort of subject pool!
- There is also an important conceptual role which is played by simulations, and we wish to emphasize this as well. Although it may be satisfying to build a model which successfully learns some behavior, especially if that behavior is complex and not obviously learnable, we feel the real value of simulating models is that we can then ask. “How did the model solve the problem?" When we run experiments with human subjects, the most we can usually do is to construct theories which are consistent with our data. This is an inferential process. At best, we may have multiple sources of evidence which converge on the same theory. But rarely do we have the opportunity to look inside our subjects’ heads and verify that the mechanism responsible for the behavior we have observed is indeed at work.
- Simulated models, on the other hand, can be opened up for inspection and analysis1. What we often find is that the mechanism which is actually responsible for the behavior we observed from the outside looks very different from what we guessed it might be. We must be careful, of course, not to assume that the model's solution is necessarily the same as the one discovered by species or children. The model can be a rich source of new hypotheses, however. Discovering new solutions to old problems is very liberating and exciting.
- So the second thing we will emphasize in this handbook is understanding what makes the models tick. We will present several techniques for network analysis and try to extract the basic principles which are responsible for the networks’ behavior. Network analysis is not easy, and the tools we suggest here only scratch the surface of what one would like to do. The problem is not unlike the one we would encounter if we actually did get to open up our subjects’ heads: It's pretty complicated in there. The difference, of course, is that our models are vastly more tolerant of being poked and probed and lesioned than are our human subjects. And they recover faster!
- Do you need to know how to program in order to do simulations? No. Ultimately it’s helpful, but a great deal can be done without actually having to do any programming, and no programming at all is required to use this handbook. The handbook is organized around a set of simulations which can be run using the software which is provided on the enclosed floppy disks2. This software includes a neural network simulator called tlearn. Having a simulator allows you to experiment with different network architectures and training regimes without having to actually program the networks yourself (e.g., by writing software in C or Fortran). So, if you do not know how to program, don’t worry. The simulator simply needs to be told what the network’s architecture is, what the training data are, and a few other parameters such as learning rate. etc. The simulator creates the network for you, does the training, and then reports the results. Other tools are included which allow you to examine the network's solutions. Thus, while you might find it eventually useful to be able to program yourself (for example, to create new stimuli), you will not need to do any programming to run the simulations described in this book. You will need to be able to use a simple text editor: whichever one you use is up to you.
- The software itself exists in several versions, and will run on Macintoshes, PCs, Unix workstations, and most other machines which have ANSI standard C compilers. Precompiled binaries exist for several common platforms, but sources are provided in case you wish to modify and recompile the simulator yourself. Chapter 3 provides an overview of the simulator; installation procedures and reference manuals are supplied in Appendices A and B. When we designed our first few exercises, we kept in mind that most readers would need explicit help in using the simulator. These exercises should help ease the reader into the process of running a simulation. We do not guarantee that the software provided with this book is free of bugs. In fact, we guarantee that if you try hard enough you will find some situations where tlearn will break. Please tell us about any software problems you experience by emailing us at innate@crl.ucsd.edu. We may not respond to your queries immediately but we will fix the bugs as time permits. You can obtain the latest versions of tlearn via anonymous ftp at crl.ucsd.edu in the pub directory or on the world wide web3 at UCSanDiego - Exercises in Rethinking Innateness or via anonymous ftp at ftp.psych.ox.ac.uk also in the pub directory.
- A very major source of support — intellectual as well as monetary — for this project came from the MacArthur Foundation through their funding of the Neural Modeling Training Program for Developmentalists which was run at the Center for Research in Language at UCSD from 1988 to 1993. ….
How to use this book
- This book is designed as a companion volume to Rethinking Innateness: .4 connectionist perspective on development. That book is our attempt to sketch out what we hope may serve as a framework for a theory of development. The book necessarily omits some details, particularly regarding finer details of simulations. One purpose of this book is to provide a way for the interested reader to pursue in depth some of the issues which we raise in the first volume.
- Our larger ambition is that we will have persuaded some readers that this is a useful methodology which can serve them in their own research. We hope that this volume will therefore provide those readers with tools which they can use on their own. There are other simulators, of course, and architectures which may require more special-purpose software. This simulator, however, has served us well and is used by us not only for pedagogical purposes, but in our own research. We hope it will serve you as well!
→ KP & JLE, Oxford & La Jolla, 1997
In-Page Footnotes ("Plunkett (Kim) & Elman (Jeffrey) - Exercises in Rethinking Innateness - A Handbook for Connectionist Simulations")
Footnote 1: Footnote 2:
- PCs don’t have floppy drives these days, and Windows 95 is long dead! However, I found an old-ish external USB-powered floppy drive which worked and allowed me (in June 2024) to import tlearn.
- It runs, but I’ve not used it in anger yet.
Footnote 3:
- The link still works, and the book is still for sale, but there is no obvious link to any newer version of the software.
- I’ve not tried doing any FTP links: I doubt this would work.
Book Comment
Neural Network Modeling and Connectionism series. A Bradford Book. The MIT Press. Massachusetts Institute of Technology Cambridge, Massachusetts. Second printing, 1998. Paperback.
Text Colour Conventions (see disclaimer)- Blue: Text by me; © Theo Todman, 2025
- Mauve: Text by correspondent(s) or other author(s); © the author(s)