Learning Curve

Mark S. Miller from the December 1996 issue

(Page 2 of 3)

Genetic algorithm research could shed new light on controversies about intellectual property rights. Evolution normally works by trying small variations on good ideas to search for better ones. The patent system inhibits exploration of small variations--because they would be infringing--biasing innovation toward taking large leaps. How does this affect overall progress? Although addressing such questions also requires careful institutional analysis, Holland gives us some useful new tools. Examining innovation as evolutionary learning would complement conventional analyses of free riders, monopolistic pricing, and transaction costs.

Classifier systems are motivated by a fundamental puzzle. Sensation necessitates classification into categories that, in turn, can only be learned by exposure to sensations. Can this be untangled? This same question motivated Hayek's The Sensory Order, and both Hayek and Holland start with the same observation: One part of a mind might sense and learn about other parts of a mind much as a whole mind senses and learns about the external world.

From here, Holland goes much further than Hayek. He constructs an ecosystem of creatures evolving by genetic algorithms in which a "chromosome" is interpreted as an if-then rule. The if part is a pattern that matches some combination of stimuli from the external world, as well as the then parts of other rules. When stimuli arrive, the classifier framework triggers those rules with closely matching if parts. The then parts of these rules are added to the "stimuli," triggering further rules. In this way, the mechanisms used to recognize patterns in raw stimuli are also used to recognize patterns of prior recognition events. The resulting uniformity enables sophisticated perceptions to grow on simpler ones.

Why do some perceptual categories prosper while others wither away? Classifier systems use market-like competition in a procedure for discovering which categories reflect regularities in the world. Some then parts trigger actions in the external world, much as minds trigger muscles. These external actions have consequences that may be good or bad. When judged good, the classifier framework "pays" the rule that triggered the action. This rule passes along some of the fee to the rules that triggered it--payment for "recognition subcontracting," as it were. These subcontractor rules, in turn, pay their subcontractors. This "flow" of payments retraces, in reverse order, the flow of causation that led from external stimuli to useful actions. Over time those "recognition businesses" not contributing to useful behavior go broke, while the others are getting paid and having sex.

Holland built this bridge between markets and learning in order to carry ideas from markets into classifier systems. What if we cross this bridge in the other direction? The network of relationships in a market economy--who contracts with whom, or even who knows whom--constantly shifts, adapting to complexity in the world. When Holland's classifier networks shift their connectivity because of payment flows, they also indirectly learn facts never known to any creature within the network. Perhaps human networks of trade learn facts not known to any individual. Indeed, once one understands classifiers, it becomes hard to suppose this is not the case. Though we cannot know these facts in particular, we may abstractly reason about the learning process, and come to realize that some of the rigidities imposed on markets (perhaps SEC constraints on investing) do vastly more damage to learning than others. Such differences would be missed by conventional social-cost and efficiency analysis.

Classifiers use tagging to determine which rules match which stimuli. The tag used to indicate a given property is arbitrary--why is cat the word for cat? A tag's "meaning" is only established by use and experience. Over time, tags evolve to profitably match recognition businesses and their potential customers, thereby reinforcing useful perceptual categories. This use of tagging suggests how to apply some of Holland's work to markets and language: A city contains many buyers and sellers, often trying to find each other. Once it becomes known that Castro Street is good for Chinese food, or Lawrence Expressway for computer equipment, buyers and sellers know where to meet. How do these meeting places emerge, and how does their character shift over time? The economist Thomas Schelling, in his book The Strategy of Conflict , describes a game played on students--they would each receive $1,000 if they met in New York without prior communication. Many went to the clock at Grand Central Station because they expected it to be a mutually vivid choice. As these expectations change, the profitable places to meet shift. As places shift, new expectations are learned.

Similarly, language involves speakers and listeners using words to try to mean the same thing. Language evolves to provide for broad agreement on a word's meaning, as well as subtle shifts of meaning over time. If words are considered as places in a space of possible sounds, we can think of the problem of agreeing on a word as one of selecting a meeting place.

Are the causes and dynamics of shifts in a word's meaning similar to the change in character of a shopping area? How do diverse incentives interact to pull a tag in different directions? Might some terminology shifts be parasitic mimicry phenomena, like the perpetual need for new euphemisms as old ones are used up? By providing a common conceptual framework across systems this different, and by mixing their metaphors with great agility, Holland provokes such cross-disciplinary questions even when his book does not ask them. To establish a new discipline, rather than a collection of somewhat related but ultimately independent fields, requires raising such questions.

Echo is a richer but less mature ecosystem designed to explore the emergence of complex aggregations, such as multi-cellular organisms or corporations. While it's too early to tell how well it will run, Echo demonstrates Holland's ability, decade after decade, to raise and explore important new questions. Evolution operates on information patterns that can replicate, he says, but what about the learning involved in growing a symbiotic arrangement? The creatures within this arrangement can replicate and evolve, but what about the arrangement itself? Echo seeks to probe such issues. It is a model of how symbiotic arrangements can be templates for forming larger, more complex, replicable creatures.

To realize this, Echo introduces adhesions, boundaries, and conditional replication. Symbiotes sufficiently interdependent come to adhere to each other, and a set of such closely coupled creatures may form a boundary--interior creatures are no longer available for interaction with outsiders, and so no longer need to be prepared for these interactions. In a differentiation process inspired by how embryos develop, once a boundary forms, the resulting "multi-agent" grows by replicating its component creatures into positions that approximately replicate their original relationships. With this transformation, the learning embodied in the structure of the arrangement becomes subject to normal evolutionary processes. Some of the organelles in our cells, such as mitochondria, started out as independent creatures. Plausibly, many vertically integrated companies form by "copying" a spontaneously grown pattern of subcontracting.

Anyone brave enough to attempt broad interdisciplinary work faces the danger of saying foolish things outside their area of expertise. Holland's courage is to be praised, but his book errs when relating his insights to economics. Were economics treated only as one cas example among many, these errors would not matter so much. However, the book's motivation relies on economics, so these errors must be dealt with.

The main discussion of flows, for instance, speaks of material flowing through a system, and the need for recycling to maintain a high concentration and avoid shortages. The biological ecosystem, however, is the only cas Holland presents for which the relevant flows are subject to material shortages. For all the others, flows are information signals and the notion of shortages makes no sense. The only attempt to show that the recycling issue is general uses a naive Keynesian analysis of money. Holland never mentions that the shortage goes away when the value of money changes. And his own ecosystems engage in sophisticated flows that make no use of recycling.

On a similar note, he states that the "tragedy of the commons" occurs simply "because each person mistrusts the moderation of others," omitting any discussion of the ways in which private property rights avoid such overuse of resources. To explain that cas suffer from problems that might be more generally investigated, the book repeatedly mentions viruses and trade balances. I can understand why viruses might be problematic, but trade balances?

Surprisingly, what Holland presents as the chief contribution his work can make in economics is to help us identify "lever points" for fixing economic problems. With better insights into how economies function, he imagines we might find those key interventions for fixing various problems. (His example is a Depression-era make-work program, the Civilian Conservation Corps, presented with scant evidence of net benefit.) This is the familiar central-planning fantasy, and everything the book says on the matter has already been well answered by Hayek. Indeed, it has even been answered by Holland's own research: He built his computational ecosystems to develop according to their internal principles, not to enable himself to meddle and nudge. Each of his systems rests on a simple framework of rules--one might say a constitutional framework--designed to allow evolution without outside intervention. Were he to find himself manipulating "lever points" to keep his system on track, he would regard that as a bug to be fixed.

Contrast this with the system where he does propose such "outside" intervention--the market. In the market, the intervener is of no greater intelligence than the creatures populating the system, and of substantially lesser intelligence than the system as a whole. Worse, the information within the system cannot be gathered together. This is the Hayekian "knowledge problem," and it cannot be sufficiently emphasized. No one can ever succeed, no matter how totalitarian their control, at bringing together in one place the dispersed information that individuals and market structures are locally adapting to but are mostly unaware of. Holland endows his learning systems with a large population so the resulting system can learn more, and can behave in ways that take more knowledge into account. If a smaller population of creatures were given levers for controlling a larger population, this would simply reduce the aggregate intelligence of the system toward that of the smaller population.