Evolution and its relationship with Agents
We think of a Genetic Algorithm as evolving an Organism, but it is really evolving particular Genes. The Organism, or abstractly the Agent, is simply the carrier for those genes. The Agent has no particular Cost Function attached to them.
If we think about the RNA World Hypothesis, little packets of RNA were able to replicate themselves (perhaps in some kind of lipid bilayer). The Agent (in this case the lipid package containing the RNA), was in the literal sense a carrier for the Gene. As time went on, presumeably, improved carriers were selected for such as
- replacing RNA with DNA
- development of cells and cellular functions
- multicellular organisms - specialized cells worked together to proliferate a single genome
- animals - needs, skills, intelligence so they could take care of their genes
We were evolved as carriers for our genes, and given “free will” as a mutation that was selected to better propogate our genome.
So what gives us the ability to “make choices” but not cellular organism or computers? When I talked with Gyorgy Turan, he suggested the idea of Inverse Reinforcement Learning, where the Agent “learns” a Reward Function or some form of tradeoff management. Perhaps we can break down some of the components of what we percieve as intelligence as
- internal understanding - some kind of low-dimensional understanding of the world and the ability to incorporate new information.
- choice-making - in the absence of a given optimization constraint, the ability to execute an action from multiple choices. How to frame this? Perhaps there are some rewards, but they are also attached predicted states, and change based on whether those states are achieved.
- consciousness - internal understanding of ones internals: a lower dimensional representation of one’s “brain”.