Possible Minds: 25 Ways of Looking At AI, A Collection of Essays Curated and Edited by John Brockman

SCRIBBLES  (Notes as I read)

-This book is a collection of essays by “thinkers” in the AI field.  *Note, the summaries for each section are what I take away from the essays, not necessarily the broadest or strongest themes.

-The book starts out with an introduction to Norbert Weiner’s writings on Cybernetics from the Cold War Era.

-First author is Seth Lloyd is a professor at MIT and a self-described “quantum mechanic” and is an expert in quantum mechanics.  Main theme that I got out of his essay:  exponential growth doesn’t sustain–at some point it plateau’s off.  There is a limit to Moore’s Law, those limitations are driven by physical properties of chips (e.g., limits on clock speeds b/c the chips themselves melt or limits on smaller size transistors running up against the properties of quantum physics (problems like leakage current and tunneling as devices get smaller and smaller).  The upper limit of processing power may still be decades away though.

-Second author is Judea Pearl who is a professor of cognitive science.  He introduced the idea of Bayesian networks to artificial intelligence – probability based model of machine reasoning to operate in complex and uncertain world– machines serve as evidence engines refining their beliefs in light of new evidence.

Pearl–humans are differentiated from machines in their ability to construct a metal map of the physical world and then alter the mental map through imagination.  Modeling and prediction analysis for computers cannot do that without a model of the world.  Thesis:  human level AI cannot emerge solely from model blind learning machines–it requires collaboration between data and models.  Current machines operate mostly in model blind world.  These systems cannot ask “what if” counterfactual questions.

Current learning machines mimic the evolutionary process of natural selection — this process explains how eagles have sharp eyesight evolved over millions of years but is not analogous to how humans have accelerated intelligence in a few hundred or thousand.  For that, machines need a model of the world in which they are to operate, sometime uniquely human.

Hierarchy of causal reasoning–can do very little with model blind inferences.  Theoretical framework of limitations – three levels:  (1) statistical reasoning (how observing one event changes belief about another); (2) actions (what happens if we raise prices); (3) counterfactual (what happens if I did something different).  Counterfactuals require models and external inputs.

-Stuart Russell–professor at UC Berkley who has concerns about AI.  He is an advocate for building AI systems with implicit uncertainty about the objects of their human creators, which would amount to a reordering of current AI research.  Also wrote a seminal book on AI.  Some scribbles from his essay:

The potential existential risk from AI–we have to put the purpose in the machine which we really desire.

Goal of AI research:  what are the principles underlying AI behavior, and how do you put them into machines?

Prevailing notion of intelligence was the ability for logical reasoning.  More recently, view of intelligence is a rational being that seeks to maximize its expected utility.

One of the current tenants of the AI field is that machines should be general purpose.  Capable of accepting purpose as input and then achieving it, rather than special purpose.  Fixed inputs vs general purpose inputs.

We need to put in the right purpose into machines–the King Midas problem of getting exactly what we ask for.  Value alignment is the technical term for putting in the right set of goals.  King Midas problem–variables not included in the objective may be set to extreme values and may be maximized to accomplish the objective.

Question is how to identify the purposes which we really desire.  Research needs to be placed on the design of our objectives.

Machines that have an off switch will take steps to disable the switch in some way in a form of self-preservation (not the biological form but in the form to achieve it’s objective).

Can AI solutions be designed with the right purposes?  For a given problem F, design AI systems to be “F’ solvers such that no matter how the AI system solves “F” we are always happy with the solution.

How not to do it:  let reward be a scalar value provided by a human to a machine based on how well a machine has behaved during a given period, and let F be the problem of maximizing the sum of rewards obtained by the machine.  The machine’s solution would not to be to perform well but it would be to take control of the human to provide the maximum rewards.  This is known as the “wire heading problem” based on observations that humans too are susceptible to this problem.

How to do it:  give machine initial uncertainty about what the rewards are and human preferences for providing the reward.  The machine may learn more about human preferences as it goes along, but it will never achieve full certainty on it.  A more precise definition:  “cooperative, inverse, reenforcement learning” (CIRL).  CIRL projects have two agents–a human and a robot.  This is a “game” problem in economic theory.  The human knows the game function but the robot doesn’t.  Example the author gives is the paper clip/staple example.  Person may receive a total of one hundred objects that are either paper clips or staples, and the person may prefer some percentage of staples over paperclips (like 51% preference staples).  The robot is to produce the paper clips and staples to make the human happy, but the robot doesn’t know the percentage of preference breakdown.  The game works such that the robot doesn’t know what the percentage break down is, the human can provide a demonstration to the robot–present either two staples, two paper clips, or one of each.  If two staples, robot will produce 90 staples.  If two paper clips, robot will produce 90 paper clips.  If one of each, robot will produce 50 of each. Even though human slightly prefers staples, if human presents two staples to robot, the robot will produce 90 staples and no paperclips.  The bottom line–the game is designed so that the human teaches the robot.

CIRL solves the off switch problem.  It can prevent the robot from disabling its off switch.  A machine that does not know the human preferences benefits from being switched off b/c it understands that the human will push the off switch if the robot is doing something counter to the preferences, so robot is incentivized to preserve the off switch.  The incentive derives from it’s uncertainty about human preferences.

Key way that CIRL is different than mechanism design problems in economics is that with CIRL one agent is being designed to benefit the another.  This approach may work because o a few things:  (1) technologies to build models of human preferences are vast; (2) there are strong near term incentives to understand human preferences (e.g., author gives example of robot cooking cat for dinner–economics will shut down the industry).  There are however difficulties to understand human preferences that originate from irrational human behavior, so there have to be good cognitive models of rational human behavior.  But this will be difficult for AI systems to do, so they have to be able to mediate between conflicting human preferences.

Author’s statement–Finding solution to the AI control problem may be the essential task of our age.  The key question is not how to maximize AI decision making, but how to maximize good AI decision making.  AI can make maximize the decisions it’s programmed to make, but preferences need to be built to make those maximized decisions good decisions.

-Another author (I’m not good about keeping track) talks about how AI will lead to the reemergence of analog devices.  In the early days of computing, analog devices were based on vacuum tube technology, then the digital computing changed all that.  Now, according to the essay, analog systems will reemerge by creating order and control out of digital systems.  At first, I didn’t understand what that meant, but his examples kind of clarified.  Take Google Maps, it’s built on a system where no one element has control.  It’s thousands (millions) of GPS data points dispersed across all users into one aggregate view of traffic.  Initially the Google traffic map provides a view of traffic, but pretty soon, because of the data in the Google traffic map itself begins to control the flow of traffic, not from the top down but from the bottom up, even though no one is actually in control.  The idea here is that analog systems will take digital and data inputs and make inferences that generally control our behavior.  Author gives an example about the human nervous system–it’s an analog system that receives digital neurological inputs and generally adapts to its environment as a result.  Would be interesting to read about neural networks, etc. in this context.

-3 dimensionality of the human brain has structural advantages than 2 dimensional lithography of existing circuits

-Quote/idea from the book “Odd John”–human beings as the noble creators of advanced species (e.g., advanced AI).

-Human understanding of the Universe is when the Universe became self-aware.  AI enables the Universe to become more self aware.  Meaning in the Universe comes from its ability to understand itself.  This is an interesting concept–why is it important to learn?  Because it gives purpose to the Universe.

-Idea of bounded optimality – finding the end state or executed operation that’s is the best result given system constraints

-Idea of value alignment– making AI goals in line with human goals–avoiding end result of a “universe full of paper clips”

-Value alignment in more detail–human/AI interaction.  Human behavior will be shaped by what AI does, and vice versa.  Value alignment needs to take into account feedback and adaptability of AI system and human reactions.

-Idea of gradient descent and how mosquito finds its target.  This was interesting, the essay was written by a former editor chief at Wired and founder of a drone company.  So, mosquitos use gradient descent to find their host.  They initially detect a human scent (about 30 feet away), and then flutter around randomly until the scent becomes stronger.  When it does, they travel in a single direction, following the scent gradient.  When it gets weaker, the go back to traveling randomly. Thus, mosquitos get to their target by following the scent gradient.  This seems like a random story, but this has implication for how software programs may interact with the real world in AI systems and neural networks.  If you apply a probabilistic technique to determine the likelihood of outcomes, and each time an outcome occurs, you strengthen the algorithmic pathway that a program used to get to that result, eventually you create program gradients, where certain program paths (or ways of thinking) become stronger than others.  This is how the software can “learn” or evolve.  Positive nodal connections can be rewarded to build deeper algorithmic pathways.

-The “deep” in “deep learning” does not refer to something that means “profound”.  Instead the word “deep” refers to the depth of math layers used by programs to make predictions about the world or the problems they are solving.

-Claude Shannon and information theory–interesting theory from Claude Shannon, father of information theory, that information acts like entropy . See this Wikipedia link

-Human body has only 20 amino acids from which all physical construction is derived.