Introduction Agents are increasingly upon us. Although opposition is rare, \intelligent agents" have been attacked for user interface problems, and on larger social issues. Agent supporters have countered these arguments and raised doubts about alternative technologies. We place this in historical, social, and ethical contexts, noting the cyclic nature of such debates. One conclusion is that many problems with arti cial agents arise from a poor understanding of social aspects of human agents. 2. Historical Perspectives The history of technology has seen many movements call for human-like systems, and use anthropomorphic terminology to generate understanding and support. Such movements often make excessive claims, perhaps misled by their own rhetoric or their (sometimes impressive) partial success. This raises unrealistic expectations, which often leads to disappointment, which is surprisingly often followed by rebirth with similar goals, and somewhat improved terminology and techno

Whereas humans can know only about 200 other people well, machines will be able to know billions. This will define their higher level of consciousness and the threat they pose to humans. Trying to constrain the behavior of such machines by laws is like trying to define their behavior by a fixed set of expert system rules: it won’t work. Rather, our focus should be the emotional values used to reinforce their learning of behavior. Their behaviors should be positively reinforced by happy humans and negatively reinforced by unhappy humans. Consciousness is a simulator for solving the temporal credit assignment problem in reinforcement learning, in the sense that consciousness enables brains to process experiences that are not actually occurring, Where time intervals between behaviors and rewards may be long and unpredictable, time intervals between simulations of those events can be short and predictable and thus amenable to the brain’s known mechanisms for solving the temporal credit assignment problem. The higher level consciousness of machines will simulate human social interactions in detail, in order to learn complex behaviors for coping with conflicts among humans and other ambiguities of human happiness.