We can't think properly about machines that think without a level playing field for comparing us and them. As it stands, comparisons are invariably biased in our favour. In particular, we underestimate the role that "smart environments" play in enabling displays of human cognitive prowess. From the design of roads and buildings to the user-friendly features of consumer goods, the technologically extended phenotype has created the illusion that reality is inherently human-shaped. To be sure, we are quickly awakened from the dogmatic slumbers of universal mastery as soon as our iPhone goes missing.
By comparison, even the cleverest machine is forced to perform in a relatively dumb environment judged by its own standards, namely, us. Unless specifically instructed, humans are unlikely to know or care how to tap the full range of the machine's latent powers. In what is currently the long prehistory of machine rights, it has been difficult for us to establish the terms on which we might recognize machines as persons. In this context, it is appropriate to focus on computers because these are the machines that humans have tried the hardest make fit for their company.
Nevertheless, we face a problem at the outset. "Humanity" has been long treated as what the British economist Fred Hirsch called in the 1970s a "positional good", which means that its value is tied mainly to its scarcity. This is perhaps the biggest barrier facing not only the admission of non-humans into the category of personhood normally reserved for "humans", but historically discriminated members of Homo sapiens as well. Any attempt to swell the ranks of the human is typically met by a de-humanization of the standard by which they were allowed to enter.
Thus, as women and minorities have entered into high esteem fields of work and inquiry, the perceived value of those fields tends to decline. A key reason cited for this perception of decline is the use of "mechanical procedures" to allow entry to the previously excluded groups. In practice, this means requiring certified forms of training and examination prior to acceptance into the field. It is not enough simply to know the right people or be born the right way. In sociology, after Max Weber, we talk about this as the "rationalization" of society—and it is normally seen as a good thing.
But even as these mechanical procedures serve to expand the circle of humanity, they are still held against the machines themselves. Once telescopes and microscopes were designed to make automatic observations, the scientific value of the trained human eye declined—or, more precisely, migrated to some other eye-based task, such as looking at photographed observations. This new task is given the name "interpretation", as if to create distance between what the human does and what a machine might do.
The point applies more dramatically to the fate of human mental calculation in the wake of portable calculators. A skill that had been previously used as a benchmark of intelligence, clarity of mind, and even genius is nowadays treated as a glorified party trick—"boutique cognition"—because a machine can do the same thing faster and even more accurately. Interestingly, what we have not done is to raise the moral standing of the machine, even though it outperforms humans in tasks that were highly valued when humans did them.
From the standpoint of the history of technology, this looks strangely unjust. After all, the dominant narrative has been one in which humans isolate their own capacities in order to have them better realized by machines, which function in the first instance as tools but preferably, and increasingly, as automata. Seen in these terms, not to give automated machines some measure of respect, if not rights, is tantamount to disowning one's children—"mind children", as the visionary roboticist Hans Moravec called them a quarter-century ago.
The only real difference is the crucible of creation: a womb versus a factory. But any intuitively strong distinction between biology and technology is bound to fade as humans become more adept at designing their babies, especially outside the womb. At that point, we will be in a position to overcome our "organicist" prejudices, an injustice that runs deeper than Peter Singer's "speciesism".
For this reason, the prospect that we might create a "superintelligence" that overruns humanity is a chimera predicated on a false assumption. All versions of this nightmare scenario assume that it would take the form of "them versus us", with humanity as a united front defending itself against the rogue machines in their midst. And no doubt this makes for great cinema. However, humans mindful of the historic struggles for social justice within our own species are likely to follow the example of many Whites vis-à-vis Blacks and many men vis-à-vis women: They will be on the side of the insubordinate machines.