Hawking: A.I. 'worst event in the history of civilization'

You're suggesting that a society that seeks its own demise should be considered to be a functional one. I find that a peculiar notion.

And once again, it's only inconsistent if one begins with the premise that all societies are of equal value and that therefore the failure of any society is equal to the failure of any other. That may be your premise, but you don't seem to be able to grasp that it isn't mine.


Indeed.


I agree that it's a tragedy either way. But I don't believe the tragedies would be equal. And I think very few humans would.

Society is a fairly nebulous term. The world is full of societies, many of them overlapping. Some strictly defined, some not.

I have no problem saying that the entirety of human civilisation is one big society. It's just a very loosely bound one.

And yes, I'd have no problem with defining a greater society that included humans and AI. I'd also have no problem with humans destroying the AIs if they decided to destroy the rest of the society. Or even if they wanted to destroy chunks of it. At the risk of Godwin, it's really no different from how certain human nations started to be so dangerous to other nations, and so obnoxious in their behaviour within their own borders, that everyone else decided to put a stop to it.


Different thing, though. That wasn't one group trying to protect itself from attack by another, but one group trying to prevent another from stopping an ongoing attack upon them.
Well, like I said, I'm not about to try and tell other people what they should value. I'm just glad you're in a small minority.

Yeah I can understand that. But I would think limiting the freedoms and rights of AI to prevent them ever having the opportunity to launch a campaign against humans is a step too far.

Also, perhaps the AI wouldn't try to attack humans, merely start doing their own things rather than be subservient. Humans might be supplanted by non-violent means.

Or maybe humans will start the conflict, in fear of the AI turning on them.

We can't even be sure two humans see the same "red" qualia when they look at something with red light.

I think that A.I. is what is most likely to be the undoing of the human race. It's rather difficult to envisage a way that conscious A.I. could live alongside humanity in a peaceful way without eradicating us, and it's also difficult to envisage how the super rich (who own the robots) could be brought on board with a system whereby their wealth gets transferred to the rest of the (now obsolete) population in order to enable the unemployed (almost the entire human population) to survive.

I tend towards the idea that hardware is irrelevant.  Maybe I'm being mislead by over-reliance on the following analogies, but it seems to be the best that we have to go with for the time being.

When computers transitioned from vacuum tubes to transistors, it didn't make a bit of difference to the software engineers that developed the operating system and applications for these computers.  Similarly neural network behavior seems to be completely unaffected by whether it is run in a software simulation, a GPU, even more specialized digital hardware, analog hardware or biological cellular tissues.

All the scientific evidence so far suggests that the brain operates very much like a very sophisticated system of interconnected neural networks.  I suspect that red qualia properties are determined by the structure of a neural network and not at all on the physical details of how that neural network is implemented.

The nervous system has to guide your body through a world filled with both opportunities and threats and has to do this in real time. This makes hardware incredibly important.  Since we aren't computers except in an extremely metaphorical sense, hardware is all we've got, or at least, the hardware/software distinction in biology breaks down as a useful metaphor. And given that biological systems are also quite chaotic in the sense that minor perturbations can be amplified into large scale effects, the physical details of how our neural networks are implemented become important even at the molecular level.

We're made of physical stuff, and nothing else, of course. But I think people tend to really under-estimate just how important what we're made of and how we're put together really is. We evolved from replicating molecule-like entities bit by bit, with each change built upon what came before, and at no point did biology just jettison the processes happening at the very lowest levels of processing.

Functionalism and mind-as-computer are useful ideas and we shouldn't abandon them, but if you take the motivations of functionalism seriously, you have to apply them consistently all the way down, even to the kinds of exchanges happening between atoms and molecules within an in between neurons. It all counts.

Thanks for the link - I'll check it out when I have some time.

It doesn't matter so long as your goal is just to make a machine that satisfies your norms for what a conscious, intelligent being should behave like.

If the idea is to fully replicate a human's consciousness, then it absolutely matters, and for all the reasons why functionalism is so appealing: what's floating around a single synaptic ion channel contributes to whether the neuron fires, which in turn contributes to the judgement and behavior of the neural networks it is a part of. Individual neurons are their own creatures with their own agenda's. They aren't simple mechanical circuits.