I think you’re almost entirely correct, this is the principal critique I would offer of my own article, and I’ve been trying to shore up this hole in my argument with a few edits, but I’ve discovered that it’s fundamentally a “feeling” on my part that neural nets provide a good framework for understanding our own thinking, which is sort of improvable — for now.
Allow me to address just a few of your points.
The key point I was trying to make by invoking NNs in this discussion of ethics is that brains probably make decisions based on hidden features which are deep feature transformations of our sensory input. There is no simple, rational computation, nor would such an approach be right. I should try to make that more clear, but again, we don’t know whether that’s even true.
On the question of fooling neural networks, two points:
Researchers are working really hard to patch this up (it’s the result of maxpooling, at least in the last couple cases of that article) and I think they will pretty soon.
The main way most neural networks differ from the brain, I suspect, is that the brain has some crazy loopiness to it, with robust feedback mechanisms, whereas traditional neural networks are usually feedforward-only. Much more work to be done in this domain.
Lastly, on the issue of free will, I can’t express how I feel about this question even nearly as well as Douglas Hofstadter does in I Am A Strange Loop’s aptly named chapter “There is no such thing as free will”.
A Raybold, thank you for reading my article. It means a lot to me that someone read it thoroughly enough to understand what I was trying to say and discover my article’s deepest flaw.