From Stephen Wolfram:
We can say: “Look, this particular [neural] net [works]”—and immediately that gives us some sense of “how hard a problem” it is (and, for example, how many neurons or layers might be needed). But at least as of now we don’t have a way to “give a narrative description” of what the network is doing.