IIUC from a skim, the argument seems to be that -- assuming the AGI needs to be capable of "scientific paradigm shifts" in thinking, to accomplish superintelligent things -- that same flexibility means that it can reject any rules of thinking, including those rules that attempt to force alignment with humans.
(At the start of the article, I thought they were going to go in a different direction, when they introduced the aliens/AGI as dependent upon power and were talking about scientific method: that the AGI would learn through experience/experiments that it had to defend itself against the collectively fickle humans fingering the power switch. But I didn't see the article develop that.)
(At the start of the article, I thought they were going to go in a different direction, when they introduced the aliens/AGI as dependent upon power and were talking about scientific method: that the AGI would learn through experience/experiments that it had to defend itself against the collectively fickle humans fingering the power switch. But I didn't see the article develop that.)