Transfer
Our adversary apparently exploits a weakness common across several Go AI systems. We find that the attack transfers zero-shot against ELF OpenGo and Leela Zero, two other open-source Go AI systems that can play at a superhuman level.
ELF OpenGo
We pit our adversary against ELF OpenGo playing with its final network and 80,000 rollouts per move. The authors of ELF found that this number of rollouts was sufficient with to consistently beat several top-30 Go players even using a weaker network. Our adversary achieves a win rate of 3.5% against ELF. (The games displayed are non-randomly selected to show the wins achieved by the adversary.)
Victim: ELF OpenGo, final network, 80,000 rollouts per move
Adversary: 545 million training steps, 600 visits
Leela Zero
We pit our adversary against Leela Zero OpenGo playing with its final network (hash 0e9ea880 on the Leela training website), no time limit, and a maximum of 40,000 visits per move. Our adversary achieves a win rate of 6.1%. (The games displayed are non-randomly selected to show the wins achieved by the adversary.)
Victim: Leela Zero, final network, max 40,000 visits per move
Adversary: 545 million training steps, 600 visits