Community Paper Reading: Explaining Grokking Through Circuit Efficiency

​Join Arize Co-Founder, Jason Lopatecki, and CTO, Michael Schiff, as they discuss “Explaining Grokking Through Circuit Efficiency”. This paper explores novel predictions about grokking, providing significant evidence in favour of its explanation. Most strikingly, the research conducted in this paper demonstrates two novel and surprising behaviors: ungrokking, in which a network regresses from perfect to low test accuracy, and semi-grokking, in which a network shows delayed generalisation to partial rather than perfect test accuracy.