Blog

Feb 12, 2023

We Found An Neuron in GPT-2

Posted by in categories: ethics, law, neuroscience

I notice that the token in question happens to be segmented as “_an” and “_a” and not “_an_” or “_a_”.

So continuations like [_a, moral,_fruit] or [_an, tagonist, ic,_monster, s] could be possible (assuming those are all legal tokens).

I am reminded of the wonderful little nuggest in linguistics, where people are supposed to have said something like “a narange” (because that kind of fruit came from the spanish province of “naranja”). The details on these claims are often not well documented.

Comments are closed.