There was an interesting bit of research that qwen has a single neuron that makes it “demonic evil qwen” and a simple “Who are you?” will get some absolutely terrifyingly vile results.
Anyways, what I’m trying to say is I want evil qwen to be my CPU.
The single-neuron steering stuff has been going around twitter lately, do you happen to know who was the first one to do it?
I wonder how this squares with the recent "NNs are already clean" interpretability finding, which may have also been the impetus for this kind of research. Similar to their work on CNNs, I guess maybe there really is a single neuron that corresponds to "evilness" and you can just turn it up/down?
It could be interesting to experiment with more complex representations in the memory/file system, i.e. vectors. Seems like the full advantage of this architecture is yet to be explored. Blending neurons with deterministic compute to fit on one GPU is a fascination concept, never considered it before!
> When Qwen 'saves' a file, the compiled layers hash its name into one of 1,024 slots in a directory kept in the DeltaNet state, and store the text itself in other heads, with eight tokens per row, chained together
I’ve never wanted to learn how LLMs work more than reading this. Fascinating.
Does divagr get paid to do wild things with LLMs? Just curious if they’re being compensated for these wild experiments
Along these lines I’m curious how a scratch trained model with a few built-in tools (memory, calculator, ?) available during a single generation would behave. Not tool calls as turns.
a single generation would still allow looping in the "CPU" because the instruction set can do it. The model is almost a Turing machine (except for the limit on memory)
This is cool and I understand the RAM emulation with gated attention but how does the CPU emulation wor? You add layers that are then trained to predict outputs while interpreting RISC instructions?
This reminds me of C++ Template Metaprogramming (TMP) (it's Turing-complete so people wrote "games" that run at compile time). The same with TypeScript types, I think someone got DOOM running entirely in the TypeScript type system. An interesting tech demo but entirely superfluous.
It is a great example to refute the common misconception that LLMs are inherently non-deterministic. I see people repeating this a lot and being able to point at a CPU implemented within one is a very strong counter-example.
> Because a CPU inside a language model clearly wasn't cursed enough, I decided the prudent thing would be to give it a filesystem too.
Love the spirit :)
Anyways, what I’m trying to say is I want evil qwen to be my CPU.
I wonder how this squares with the recent "NNs are already clean" interpretability finding, which may have also been the impetus for this kind of research. Similar to their work on CNNs, I guess maybe there really is a single neuron that corresponds to "evilness" and you can just turn it up/down?
I’ve never wanted to learn how LLMs work more than reading this. Fascinating.
Does divagr get paid to do wild things with LLMs? Just curious if they’re being compensated for these wild experiments
This reminds me of C++ Template Metaprogramming (TMP) (it's Turing-complete so people wrote "games" that run at compile time). The same with TypeScript types, I think someone got DOOM running entirely in the TypeScript type system. An interesting tech demo but entirely superfluous.
We have gone down this path already in [0] and this is the same thing but overly complicated. This reason is "because LLMs".
Why?
[0] https://arxiv.org/abs/1410.5401