New research introduces MemoryLLM, a plug-n-play module enhancing transformer interpretability. This work decouples feed-forward networks (FFNs) from self-attention, enabling the study of FFNs as context-free, token-wise retrieval memories and investigating their importance across various tasks.
Opening Kapyn…