Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers
Dai, Damai, Yutao Sun, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, and Furu Wei. 2023. “Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers.” arXiv:2212.10559 [cs.CL]. arxiv.org/abs/2212.10559
BibTeX
@misc{dai2023gptlearnincontextlanguage,
title = {Why Can {{GPT}} Learn In-Context? {{Language}} Models Implicitly Perform Gradient Descent as Meta-Optimizers},
author = {Dai, Damai and Sun, Yutao and Dong, Li and Hao, Yaru and Ma, Shuming and Sui, Zhifang and Wei, Furu},
year = 2023,
eprint = {2212.10559},
primaryclass = {cs.CL},
url = {https://arxiv.org/abs/2212.10559},
archiveprefix = {arXiv}
}