# GPT

- Page: https://shared.btbytes.com/p/gpt
- Date: 2023-02-09
- Tags: llm, deep-learning, gpt

GPT = Generative Pre-trained Transformer

- [GPT in 60 Lines of NumPy | Jay Mody](https://jaykmody.com/blog/gpt-from-scratch/)
- [How GPT3 Works - Visualizations and Animations – Jay Alammar – Visualizing machine learning one concept at a time.](https://jalammar.github.io/how-gpt3-works-visualizations-animations/)
- [Transformers from Scratch](https://e2eml.school/transformers.html)
- [The GPT-3 Architecture, on a Napkin](https://dugas.ch/artificial_curiosity/GPT_architecture.html); [HN](https://news.ycombinator.com/item?id=33942597)
- [GPT-3 - Wikipedia](https://en.wikipedia.org/wiki/GPT-3)
- [A GPT in 60 Lines of NumPy](https://jaykmody.com/blog/gpt-from-scratch/)
- [Let's build GPT: from scratch, in code, spelled out. - YouTube](https://www.youtube.com/watch?v=kCc8FmEb1nY) by Andrej Karpathy.
- [mlx-gpt2: gpt-2 from scratch in mlx](https://github.com/btbytes/mlx-gpt2) code from  [GPT from scratch with MLX](https://towardsdatascience.com/gpt-from-scratch-with-mlx-acf2defda30e)
- [karpathy/minGPT: A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training](https://github.com/karpathy/minGPT)
- [TheR1D/shell_gpt: A command-line interface (CLI) productivity tool powered by OpenAI's text-davinci-003 model, will help you accomplish your tasks faster and more efficiently.](https://github.com/TheR1D/shell_gpt); Get API keys [here](https://platform.openai.com/account/api-keys).
- [Simply explained: how does GPT work? | Confused bit](https://confusedbit.dev/posts/how_does_gpt_work/)
- [GPT4 should be part of your toolkit • Buttondown](https://buttondown.email/hillelwayne/archive/gpt4-should-be-part-of-your-toolkit/)
- [imartinez/privateGPT: Interact privately with your documents using the power of GPT, 100% privately, no data leaks](https://github.com/imartinez/privateGPT) -- a really promising
-  [Understanding GPT tokenizers](https://simonwillison.net/2023/Jun/8/gpt-tokenizers/)
- [[2106.06981] Thinking Like Transformers](https://arxiv.org/abs/2106.06981) ([Weiss et al. 2021](https://arxiv.org/abs/2106.06981)) and explanation blog post - [Thinking Like Transformers](https://srush.github.io/raspy/) by [Alexander Rush](https://rush-nlp.com/)


See: [AutoGPT](https://g.co/gemini/share/9176c3f1119e)

## References

- Weiss, Goldberg, Yahav (2021). [Thinking Like Transformers](https://arxiv.org/abs/2106.06981).