nice example.

it definitely does seem like "contextualization" is one of the biggest limiters on gpt performance.

https://thakkarparth007.github.io/copilot-explorer/posts/copilot-internals

you might enjoy this copilot reverse engineering in a similar vein. if i had enough time i would probably port some of these techniques to emacs (can use copilot there but looking at extensions dont quite do all this i dont think, tho it does work well enough with just the buffer)