gpt-2 : add batched decoding example (#572)

gpt-2 : add batched decoding example (#572)

* Initial attempt to make gpt2 do parallel decoding

* Fix crash on trying to use empty embd

* Make it work for n_parallel=1

* Add short way of passing n_parallel argument

* Move gpt-2 batched to a separate target and cpp file

* Add batched sample output to README and remove hardcoded model path and prompt

* gpt-2-batched : fix n_kv heuristic

* Free batch at end of example

* gpt-2-batched : simplify kv cache stuff (#574)

ggml-ci

* Fix not generating n_predict tokens and fix warn

* minor : readme

* Add check for end token and mark the stream as finished

---------

Co-authored-by: Georgi Gerganov <redacted>
Co-authored-by: YavorGIvanov <redacted>