Submitted by Emergency_Apricot_77 t3_zmd6l8 in MachineLearning
dojoteef t1_j0ayqqq wrote
See the graphs in the paper that introduced nucleus sampling: The Curious Case of Neural Text Degeneration. They visualize how human authored text has different statistical properties from machine generated text. That's mainly a tradeoff between fluency and coherence. Sampling procedures like top-k or nucleus sampling restrict the tokens that can be emitted and thus introduce statistical bias in the generated text, but produce more fluent text. Rather, sampling from the full distribution gets closer to the distribution of human-authored text, but often degenerates into incoherence (hence the title of the paper).
Emergency_Apricot_77 OP t1_j0c3cii wrote
This is VERY similar to what I was looking for. Thanks a LOT for this
Viewing a single comment thread. View all comments