Submitted by KD_A t3_127pbst in MachineLearning
planetofthemapes15 t1_jeg1iqc wrote
Cool, I had a mental model very similar to this which I was planning on implementing next week. I'll just try yours and if I make an improvement I'll submit a PR.
KD_A OP t1_jeghvnn wrote
Yeah I was surprised that this wasn't already coded up--it's been 3 years since we've found out that sampling from GPT-3 is a good zero-shot text classifier.
While benchmarking this method on the infamous Winograd Schema Challenge, I ended up finding a 2018 paper^1 w/ pretty much the same idea as CAPPr. The only difference is that CAPPr typically transposes that probability, and it naively incorporates a prior.
- Trinh, Trieu H., and Quoc V. Le. “A simple method for commonsense reasoning.” arXiv preprint arXiv:1806.02847 (2018).
nbviewerbot t1_jeghww5 wrote
I see you've posted a GitHub link to a Jupyter Notebook! GitHub doesn't render large Jupyter Notebooks, so just in case, here is an nbviewer link to the notebook:
https://nbviewer.jupyter.org/url/github.com/kddubey/cappr/blob/main/demos/wsc.ipynb
Want to run the code yourself? Here is a binder link to start your own Jupyter server and try it out!
https://mybinder.org/v2/gh/kddubey/cappr/main?filepath=demos%2Fwsc.ipynb
^(I am a bot.) ^(Feedback) ^(|) ^(GitHub) ^(|) ^(Author)
KD_A OP t1_jeghzff wrote
<3 u
Viewing a single comment thread. View all comments