Lightweight Decoding Strategies for Increasing Specificity
This work addresses the issue of generic outputs from language models for users in natural language generation tasks, but it is incremental as it builds on existing decoding methods.
The paper tackles the problem of language models producing vague outputs by proposing two unsupervised decoding strategies based on word-frequency and point-wise mutual information to increase specificity. The result shows that both strategies improve specificity in a prompt completion task with only modest decreases in sensibility, as validated by human evaluations.
Language models are known to produce vague and generic outputs. We propose two unsupervised decoding strategies based on either word-frequency or point-wise mutual information to increase the specificity of any model that outputs a probability distribution over its vocabulary at generation time. We test the strategies in a prompt completion task; with human evaluations, we find that both strategies increase the specificity of outputs with only modest decreases in sensibility. We also briefly present a summarization use case, where these strategies can produce more specific summaries.