Viewing a single comment thread. View all comments

neuralbeans OP t1_j6n0ima wrote

Reply to comment by chatterbox272 in Best practice for capping a softmax by neuralbeans

I want the output to remain a proper distribution.

1

chatterbox272 t1_j6n3vx6 wrote

My proposed function does that. Let's say you have two outputs, and don't want either to go below 0.25. Your minimum value already adds up to 0.5, so you rescale the softmax to add up to 0.5 as well, giving you a sum of 1 and a valid distribution.

2