Blutorangensaft t1_iz9pl46 wrote on December 7, 2022 at 2:52 PM

Reply to comment by ThisIsMyStonerAcount in [D] If you had to pick 10-20 significant papers that summarize the research trajectory of AI from the past 100 years what would they be by versaceblues

I get that, but I find it a little hard to believe that nobody immediately pointed out a nonlinearity would solve the issue. Or is that just hindsight bias, thinking it was easier because of what we know today?

ThisIsMyStonerAcount t1_iza1qzy wrote on December 7, 2022 at 4:16 PM

What nonlinearity would solve the issue? The usual ones we use today certainly wouldn't. Are you thinking a 2nd order polynomial? I'm not sure that's a generally applicable function, with being non-monotonical and all?

(Or do you mean a hidden layer? If so: yeah, that's absolutely hindsight bias).

Blutorangensaft t1_iza9zrt wrote on December 7, 2022 at 5:11 PM

I see. I meant the combination of a nonlinear activation function and another hidden layer. Was curious what people thought, thanks for your comment.

chaosmosis t1_izawfsz wrote on December 7, 2022 at 7:37 PM

Non-monotonic activation functions can allow for single layers to solve xor, but they take forever to converge.