You’re thinking of the humans-doing-the-computation concept as a reductio ad absurdum, but have you even an order-of-magnitude idea of just how long it would take for humans to simulate an AGI? If you had a coherent sect of humans spending thousands of years doing rituals they couldn’t possibly understand, yet those rituals resulted in (very slow!) intelligent predictions…

Fair point: conscious things not made of evolved meat are still hypothetical, as a far as we know. We don’t yet know what the secret sauce is.

There are definitely learned behaviours which are passed on socially, and some populations have them and some don’t. An example here in Australia is swooping magpies: in many parts they become homicidal in the nesting season, whereas where I live they are calm around humans all year round.

Being really fast is not the same as being sufficiently complex, though. “Complex, in the right way” is important.