There is a LOT more than five senses, and most of them are definitely not a small amount of data.

Dear god how I'd love to see that ran through the SUMMON code. What eldritch furnishings are they walking over? Stumbling upon??

I know, but the jitter combined with the colorless subsurface-scattered model make this look like straight up David Lewandowski videos

Love how natural and human those movements seem.

I imagine it's important when you're theorycrafting about whether a novel architecture will be able to propagate gradients in a way that might facilitate learning things, but yeah for the most part it seems about intuition and copying successful approaches more than anything.

One day somebody will implement git glue that stitches code with LLMs and you'll never type any other command again.

Let me just post this as an example: https://www.youtube.com/watch?v=kBFMsY5ZP0o

A model trained to detect humans through walls using bouncing wifi signals, based on teacher-student training.

Even though all the original data has been generated by humans, I feel pretty confident in claiming it probably outperforms us at estimating body position from wifi signals.

This obviously transfers to other domains as well- there's certain types of features our brains are wired to pick up(vision, sound, proprioception), some types it is capable of learning(written language, math), and plenty it isn't though to be fair we're pretty bad at most of our learnable tasks- we read and calculate ridiculously slowly, and our ability to break that work down into chunks and memorize intermediate results is the only thing that saves us.

Compared to that, a DNN is a tabula rasa ready to learn how to extract arbitrary features independent of how our cognition of them works, without having to jump through repurposing brain networks to decipher non-native stimuli the way our brains do.

Interesting! I thought about something similar, a "no parameter is left unused" during training, but using unused regions for fine-tuning sounds like a much more clever application of the principle.

But is it the most efficient and effective method?

I'd imagine it's likely possible to converge much faster, and that at some point into training, you likely run into a "limit" where the "signal"(learnable features) can no longer overcome the "noise"(the "pull effect").

What a fascinating paper- reminds me of an idea I had to store some sort of secondary value in weights that contribute to correct outputs that prevents unlearning their features, but had no specific idea of how to execute it- can't believe I didn't think of what this paper's authors did. Thank you.

Similar but not identical? What effect do you mean?

But yeah, the way I see it, the network isn't navigating a single gradient towards "a good classifier" optima, but rather down whatever gradient is left after the otherwise-destructive inference of gradients of individual training examples, as opposed to a more "purposeful" extraction of features.

Which happens to result in a gradual movement towards being a decent classifier, but it strictly relies on balanced, large, and well-crafted datasets to balance the "pull vectors" out to "zero" so the convergence effect dominates, as well as incredibly high training costs.

I don't know how it would look, but surely a more "cooperative" learning process would learn faster if not better.

I assume this is the case early into training, but eventually the training process starts needing to "compress" information so a given parameter handles more than one very specific case, at which point it'll be subject to this phenomenon again- any dog example will want "not dog" neurons inactive, any dog example will want neurons contributing to classification of other classes inactive.

Sure, statistically you're still descending down the slope of a network that's good at each class, but this is only the case when your classes - and thus the "pull effects" are balanced, not as an intrinsic ability of the network to extract differentiating features.

Are there any articles or papers benchmarking this, or exploring more elaborate solutions than just batching?

I never really had hypoglycemia after an actual meal(I know you can get the adrenergic subset of hypoglycemia symptoms without objective hypoglycemia, but it doesn't seem to fit the bill), and my fatigue seems independent of glucose levels- sometimes it hits while they're still high, sometimes after they go back down, sometimes minimal symptoms start before it reaches the peak. I get hours of horrid brain fog, restlessness, anxiety, overwhelming desire to lie down, spend 45-90min in a food coma, and be groggy for several hours afterwards. It gets lighter in the evening, and it's bad enough that I'm afraid of eating during the day. But I'll have to force myself since science suggests that skipping breakfast and one meal a day worsen glucose tolerance, which tracks with my symptoms getting worse after I switched to this.

I'm trying to limit my carb intake, but it's very difficult. For one I'm vegetarian, for two, between ADHD, Sleep Disordered Breathing, and the part of fatigue I now presume to by dysglycemia related, I'm an extremely low functioning person and taking care of myself stretches my capacity very thin.

A lot of the time it's a choice between having some bread and going hungry, and I've been going hungry for a longer time now, which is also bad for my energy levels.

Glucose receptors, or transporters?

How does cytosolic signal impedance develop, here?

Oh, sorry. I'm trying to build a working model of this whole thing, but have been struggling with substantial brain fog that in retrospect has roughly correlated with my fasting insulin and postprandial fatigue getting worse over the past few years...

Early in the year had a HOMA-IR(glucose, insulin) of 0.51(94mg/dl, 2.2 uIU/ml), more recently 1.2(105mg/dl, 4.6uIU/ml). OGTT at both timepoints showed a glucose drop below fasting, with reasonable insulin numbers, which I suspect is rapid gastric emptying that sometimes occurs in prediabetes/T2DM, and is often associated with largely the same set of postprandial fatigue/brainfog symptoms I get(going by Sigstad’s score criteria). Glucose peak likely occurred before 1h mark. On a glucose meter, my fasting glucose is consistently ~105, which very easily jumps to 150-170 even with a single sandwich or some lowish-glycemic-index buckwheat, and to 180-220 with what I'd consider "a normal meal".

I thought that impaired glucose tolerance was a consequence of impaired first-phase insulin secretion, which in turn strongly correlates with fasting insulin levels, which I thought was a function of insulin resistance... I don't know, I just hope so badly that treating this stuff will fix the brain fog that's been getting worse and worse for years now :(

About apnea, it's likely apnea->dysglycemia- glucose metabolism follows a diurnal cycle susceptible to disruption even by sleep restriction, and sleep disordered breathing as a spectrum causes abnormal autonomic tone, while much less of a case can be made for dysglycemia->apnea. Maybe damage to upper airway dilator muscle innervation, airway edema, or tongue base fat deposition... though most likely each of these would require anatomically compromised airway just like most cases do.

I've only had it checked twice, but curiously enough I don't have hyperinsulinemia, but still have elevated fasting glucose and huge spikes after food(on OGTT had a drop below fasting values instead). I'm also on the verge of being underweight, what I struggle with is persistent lack of appetite rather than weight gain.

I'm also a young person with "treated" sleep apnea. Quotes because my impression is CPAP et al largely just convert apneas into subtler microarousals that have milder physiological consequences but comparable effects on sleep quality.

About T2DM, my impression from reading literature has been that while full blown diabetes are fairly similar to each other, earlier dysglycemia has a number of phenotypes that only start to converge after beta cells begin failing.

If it's the repair/failure, then would muscle hypertrophy occur after intentional cellular stress like for example happens with fat when subjected to prolonged suction requiring angiogenesis?

Is protein necessary for exercise's benefits in improving glycemic control as well?

What is "cytosolic signal impedance"? This is the first time I have come across this term.

Oh, I understand, so the impairments in beta cell function are a secondary, later, consequence of insulin insensitivity, which typically is at first driven by... what exactly?

Aside from uptake by muscle, what else plays a role here?

Does the amount/strength of muscles also come into play, or is it mainly about their glucose turnover and participation or lack thereof in glucose utilization?

What about hepatic and brain insulin resistance?

Where does glucotoxicity come into all this? Does avoiding glucose spikes alone help alleviate insulin resistance, or do you need utilization specifically?

What happens to pancreatic beta cells when you change your lifestyle and/or take diabetes medication that limits glucose spikes?

Is this what "peripheral insulin sensitivity"(term I've come across in literature) refers to? How does it differ from pancreatic insulin sensitivity?

Do I understand you correctly that in this sense, above-basal levels of insulin are the body's second line mechanism against supraphysiological elevations of glucose?

I wonder if this is why Pioglitazone acts as an insulin sensitizer, being an agonist of PPAR-gamma, one of the receptors involved in mediating the benefits of physical activity.

Hi, thanks for your answer, but I've already done some reading of related literature, and was looking for a more technical explanation of what happens on a cellular level. Pathways, etc.