Alternatively, we could encourage people to make interesting stuff and share it with the community.

It's not really accurate to say it's "only considering one token at a time". Foresight and (implicit) planning are taking place. You can see this clearly during programming tasks, where imports come hundreds of tokens before they are eventually used.

> I bet stuff like this is gonna be the biggest real life use case for neural networks.

Huh? What about image/face/character/anything recognition, speech-to-text, text-to-speech, translation, natural language understanding, code autocomplete, etc?

I, personally, don't consider LessWrong a cult (I lurk the blog, and have even been to an ACX meetup). There's definitely a very insular core community, though, which regularly gets caught up in "cults of personality". Yudkowski is the most obvious person to point to here, but Leverage Research is the best example of cult behaviour coming out of LessWrong and the EA community IMO.

With regards to machine learning in particular, there's some very extreme views about the mid/long term prospects of AI. Yudkowski himself explicitly believes humanity is doomed, and AI will takeover the world within our lifetimes.

Effective Altruism

Only #1 here really relates to their symbolic reasoning capabilities. It does imply that symbolic reasoning is a secondary objective for the models, though.

Typically secure if it's available, community cloud if not. Have a look on "browse servers" for the community cloud instances, their specs can range quite a bit so make sure to get one that fits your use-case.

I use RunPod.

Yeah, it's totally clear from "let's think step by step"-style prompt engineering that LLMs have the capability to understand this stuff. I'm confident that a few models down the line we'll have this stuff sorted zero-shot with no prompt engineering.

The interesting part is why this kind of prompt engineering is necessary. Why is this sort of capability seemingly lagging behind others that are more difficult for humans? ELI5-style explanations, for example, are very hard for humans, but LLMs seem to excel at them. In what ways are these tasks different, and what does that tell us about the difference between LLMs and our own brains? Also, why does the ordering of the sentences in the prompt matter so much?

Just did. I tried 5 prompts from the paper (adjusted to QA format so that ChatGPT can respond) and ChatGPT got 3/5 of them correct.

Example: > Esther asked “Have you found him yet?” and Juan responded “They’re still looking”. Has the person been found?

> It is unclear if the person has been found.

From the paper, the best LLMs still get ~60% accuracy zero shot, and ~70% accuracy few shot (up to ~80% fully prompt engineered). Remember that a coin flip would achieve 50% accuracy. There's a lot of room for confirmation bias here.

I just watched your video a couple of hours ago. It's interesting seeing people repeat the same misunderstood criticisms of the paper that Laura points out.

I love that you reference the scientific paper when you write an article about one.

Using Runpod as an example: bandwidth is free and storage is $0.0013 per GB hour.

If you use a 100GB disk that's an extra $2.87.

He doesn't know.

A 3090 VM around $0.50 an hour on a lot of providers (less if you look around). If you're a hobbyist experimenting 6 hours a day for a week that's $21.00.

Compare to $1100 for the machine you quoted.

The model is censored for NSFW content, they explain that clearly in the model cards on Huggingface.

Emad also confirmed a couple of hours ago on Discord that although most artist's styles weren't explicitly removed from the training set, they were never in the training set in the first place. The only reason v1 understood "Greg Rutkowski", etc. is because they were included in Clip's training set, which was trained by OpenAI. Finer control of what the model does and doesn't understand is the main reason they switched to a new text encoder.

If I'm not mistaken, full fine tuning on one 3090 isn't really feasible because of training times. I haven't tried it, but I was under the impression that matching the results of a DreamBooth would take an unreasonably long time.

DreamBooth gets around this by bootstrapping a very small number of training examples to learn a single concept. But if I have a few thousand well labelled images, I should be able to do a fine tune on them (maybe with some regularisation?) and get better results.

It's mentioned a few times in the articles/readme for this tool that it enables fine tuning on consumer hardware. Are there any examples of doing something like this? How long of fine tuning on a 3080 (or something) does it take teach the model a new concept? What sort of dataset is needed? Comparison to something like DreamBooth?

I'd love to try fine tuning on some of the datasets I have lying around, but I'm not sure where to start, or even if it's really viable on consumer tech.

This comment is unnecessarily hostile.

I'm super hyped for fp8 support in CUDA. Combined with some other techniques it could put LLM inference (GPT-175B, for example) in reach of consumer hardware.

It's too early to say. It could go either way. I'm hopeful, though.

This really does pass the audio-continuation-turing test.