>I don't see why you're taking an extreme stance like that. Nobody said there wasn't any concern

Well when you say things like this:

>You're making a lot of false assumptions. AGI or ASI won't do anything on its own unless we give it the ability to, because it will have no inherent desires outside of the ones it has been programmed with.

You are already dismissing one of the largest concerns many alignment researchers have. I appreciate that the movie version of an AI run amok is distasteful, and maybe not even the likeliest way that a powerful AI can be an existential threat, but it's just confusing how you can tell people that they are making a lot of assumptions about the future of AI, and then so readily say that a future unknown model will never have any agency, which is a huge concern that people are spending a lot of time trying to understand.

Demis Hassabis, for example, regularly talks about it. He thinks he would be a large concern if we made a model with agency, and thinks it is possible, but wants us to be really careful and avoid doing so. He's not the only one, there are many researchers who are worried about accidentally giving models agency.

Why are you so confident that we will never do so? How are you so confident?

I hope you talk to the people in your life better than this

I'm basically going to ignore the ad hominem's, but just as a tip - that sort of stuff makes you look worse, not me.

So your argument against the idea that we would replace a significant portion of the infrastructure of the world with automated processes run by AGI is that people would be too bored, so they would want to have what are the equivalent of Jetson's button pressers? I have a few critiques of this argument....

Let me try a more casual one.

So AGI takes over, suddenly all human work is unnecessary. AI does it better, faster, and cheaper than people. Bob though used to run the waste disposal plant in your city. He really wants to keep working that job, so he just.... Walks into this new robot run facility, understands how everything is working even though it's all changed, and now his job is what... Making sure the AI doesn't make a mistake, or take over? Meanwhile his buddies are at the cottage, having a beer and not having to work. You think Bob's work is so satisfying and valuable that this is a tenable situation?

Maybe you can give me an example of how you think this plays out? Do you think Bob is in a position to protect us from malicious AI? Do you think people like Bob exist, or at least enough to have a handle on all important infrastructure? You think Bob wouldn't rather spend time on his woodworking hobby?

>I'm sorry that you cant pick up on peoples intent but that's not my fault.

Maybe you can help me - where in their post does this person say that we won't be doing anything in their free time?

> I'm also sorry that you're confused, yet again, on both points. Please really give some thought to the slow down I mentioned. What will people be doing when theyre not working? Why would they give up being curious about things? Because you're not a curious person?

Before we get there, maybe clarify, what point are you making? This post is about AI "taking over", the person who's comment we're replying to was suggesting how that AI would be able to take over much of our infrastructure and processes.

Are you trying to say that, no that won't happen because people will want to spend their free time... Managing waste treatment facilities, dealing with our food production, working in warehouses and factories? Is that what you are trying to get at?

Hmmm, let me ask you a question.

Do you think the people who work on AI - like the best of the best, researchers, computer scientists, ethicists, etc - do you think that these people are confident that AGI/ASI "won't do anything on it's own unless we give it the ability to"? Like... Do you think they're not worrying about it at all because it's not a real thing to be nervous about?

>No shit chatgpt doesnt give you anything unprompted, you don't see the difference between prompting for a poem and prompting for a poem specifically about taking over mankind?

Where did they say that ChatGPT provided that poem without being given that theme? You're making a lot of assumptions about their intent, but anyone who has used ChatGPT for like... 5 minutes (basically everyone who posts here) would understand that ChatGPT isn't popping out poems like that without any lead up.

> And their point is not that we will relinquish control willingly, their point is we will do that and do nothing in the new free time. That is what I find silly. So it seems you didn't understand either point, maybe slow down on the reply button.

Where are you seeing that? What do you even mean "will do nothing"? I literally have no idea how you are pulling these insinuations from what seems to be a very clear post to me

Well sorry it just seems like it's something odd to be so incredulous about - do you know about the alignment community?

Who is "we"? Do you think there will only be one place where AGI will be made? One company? One country? How do you think people would interact with it?

This problem I'm describing isn't a particularly novel one, and there are really clever potential solutions (one I've heard is to convince the model that it was always in a layered simulation, so any attempt of breaking out would trigger an automatic alarm that would destroy it) - but I'm just surprised you have such confidence.

I'm a very very optimistic person, and I'm hopeful we'll be able to make an aligned AGI that is entirely benevolent, and I don't think people who are worried about this problem are being crazy - why do you seem to look down on people who do? Do you look down on people like https://en.m.wikipedia.org/wiki/Eliezer_Yudkowsky?

Well here's a really contrived example. Let's say that collectively, the entire world decides to not let any AGI on the internet, and to lock it all up in a computer without Ethernet ports.

Someone, in one of these many buildings, decides to talk to the AGI. The AGI hypothetically, thinks that the best way for it to do is job (save humanity) is to break out and take over. So it decides that tricking this person to let it out is justified. Are you confident that it couldn't trick that person to let it out?

Well there's a reason that alignment is a significant issue that has many many smart people terrified. There have been years of intellectual exercises, experiments, and both philosophical and technical efforts to understand the threat of unaligned AGI.

The plot of Ex Machina is a real simple example of one. We know as humans, that we are susceptible to being manipulated with words. We know that there are people who are better at that than average, indicating that it is a skill that can be improved upon. A super intelligence that is not barred from this skill, theoretically, would be able to manipulate its jailors, assuming it was locked up tight.

It's not a guarantee that ASI will want to do anything, but it's not like we have a clear idea of whether or not "qualia" and the like are emergent properties from our models as we scale them up and create more complex and powerful architecture.

The point of this, fundamentally, is that it's not a problem that many people are confident is "solved", or even that we have a clear path to solving it.

>What a silly post. You act like ChatGPT just threw that out there instead of was prompted "write a poem about an ai taking over the world"

Is that what that came off as to you? I feel like everyone here's knows how ChatGPT works... It doesn't provide you anything unprompted.

> Also your logic is not that great because the vast vast vast majority of people don't know how anything works right now anyways. Ill take computers in charge over the 1% 100% of the time.

I don't think I understand your point, but I understand theirs. Their point is that we will relinquish control willingly, because it's better than having us in control in terms of output. What about their point do you find silly?

Should be interesting to see how well this works. I was reading a Twitter thread from someone who worked on some of the progenitor technology to this and they weren't confident it would be anywhere near as accurate as claimed - this will be our first demo

This makes a lot of sense.

A lot of what instruct fine tuning and rlhf is that if you provide some high quality, specifically created data to an LLM while it's being fine tuned, you get a significant jump in results for this fine tuned model - versus just giving them more of the same structured data.

In some of the papers I read, a lot of the conclusions are akin to "next steps is trying to see if more instruction data will improve results".

Some of the challenges with this instruction data is that well we just don't have a lot. We don't have for example... A lot of the recordings of people using computers to complete tasks. Like keystrokes and screen recording.

I don't think this sounds like they are getting "screen" recordings (AdeptAI for example is doing that with their model, but with a browser only for now). It sounds more like just accompanying natural language descriptions with the fine tuned data is enough to get an improvement. Which makes sense from my limited experience with LLMs.

Should be interesting. I imagine this is for fine tuning GPT4. The "Codex 2.0", better base model (GPT), better instruct tuning probably as well.

chatGPT (so take it with many grains of salt)

> The paper is discussing a machine that can create new songs or music. They are testing to see if the machine is able to memorize songs or if it can come up with new ones. They are looking at how well the machine does when given different amounts of information to work with. They found that even when given a lot of information, the machine is not able to create exact copies of songs. However, it can create similar songs. They also found that when the machine is given very little information, the songs it creates are not very diverse. They include examples of the machine's output in the accompanying material.

I just saw this, HOLY CRAP AT THE FIRST EXAMPLE

Just because I don't physically have access to these models, doesn't mean they don't exist. Google regularly works with other institutions when running research with PaLM and their other advancements, and people frequently duplicate their findings.

Additionally, we have access to things like Flan-T5, tiny models fine tuned with their latest work that are about as powerful as gpt3, 5b vs 170b parameters.

I don't think that's the case. Please read the papers, look into the actual research - it sounds like you are like... Mad at Google, but that's a separate consideration than the tech they have. It's unquestionably better than any other LLMs we know about, regardless of how you feel about Google

Not only has Google managed it, they have most likely the best models in the world. PaLM is already the benchmark for basically all LLM tests, and it's even been fine tuned - for example medPaLM recently was shown in a paper that puts its diagnostic skills a hairsbreadth away from marching a clinician.

I think I just assume that everyone already... Knows this, at least in this sub, but Google is far and away the technical leader in this, not even when including DeepMind.

Have you actually seen the models coming out of Google? Read their research papers? There is no question that they are not just pioneers, they literally set the benchmarks

This all sounds too specific to be not true, we'll just have to see how it turns out. It sounds like Google wants to make sure everyone knows they have the best stuff - but I wonder how open their APIs will be

So you think you'll be able to put in a prompt and get a full, 90 minute coherent video with this update? I would bet big money against that - not even considering things like audio.

The models that Stability have experience with and have shown so far would absolutely not be able to handle anything like that.

However if it's something more like... Style transfer on a full video? I would not be surprised.

It's just the current state of the video generating models that exist. First - the best of the best are at Google, and we've seen what they currently can do. Even if Stability has been able to spend the last few months replicating the research out of Google, I can't imagine them being able to create a model that can output more than 1 minute of somewhat coherent video. The current large challenge is the inefficiency of these models, the longer the context the MUCH larger the memory and processing power required.

These are problems I would be very surprised to be solved first anywhere other than Google.

What I imagine is more likely is a sort of StyleGAN system that can be applied on a whole video, with some level of coherence.

And yes, this is maybe the greatest point. If any of this technology can be monetized, people are incentivized to make it cheap enough for having as many customers as possible.

>Yes, but who’s to say the barrier of cost will be taken down? It’s not as easily reproducible as you say, training it takes tons of data and computing power etc.

It's easy enough, we have for example about a dozen open source language models that we can run, and the quality of them improves even as their size decreases (Flan-T5 is a good example). We can also see that with image models - cheaper, faster, and more variety. We already have quite a few different models, not including fine tuned off shoots.

And even for new models today, the costs are measured in the hundreds of thousands to millions. Those costs, while not cheap enough for me to build, are still incredibly cheap. And it will get cheaper to build the same models as techniques improve and hardware improves, there's no reason that trend wouldn't continue.

> For all we know if things like posthumanism etc. become real they might as well just charge an unreasonable price and only the select few will get it, leading to a gattaca-like scenario in real life. These sorts of dystopias are a genuine possibility with the advancements we’re seeing.

But that's a fear based conclusion, not something you are really coming to from an informed place. This isn't how technology has worked so far, and technology has made us all more "powerful", the internet, smartphones, and now these models. Why assume that at some arbitrary point this will suddenly no longer be possible? Why assume that the world is filled with mustache twirling rich villains?