I guess this kinda puts the argument "how will the AGI/ASI interact with the world" to bed as a reason not to be concerned about alignment (which seems to be en vogue at the moment. )

How has this narrative sprung up so quickly and spread so widely.

https://en.wikipedia.org/wiki/Open_Letter_on_Artificial_Intelligence

https://futureoflife.org/open-letter/ai-open-letter/

Back in 2015 the same org drafted an open letter and announced potential issues with AI's and that was years before any sort of commercialization effort.

There are alignment researchers who have signed the letter, both times.

Current models cannot be controlled or explained in fine grain enough detail to control (the problem is being worked on by people like Neel Nanda and Chris Olah but it's still very early stages and they need more time and people working on the problem)

The current 'safety' measures are bashing at a near infinite whack-a-mole board whenever it outputs something deemed wrong and it is far from 'safe'

>a lot of them look like randoms so far.

...

>Population

>We contacted approximately 4271 researchers who published at the conferences NeurIPS or ICML in 2021.

I mean just exactly who do you want to tell you these things. I can pull quotes from people at OpenAI saying they are worried what might be coming in future.

> There's consideration from the people working on these machines.

https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/

>In 2022, over 700 top academics and researchers behind the leading artificial intelligence companies were asked in a survey about future A.I. risk. Half of those surveyed stated that there was a 10 percent or greater chance of human extinction (or similarly permanent and severe disempowerment) from future A.I. systems.

If half the engineers that designed a plane were telling you there is a 10% chance it'll drop out of the sky, would you ride it?

edit: as for the people from the survey:

> Population

> We contacted approximately 4271 researchers who published at the conferences NeurIPS or ICML in 2021.

Was Elon Musk planning this back in 2014 too?

Playing the long game?

https://en.wikipedia.org/wiki/Open_Letter_on_Artificial_Intelligence

https://futureoflife.org/open-letter/ai-open-letter/

what will ever happen? Interpretability? it's being worked on right now, there are already some interesting results. It's just an early field that will need time and money and researches put into it. Alignment as a whole needs more time money and researchers.

> 3. it will be initially unaligned

if we had:

1. a provable mathematical solve for alignment...

2. the ability to directly reach into the shogoths brain, watch it thinking, know what it's thinking and prevent eventualities that people consider negative outputs...

...that worked 100% on existing models. I'd be a lot happier about our chances right now.

As in the fact that the current models cannot be controlled or explained in fine grain enough detail (the problem is being worked on but it's still very early stages) what makes you think making larger models will make them easier to analyze or control.

The current 'safety' measures are bashing at a near infinite whack-a-mole board whenever it outputs something deemed wrong.

As has been shown. OpenAI has not found all the ways in which to coax out negative outputs. The internet contains far more people than OpenAI's alignment researches, and those internet denizens will be more driven to find flaws.

Basically until the AI 'brain' can be exposed and interpreted and safety check added at that level we have no way of preventing some clever sod working out a way to break the safety protocols imposed on the surface level.

> "growth at all cost"

so cancer.

Anyone coming here never having seen the comedy stylings of Lily Savage, watch this:

https://www.youtube.com/watch?v=_RPpicb6ArY

> AGI, Theory of Mind, Creativity

Marvin Minsky classified words such as these as “suitcase words”. As in a word into which people attribute (or pack) multiple meanings.

These words are almost like thought terminating cliches, as in when they are spoken it assures the derailment of the conversation. Further comments will be arguing about what to put in the suitcase rather than the initial point of discussion.

> with specialized expert data from literally 50 experts in various fields that worked on the response quality in their domain.

Sounds like a future goal for Open Assistant.

If one were being unethical... create a bot to post the current Open Assistant answers to technical questions in small specialist subreddits and wait for Cunningham's_Law to come into effect. (I'm only half joking)

>So with GPT-2 medium, what we really do here is to parent a dumb kid, instead of a "supernaturally precocious child" like GPT-3. What interested me is that RLHF does actually help to parent this dumb kid to be more socially acceptable.

> In other words, if we discover the power of alignment and RLHF earlier, we might foresee the ChatGPT moment much earlier when GPT-2 is out in 2019.

That just reads to me as capability overhang. If there is "one simple trick" to make the model "behave" what's to say there that this is the only one. (or that the capabilities derived from the current behavior modification are the 'best they can be') Scary thought.

After refusing to say how many parameters GPT4 has, refusing to give over any details of training dataset or methodology and doing so in the name of staying 'competitive' I'm taking the stance that they are going to do everything in their power to obfuscate the size of the model and how much it costs to run.

e.g. Sam Altman has said in the past that the model would be a lot smaller than people expect and that more data can be crammed into smaller models. (Chinchilla and especially the very recent Llama papers prove this)

Would I put it past the new 'competitive' profit driven OpenAI to rate limit a GPT4 that is actually similar in size to GPT3 to give the impression the model is bigger and takes more compute to generate answers? No (as the difference in inference cost is pure profit)

https://github.com/ggerganov/llama.cpp [CPU loading with comparatively low memory requirements (LLaMA 7b running on phones and Raspberry Pi 4) - no fancy front end yet]

https://github.com/oobabooga/text-generation-webui [GPU loading with a nice front end with multiple chat and memory options]

/r/LocalLLaMA

/r/LocalLLaMA

What are your thoughts on Microsoft Office 365 Copilot ?

depends on the age of the cow I suppose.

how much of that food can be sown, grown, kept. and harvested without the aid of machinery, fertilizers or anything else reliant on just-in-time trade infrastructure.

> I'm dreading the day AI can write code.

Self fixing code generation is already in the pipeline for simple programs. (that was the middle of last year. ): https://www.youtube.com/watch?v=_3MBQm7GFIM&t=260s @ 4.20

GPT4 can do some impressive things:

>"Not only have I asked GPT-4 to implement a functional Flappy Bird, but I also asked it to train an AI to learn how to play. In one minute, it implemented a DQN algorithm that started training on the first try."

https://twitter.com/DotCSV/status/1635991167614459904

a scrip dubbed "Wolverine" that hooks into GPT4 and recursivly resolves errors in python scripts.

https://twitter.com/bio_bootloader/status/1636880208304431104

Another step closer to the "Infinite Simpsons Generator"

Have you seen that Microsoft are directly integrating it into their office suite under the banner of "Office 365 Copilot"?

Here are some timestamped links to the presentation.

Auto Writing Personal Stuff: @ 10.12

Business document generation > Powerpoint : @ 15.04

Control Excel using natural language: @ 17.57

Auto Email writing w/document references in Outlook: @ 19.33

Auto Summaries and recaps of Teams meeting: @ 23.34

Exactly.

I'm just eager to see what fine tunes are going to be made on LLaMA now, and how model merging effects them. The combination of those two techniques has lead to some crazy advancements in the Stable Diffusion world. No idea if merging will work with LLMs as it does for diffusion models. (has anyone even tried yet?)

Is it possible to split the model and do inference across multiple lower VRAM GPUs or does a single card have to have the minimum 16gig VRAM?

The 'pro gamer move' That seems to be less meme-y by the day.