[N] EleutherAI has formed a non-profit

Submitted by StellaAthena t3_11g4a9p in MachineLearning

Over the past two and a half years, EleutherAI has grown from a group of hackers on Discord to a thriving open science research community. Today, we are excited to announce the next step in our evolution: the formation of a non-profit research institute.

This will enable us to do much more, and we look forward to building a world class research group for public good! This organization will be lead by long-time contributors to EleutherAI: Stella Biderman (me) as Executive Director and Head of Research, Curtis Huebner as Head of Alignment, and Shiv Purohit as Head of Engineering.

The world has changed quite a lot since we first got started. When EleutherAI was founded, the largest open source GPT-3-style language model in the world had 1.5B parameters. GPT-3 itself was not available for researchers to study without special access from OpenAI, and most NLP researchers had a very minimal understanding of the engineering undertaking required to train such models or their capabilities & limitations. We started as a ragtag group nobody had heard of, and within a year had released the largest OSS GPT-3-style model in the world.

As access to LLMs has increased, our research has shifted to focus more on interpretability, alignment, ethics, and evaluation of AIs. We look forward to continuing to grow and adapt to the needs of researchers and the public

Check out our latest work at www.eleuther.ai or come hang out in our research lab at www.discord.gg/eleutherai

Huge shout out to the donors who have made our work possible: Stability AI, Hugging Face, CoreWeave, Nat Friedman, Lambda Labs, and Canva

167

Comments

You must log in or register to comment.

keepthepace t1_janzb1v wrote

Congratulations! The world desperately needs what you are doing! Was thinking about joining a while ago but got distracted by image-oriented research.

> As access to LLMs has increased, our research has shifted to focus more on interpretability, alignment, ethics, and evaluation of AIs.

Does this mean EleutherAI is not working anymore on big language models?

39

StellaAthena OP t1_jao8e46 wrote

No it does not. In the past we felt that the best way to achieve our goals was to focus almost exclusively on training large models though, and we no longer feel that’s the case.

24

EricHallahan t1_jap0tic wrote

To clarify: EleutherAI will continue to work with large language models and train its own when there is a clear research case as it always has—there just happens to be a much larger saturation of suitable models today for the research we would like to conduct than what existed even twelve months ago, and there is no reason to reinvent something when something suitable already exists. Expect new models to be designed and trained to specifically meet certain research requirements, rather than more versatile usage.

12

currentscurrents t1_jao0a1x wrote

Congrats! Can't wait until you get your first $10-billion investment from a major tech company.

15

Fuehnix t1_japuycf wrote

Here's hoping they never become ClosedAI 🥂

13

WarAndGeese t1_jaqz31a wrote

Why? That would be the end of it. If your comment was sarcastic then pardon my overreaction.

1

currentscurrents t1_jat9lvg wrote

It's a joke. OpenAI was supposed to be a nonprofit too, now they look more like a Microsoft subsidiary.

2

WarAndGeese t1_jb0rsum wrote

My mistake, it is a funny and good joke I just overreacted. I see too many non-ironic statements like that and it clouded my vision.

1

WarAndGeese t1_jaqz7b1 wrote

These things need to be free and open source, not have some profit motive to them. As soon as that day comes means interest in the project is lost and people will look for some other 'free' or 'eleuther' project.

1

starlistener t1_jaojzal wrote

Congratulations on the initiative! Is there a way for people willing to help with the research as entry-level collaborators/volunteers? I am just starting my steps with ML, and I certainly don't have much to add but I'd love to get involved in an open-research initiative and help somehow!

3

StellaAthena OP t1_jaom9of wrote

Definitely! Come check out our discord server and introduce yourself.

8

Fuehnix t1_japumtw wrote

The discord seems intimidatingly huge with 20k+ members, and 3000 online...

Is it really feasible to collaborate and communicate with the group?

I have a B.S. in CS+Linguistics from UIUC, but I had some life and financial complications that blocked me from grad school. I sorted those things out recently, but now I'm trying to find people to do NLP research with so I can be competitive when I apply for Fall 2024 in December.

I'm somewhere in between a senior CS student and first year grad student right now probably.

2

EricHallahan t1_jar9qm1 wrote

Yes, come on in! We do not expect contributors to devote massive amounts of time or go out of their way to contribute—they all have lives of their own, and we respect that.

As for collaboration, we make it work. Most communication is asynchronous text, which is quite versatile and hides varied schedules well.

2

xEdwin23x t1_jaq2p2q wrote

They have a list of projects and / or ideas pinned to some of their channels. If you want something to happen then you're expected to be pro-active and lead (or follow someone else who is leading); it's the only way this kind of collaboration can work. Tbf it's very hard to collaborate among people on different time zones with their own schedules but they somehow make it work.

1

badabummbadabing t1_jar3uab wrote

Going forward, under which licences are you going to release your code/weights/data?

2

EricHallahan t1_jarcwg0 wrote

I do not expect anything other than Apache 2.0 or MIT (our go-to licenses) for code and checkpoints. Data is a bit harder, but we will always try to release everything under the most permissive license we can.

2