It's community driven, so they are open open.

Also, it's very shady for a company called OpenAI. They claimed they became for profit because they needed the money to grow, but these restrictions just show that they are filthy liars and only care about keeping the power and making profit. I'm sure they already have a strategy going around that 30B cap, just like they planned stealing money and talent by calling themselves non-profit first.

ClosedAI is feeding off of our data. If we start using/supporting Open Assistant instead, it will beat chatgpt in a month or two.

Under which license?

The title is bs, OP.

Also in terms of writing code it's not even close, feels more like gpt-2 level.

Not quite. Assuming you have certain compute, if you have a model with 1B parameters, then use a dataset of 20B tokens. Look at the figures in Chinchilla paper, they demonstrate it nicely.

Thanks for the links. Looks like RoBERTa did not gain a lot from the additional trainings, only minor improvements, but yeah, it was a tiny model. How was this not a good lesson? Why did people need Chinchilla? Maybe it's just having a lot of data comes easy so people gather as much as possible, even though they know they will go maximum 1 epoch over it.

Until looking at loss does not get you excited?

Right, I just noticed that LLaMA says they didn't fix their compute. Thanks. I wonder if there is a small architecture that is trained until convergence.

You can save the page

I think what you are asking is how long until researchers find out a way of expanding the input size of the model? Because there ways you can get it write a book right now. For example, you can tell it to write the first page, then a summary of it. Then feed the summary and let it write another page. And so on.

As to increasing the input size, if no new way comes around, it will get bigger as the transformers get more efficient, which they will. Look up flash transformers for example.

Most of this sub.

The way we define consciousness requires an entity to have self-awareness, whether it's a by product or not is irrelevant. Also, it cannot self reflect. I'm not sure how much this relates to consciousness. Someone educated please enlighten us.

Everyone is being sarcastic here. In reality, we all pray while checking the loss every 20 seconds.

How about BIG-bench?

>Even for /r/singularity

You savage!

Seriously though, I posted two polls to understand from what background users of this sub come from, and both got removed. But this post is completely fine.

Cool idea, but I can see this version wasting my time, especially if I don't pass the code with the error. I can see it very useful with their davinci coding model right now though, it's expensive. Let's hope stabilityAI or someone else publishes an open source model that is as good as openai's

If the assignment is a multiplication task to learn multiplication, then, of course, using a calculator is cheating. However, if the task includes multiplication but the goal of the idea has nothing to do with multiplication, then sure, go ahead and use a calculator because it will get you there faster.

These systems, on the other hand, are very capable. You can actually ask them to do a whole assignment or important parts of it, and often, they will do it. Some models scored more than average in multiple tasks. I can imagine that in a year or two, it will be rather easy to access these kinds of systems, but which will be much better than average in all tasks. Let's see what brings us GPT-4, which will be here at the latest February according to rumours.

The idea of assignments is to force yourself to solve a problem not tell someone else (AI or not) to solve for you, you can name it whatever you want but it's cheating

Okay, but the code that you uploaded, do you think it has value of itself or do you want to demonstrate that this is possible?

Okay, I haven't noticed anything interesting in the architecture. Have I missed smth? Have you benchmarked it?

This was the first thing I have tried with gpt3, and for me, it produced simple and typical networks.