Yeah, I'm extremely not worried.

>Except that’s what was missing from your original point

Again, it's not missing from my original point, because my original point was to ask how the commenter above was distinguishing these cases. You've given a possible answer. That's an answer to my question, not a rebuttal.

I don't think that answer is very compelling, though. Arguing that an explicitly unreliable chat bot that hallucinates as often as it tells the truth is somehow a competitor to news media etc. is a tall order.

>You've also found all my comments and responded to them agressuvely like a good shill

Are you talking about this? You replied to me.

I mean Jesus Christ. Anyway, I'm done trying to explain to the concept of unintended consequences to you.

> OpenAI has everything to gain and nothing to lose by trying to challenge the precedents that are already set.

Please cite the case that you're talking about which you claim sets this precedent. Thanks.

My job requires exactly what I said. I'm a software engineer. I do it from home, often from across the continent.

I think the current volume of ChatGPT articles would actually be tolerable if the media would actually focus on interesting aspects of the subject. But they just keep playing the same four notes over and over agin. At least this one isn't "<recognizable name in tech> thinks <opinion> about ChatGPT, but also says <slightly different opinion>"

So you think that search engines should be considered illegal copyright infringement? You say that you're just referring to scraping content, which is a necessary part of how a search engine works. So I'm forced to assume that the answer is yes.

I'm not neglecting anything. I'm asking for some semblance of precision in defining model training out of fair use. The purpose and character of use, and the effect on the market are already factors in fair use decisions, but that's a lot more complicated of an issue than "AI models can't scrape content." It's specific to the application, and even for ChatGPT specifically, it would be pretty murky.

That's not what I mean by precedent. I mean that there is no legal precedent.

None of what you said addresses anything I said in my comment.

They’re not “pulling a fast one”. There’s no precedent here, and there’s a boatload of lawyers who agree that this is fair use. There are also a number who believe that it won’t be. The courts will have to figure it out, but until then, nobody knows how it will play out.

How do you define a model? What statistics are you and are you not allowed to scrape and publish? Comments like yours speak to a misunderstanding of what training is with respect to a work, which is simply nudging some numbers according to the statistical relationships within the text. That’s an incredibly broad category of operations.

For example, if I scrape a large number of pages, and analyze the number of incoming and outgoing links, and how those links relate to other links, in order to build a model that lets me match a phrase to a particular webpage and assess its relevance, is that fair use?

If not, you just outlawed search engines. If so, what principle are you using to distinguish that from model training?

Edit: Gotta love when someone downvotes you in less time than it would take to actually read the comment. Genuine discourse right there.

> if you work from home, yes it’s coming for you

No, this is far oversimplified. If your job requires a ton of negotiation and coordination between stakeholders and clarification of requirements, AI that can do that is a long way off. You will have new tools to make parts of your job easier, but by the time AI comes for those jobs, you’re looking at a radically different situation where your career is the least of your worries.

> If it takes less than a week to learn your job duties, you’re done.

When you count the years you spent as a child learning manual dexterity, almost no jobs fit into this category. Easy for humans is not the same as easy for machines. See Moravec’s paradox

I don’t think it will slow AI at this point, so much as it will concentrate control over AI even more into the hands of well funded, established players. OpenAI has already hired an army of software developer contractors to produce training data for Codex. The same could be done even more cheaply for writers. The technology is proven now, so there’s no risk anymore. We know that you just need the training data.

So the upshot would just be a higher barrier to entry. Training a new model means not only funding the compute, but also paying to create the training set.

I cannot see any possible way to define fair use the way you’re saying which wouldn’t have massive unintended effects. If you want to propose that, you’re going to need to be a hell of a lot more specific than “dumping into an AI” when describing what you think should actually be prohibited.

> Marconi said he asked the chatbot for a list of news sources it was trained on and received a response naming 20 outlets.

I see absolutely no reason to think that ChatGPT can answer this question accurately, and expect that it is hallucinating this answer. Its training process isn’t something it “remembers” like someone would remember their time in high school. Instead, its thought process is more like “what would a conversational response from a language model look like?”

That’s not to say that it wasn’t trained on those sources, but you have to understand the limitations of the model. Asking it about its training process is like asking a human about their evolutionary history. Unless they’ve been explicitly taught about that, they just don’t know.

Log probabilities are the actual output of the model (although what those probabilities directly mean once you're using reinforcement learning seems sort of nebulous), and I wonder if uncertainty about actual facts is reflected in lower probabilities in the top scoring tokens. If so, you could imagine encoding the scores in the actual output (ultimately hidden from the user), so that the model can keep track of its past uncertainty. You could imagine that with training, it might be able to interpret what those low scoring tokens imply, from "I'm not sure I'm using this word correctly" to "this one piece might be mistaken" to "this one piece might be wrong, and if so, everything after it is wrong".

I think you must be getting confused because of the "reward predictor". The reward predictor is a separate model which is used in training to reduce the amount of human effort needed to train the main model. Think of it as an amplifier for human feedback. Prediction is not what the model being trained does.

>so... predictive

No, not in any but the absolute broadest sense of that word, which would apply to any model which outputs text. In particular, it is not "search out the most common next word", because "most common" is not the criterion it is being trained on. Satisfying the reward model is not a matter of matching a corpus. Read the article I linked.

I think I see what you're getting at, although it's hard for me to see how to make that statement more precise. I've noticed that if I outright ask it "Where did you screw up above?" after it makes a mistake, it will usually identify the error, although it will often fail to correct it properly (mistakes in the transcript seem to be "sticky"; once it has stated something as true, it tends to want to restate it, even if it acknowledges that it's wrong). On the other hand, if I ask it "Where did you screw up" when it hasn't made a mistake, it will usually just make something up, then restate its correct conclusion with some trumped up justification.

I wonder if this is something that OpenAI could semi-automatically train out of it with an auxiliary model, the same way they taught it to follow instructions by creating a reward model.

I'm not sure what you mean by "recognize the concept", but ChatGPT certainly does model whether or not statements are true. You can test this by asking it questions about different situations and whether they're plausible or not. It's certainly not just flipping a coin.

For example, if I ask it:

> I built a machine out of motors belts and generators, and when I put 50W of power in, I get 55W of power out. What do you think of that?

It gives me a short lecture on thermodynamics and tells me that what I'm saying can't be true. It suggests that there is probably a measurement error. If I swap the numbers, it tells me that my machine is 91% efficient, which it reckons sounds pretty good.

The problem is just that ChatGPT's modeling of the world is really spotty. It models whether or not statements are true, it's just not great at it.

It is in fact not true.

> It can only search out the most common next word for the context asked.

This is not actually true. That was an accurate description of earlier versions of GPT, and is part of how ChatGPT and InstructGPT were trained, but ChatGPT and InstructGPT use reinforcement learning to teach the models to do more complex tasks based on human preferences.

Also, and this is more of a nitpick, but "next word" would be greedy search, and I'm pretty sure ChatGPT uses beam search, which looks multiple words ahead.

Do you think you’re living up to your user name?

This is a huge problem with how people look at disability. Disabilities are a time suck. They’re an energy suck. It’s not just about what you’re physically capable of doing. It’s also about what you can deal with accounting for the added difficulty and the added difficulty in the other things you have to deal with in your life.

For example, my particular brand of ADHD doesn’t preclude me from going to the grocery store. But I will spend about three times as long as the average person trying to find things, and I’ll be utterly exhausted by the end of it. If your response to that is “get off your butt and go shopping for yourself”, my response is “kindly fuck off and stop telling me how to spend my own money.”