> which supposedly require some amount of creative reasoning.

The dont which is exactly has been part of the complaints of teachers in regards to standardized testing

The structure of pytorch-lightning makes sense for like 99% of workflows which is why other similar libraries use similar structures for what the generic DL process is like. If you are having trouble migrating you are either in a rare use case or really should think about how you structured your code

Its more environmentally friendly to have a car driving around on the hopes of going on a trip

I currently am sadly tethered to living in the present

Especially given the compute used on OpenAI end. This isn’t sustainable

>The subcontractors for that autonomous F-16 fighter from the news last month are not underpaid, nor are the Palantir guys making the software used to target who autonomous drones hit, nor are the ML models guiding real-estate investment corporations that bought a quarter of all homes this year.

You are equivocating the profits of the corporations vs the wages of the workers. Also you are equivocating "Investment Banking" with "Retail Banking" the person making lending models isnt getting the same TC as someone at Two Sigma.

None of those places (retail banking, defense) you list are the highest comp employers. The may be massively profitable but that doesnt necessarily translates to wages.

> Ones that unfairly send certain people to jail, ones that re-enforce unfair lending practices, ones that will target the wrong people even more aggressively than humans target the wrong people today.

Those examples are what I was alluding to with maybe a little too much hyperbole with saying “interns”. The most senior or best people are absolutely not building those models. Those models are being built by contractors who are subcontracting that work out which means its being built by people who are not getting paid well ie not senior or experienced folks.

Those jobs aren’t exciting and arent being rewarded financially by the market and I understand that I am not personally helping the situation but I am not going to take a huge paycut to work on those problems especially when that paycut would be at my expense for the benefit of contractors who have been historically scummy.

I worry about a lot of bad AI/ML made by interns making decisions that have huge impact like in the justice system, real estate ect.

Its always been there. This sub because of the sheer numbers game is flooded by non practitioners. It used to be worse because in the past OP would have been downvoted to hell

>You mentioned MP3 (compressed versions) as comparable in functionality,

Facepalm. For the identity part not the whole thing.

>That's beside the point,

It does for the comment thread which was about copyright

> my point is that the MP3 compression comparison doesn't work,

It does for the part that is actually the point (copyright law).

> They can't recreate other songs that has never been created or thought of.

AFAIK having a not copyrighting violating use doesnt excuse a copyright violating use.

> That's pretty much what's going on here.

No its not. We wouldn’t need training sets if that was the case like in the scenario described where you can generate the dataset using a known algo

Thats a bad argument . MP3s are compressed versions for the original file for many songs so the original isn’t exactly in the MP3 until the decompression is applied. Would anybody argue that since a transformation is applied in the form of a decompression algo that Napster was actually in the clear legally

>Is this really that surprising?

It should be to all the people who claim these models are solely transformative in all the threads about the court cases related to generative model.

I am starting to suspect our illicited mass climate change might kill alll the fish before we could fish them to extinction

Club play is higher than international play because clubs can just dump money and form the best teams irrespective of nationality. Problem is they have to form "best teams" not "best players" and those 2 may not be perfectly aligned.

> Intent and context are not purely symbolic.

Yup . Thats why reasoning comes in and what makes what Demis from DeepMind said make sense

You are 100% right. However people will do like DALL-E and make a budget mickey mouse version and pretend its the exact same thing without measuring any quantitative metrics between the original implementation and theirs.

Yup. The training techniques have got a lot better since that first GPT-3 paper.

This is boring. I am still waiting on those details.

No hand wavy shit, explain with examples showing its impressively longer especially since your position is some magical shit not in the paper/blog is happening.

> The 822 number is irrelevant (given that OpenAI itself tells us that the window is much longer)

OpenAI says the "cache" is '3000 words (or 4000 tokens)". I dont see anything about the input being that. The test case the poster in the twitter thread with spanish is indicative of input being the lower bound which also aligns with what the base GPT3.5 model in the paper has. The other stress test was trivial

https://help.openai.com/en/articles/6787051-does-chatgpt-remember-what-happened-earlier-in-the-conversation

> ...the whole twitter thread, and my direct link to OpenAI, are about the upper bound.

Details. No hand wavy shit, explain with examples why its longer especially since your position is some magical shit not in the paper/blog is happening.

Dude the freaking logs on chrome show OpenAI concats the prompts.

>You then proceeded to pull a tweet within that thread which was entirely irrelevant

your exact words. Try standing by them

> (other than being a lower bound).

A lower bound is relevant its basic math. Freaking proofs are devoted to setting lower bounds

I am still waiting on any proof of any extraordinary for a GPT3 type model memory . Since it is extremely relevant for explaining something ,is to know it exist in the first place

It probably does some basic checks for adversarial text like putting AAAAAAAAA*, BBBBBBBBBBBBB*, [[[[[[[[*, or profanity profanity profanity then preprocesses the text before inputting.

EDIT: Only mentioning since some folks will argue chatGPT has a long crazy memory (10K tokens) because you sandwich stuff around some trivial 9.5k tokens of repetitions. They likely added a bunch of defenses against different basic prompt engineering attacks so people dont get it to say certain things too.

You brought that source into the thread and now are claiming the discussion in that thread is off topic?

You still havent shown proof that the context window is crazy long for a GPT model. I hope that test case in the thread with a bunch of AAAA's isnt your evidence.