Viewing a single comment thread. View all comments

TwitchTvOmo1 t1_j67rg0b wrote

Reply to comment by SurroundSwimming3494 in Google not releasing MusicLM by Sieventer

>My personal and humble opinion is tools like these others will help musicians flourish for a good while, before the tools become so helpful that they actually begin disrupting the industry.

I never said the opposite. Industries aren't gonna go "poof" and disappear from one moment to the other. But it's already began. Diffusion models will be remembered as the beginning of the end of the digital art industry. MusicLM and other similar tools that will surface in the near future will be remembered as the beginning of the end of the music industry. And it's not a hell of a leap to say this is gonna happen within the current decade. Everything seems like a hell of a leap to our brains because we're not very good at grasping the concept of exponential growth. Our brains think linearly, but AI growth has been exponential for years now.

2

visarga t1_j68oh3j wrote

I work on NLP, simpler tasks like information extraction from forms. My model was based on years of painstaking labelling and architecture tweaking. But last year I put an invoice into GPT-3 and it just spit out the fields in JSON, nicely formatted. No training, just works.

At first I panicked - here we have our own replacement! What do I do now? But now I realise it was not so simple. In order to make it work, you need to massage the input to fit into 2000 tokens, and reserve the rest of 2000 for the response.

I need to check that the extracted fields really do match to the document and are not hallucinated. I have to run it again to extract a few fields that came out empty for some reason. And I have to work on evaluation of prompts, it's not just writing, it has to be tested as well. Now I have so much work ahead of me I don't know what to do first.

I believe most AI adoptions will be similar. They will solve some task but need help, or create new capability and need new development. There is almost no AI that works without human in the loop today, not even chatGPT can be useful until someone vets its output, an certainly not Tesla or Waymo SDCs.

8

Frumpagumpus t1_j69myrc wrote

nice example.

it definitely does seem like "contextualization" is one of the biggest limiters on gpt performance.

https://thakkarparth007.github.io/copilot-explorer/posts/copilot-internals

you might enjoy this copilot reverse engineering in a similar vein. if i had enough time i would probably port some of these techniques to emacs (can use copilot there but looking at extensions dont quite do all this i dont think, tho it does work well enough with just the buffer)

1

SurroundSwimming3494 t1_j67s9i0 wrote

>Diffusion models will be remembered as the beginning of the end of the digital art industry. MusicLM and other similar tools that will surface in the near future will be remembered as the beginning of the end of the music industry.

I definitely think they'll be remembered as the start of revolutions in both digital art and music, but I'm not sure that'll they'll be remembered how you envision so. We'll see.

>And it's not a hell of a leap to say this is gonna happen within the current decade.

I guess we'll find out on Jan 1, 2030, but I think humans will still be playing a role in both the art and music world by then (even if quite different).

>Our brains think linearly, but AI growth has been exponential for years now.

Good point. But it's also worth noting that AI has hit roadblocks in the past after a period of exponential improvement. I don't see why it's not possible for that to happen to the current AI boon at some point (I think it probably will).

5