It's just difficult to wrangle all of the dependencies; I want to be able to wrap an entire model in a complete isolated black box that I can call into with a C API or similar.

That is, I'd like something like https://github.com/ggerganov/llama.cpp/blob/master/llama.h without having to rewrite the entire model.

For my use cases, native would be good, but web would be a nice to have. (With enough magic, a native solution could be potentially compiled to WebAssembly?)

Deploying anything developed with Python to a end-user's machine

ReAct

Changing the video player you're using to watch a movie doesn't make the movie any less copyrighted; the same kind of mechanics would apply here.

As the other commenter said, it's unlikely anyone will advertise a service like this as LLaMA's license terms don't allow for it. In your situation, I'd just rent a cloud GPU server (Lambda Labs etc) and test the models you care about. It'll only end up being a dollar or two if you're quick with your use.

I invite you to compare the GPT summary and dotpoints in the article and to tell me they are the same

Agreed. It also complicates productionising the model if you're reliant on features that are only available in the Python interface. Of course, there are ways around that (like just rewriting the relevant bits), but it's still unfortunate.

Oh no, someone used a state of the art language model to summarise some text instead of doing it themselves. However will we live with this incalculable slight against norms of discussion on Reddit?

The usual answer to this is "multimodal" and I think that's still true, especially with recent advances. We'll see in the next few months :)

As far as I can tell, the sparse documentation is just because they've been in pure R&D mode. I've played around with it in their Discord server and can confirm it does perform well, but I've struggled to get it working locally.

You are probably fine, but note that a) people will likely be very angry with you, whether or not the licensing permits it and b) this is a non-trivial problem and even more non-trivial to train.

Good luck, though!

The issue is less with the platform they're using and more with where they're sourcing the data from. They're asking if there are any issues with taking people's uploaded builds and using them to train a generative system.

There's information about this in the README, but I'll admit that it's a little too technical and doesn't have a high-level description of the ideas. Looking forward to the paper!

https://papers.labml.ai/papers/weekly

Eh, I still watch the videos but it's clear that the audience has shifted. They're often skint on details and come out a while after the initial release of a paper. They're great for showing people not in the field what's going on, though :)

Awesome! I'll add it to my reading list :)

Honestly, I'm not convinced it needs a hugely complex language model, as (to me) it seems like a primarily classification task, and not one that would need a deep level of understanding. It'd be a level or two above standard spam filters, maybe?

The two primary NN-in-web solutions I'm aware of are tf.js and ONNX Runtime Web, both of which do CPU inference, but the latter is developing some GPU inference. As you say, it only needs to be done once, so having a button that scans through the transcript and classifies sentence probabilities as sponsor-read or not, and then automatically selects the boundaries of the probabilities seems readily doable. Even if it takes some noticeable amount of time for the user, it's pretty quickly amortised across the entire viewing population.

The only real concern I'd have at that point is... is it worth it for the average user over just hitting the right arrow two times and/or manually submitting the timestamps themselves? I suspect that's why it hasn't been done yet

I've thought about this and it seems doable (especially with the availability of both YouTube transcripts and Whisper), but the cost of training would be quite tedious for a hobbyist. Am excited to see if anyone tackles it, though.

For this, I'd infer on the client (especially if you train on the YouTube transcript, so that you don't need to run Whisper over the audio track). Of course, it's much harder to make it a paid product then 😅

> we are just predicting if the image is of a cat or a dog...

And there's no way automated detection of specific traits could be weaponised, right?

I generally agree that it may be too early for regulation, but that doesn't mean you can abdicate moral responsibility altogether. One should consider the societal impacts of their work. There's a reason why Joseph Redmon quit ML.

Awesome, thanks! I'll give it a try sometime :)

Not sure if these are supported already (just skimmed over the website and the list you've put here), but these are things I've wanted for while reading papers:

• Dark mode. Viewing white PDFs before bed is unpleasant.
• A way to view two-column papers as a single column (putting the right column below the left column?) so that I don't have to keep moving around

Those are probably the two things that bother me the most. In terms of exciting future work and keeping with the theme, you could also consider doing some automatic summarisation of text / papers (especially for citations), but that's much less necessary.

please do not post while high