treedmt
treedmt t1_j29yope wrote
Reply to comment by blueSGL in OpenAI might have shot themselves in the foot with ChatGPT by Kaarssteun
LUCI is also built on a fine tuned gpt3.5 model, so pretty close to chatgpt in terms of capabilities.
They have a very different monetisation model afaik. They are tokenising the promise of future revenue to monetise, instead of charging customers up front.
> if the training data is worth less than the inference cost.
The thesis is that training data could be worth much more than inference cost, if it is high quality, unique, and targeted to one format (eg. problem:solution or question:answer)
In fact, I believe they’re rolling out “ask-to-earn” very shortly, which will reward users for asking high quality questions and rating the answers, in Luci credits. The focus appears to be solely on accumulating a massive high quality QA database, which will have far more value in the future.
I’m not aware of any rate limits yet but naturally they may be applied to prevent spam etc., however keeping the base model free is core to their data collection strategy.
treedmt t1_j29v2zd wrote
Reply to comment by blueSGL in OpenAI might have shot themselves in the foot with ChatGPT by Kaarssteun
LUCI for one. Not exactly chat format but generative single turn question answering. Http://askluci.tech/QA
treedmt t1_j29ty59 wrote
Reply to comment by blueSGL in OpenAI might have shot themselves in the foot with ChatGPT by Kaarssteun
That would be awesome for the free competitors though.
treedmt t1_j28o94z wrote
Reply to comment by blueSGL in ChatGPT Could End Open Research in Deep Learning, Says Ex-Google Employee by lambolifeofficial
Surely there’s some trade off between qualitative vs quantitative data?
Eg. 50 billion high quality QA pairs may beat 500B random google queries as training data.
treedmt t1_j219ah9 wrote
Reply to comment by visarga in ChatGPT Could End Open Research in Deep Learning, Says Ex-Google Employee by lambolifeofficial
Could better, larger datasets be solution to the hallucination problem? Ref chinchilla for example- but maybe even an order of magnitude bigger than that?
treedmt t1_j21925t wrote
Reply to comment by icest0 in ChatGPT Could End Open Research in Deep Learning, Says Ex-Google Employee by lambolifeofficial
Open source code could still win, if the secret sauce lies in a massive closed source dataset.
treedmt t1_j218w1d wrote
Reply to comment by blueSGL in ChatGPT Could End Open Research in Deep Learning, Says Ex-Google Employee by lambolifeofficial
This is an interesting point. Do you think the dataset that google has is high quality enough to actually train ai? In particular, search queries etc aren’t mapped to specific answers to be useful for supervised learning. Maybe I’m missing something?
treedmt t1_j4cjkpw wrote
Reply to Don't add "moral bloatware" to GPT-4. by SpinRed
Thoughts on anthropic AIs “constitutional AI” approach? At least they explicitly note the helpfulness vs harmlessness trade off curve and actively try to maximise both: unlike OpenAI, which just wants to make it harmless, even at the cost of helpfulness.