Seems like the author had a conclusion in mind (“Write an article about how LLMs could be bad because they might misinform people”) and then tried to find the most sensational way to frame that conclusion.

Bing cited the source of the incorrect claim, so you could independently verify it, it doesn’t consistently seem to make the claim (presumably just due to how the technology works), and the claim isn’t even something I can’t imagine a human also thinking, if they had just Googled and skimmed the topic for keywords without much prior understanding of it.

This just seems like an updated variation of the “Wikipedia isn’t a good source” claim from the early 2000s. Like, it’s still largely a true claim, Wikipedia has lots of wrong information in it that seems very factual, but it’s also a very good tool for reference, if you use common sense and have some prior understanding of the subjects involved (and also have some good heuristics about which pages are likely to be more factual and updated than others).

Seems similar with LLMs. You have to have some prior intuition about what they are good at and bad at in order to make them more useful. Idiots are always going to find a way to hurt themselves with tools.

You're better off just following the "e/acc" part of Twitter, if what you're looking for is well-informed takes and good vibes.

This place has already started the slide toward the kind of depressing, poorly-informed, equilibrium reached in /r/Futurology and /r/technology.

> The agreement won't slow them much.

It absolutely will. They've been trying since 2013 to develop a cutting-edge indigenous semiconductor industry (the "Big Fund"), under fewer constraints, and haven't succeeded at anything but burning a lot of cash, by focusing on many of the "wrong" things (less fundamental science, chemical supply chain, and manufacturing equipment, more spent on value-add at far later stages of the manufacturing process, which has not helped them progress toward become self-reliant, but further deepened their reliance on the same players from Japan, Netherlands, the US, etc). This basically comes down to the fact that Chinese firms are extremely good at figuring out how to position themselves to take advantage of well-funded political priorities, like semiconductors were, and less good at.. uh.. doing the thing they say they're going to do. Many, many words have written about this subject, especially after the Gao Songtao was "disappeared" in a CCDI crackdown in 2021, and the government's financial commitment to the fund began wavering in January.

More constraints will certainly slow them down further, which is why the US did this. Frankly, if the US wanted to hurt them even more in this area, they absolutely could, and there have been numerous good suggestions for how they could severely restrict their access to even 5nm, 7nm and 14nm processes. I suspect the reason they haven't is more down to the fact that the US doesn't want to make this an "existential" issue for China, at least at this point.

My point is, it's not like theses sanctions flipped a switch, and now they'll start trying industrial espionage, or start "really for srs" trying to stop importing 90% of the value-add of their semiconductors. They've been stealing shit forever, it doesn't really put a dent in the fact that they just don't have the domestic expertise necessary to do this, or the economic or political environment to make it an achievable goal for them, even if the government makes it reeeeeally super clear to everyone that it is.

It's the same with lots of complex, high-tolerance electronics. If the US government woke up tomorrow and seriously set about the business of preventing China from accessing avionics, I doubt China would be able to build a modern airplane by themselves.

> Why can't large scale expensive AI models work when the organisations reinvest the net money they earn?

In order for it to be nonprofit, it can't have shareholders.

There's three good answers I can think of as to why it makes sense to have shareholders (and why "being for-profit" is good):

• Typically, no investor-funded tech company "earns anything" for the first 10+ years of its existence. The way it generally works is that they produce an MVP/idea, demonstrate PMF, and then pitch it to investors. They spend the money they raise, and all their revenue (if any), trying to massively grow the company, and when they run out of that money, they go raise another round, at a higher valuation justified by their growth, essentially until they IPO or get acquired by one of the other, larger, tech companies. These businesses are almost never "self sustaining", because the logic is that you're forgoing growth by not spending every available dollar to grow, and they're principally valued on growth. The way investors in prior rounds "make money" is by selling to investors in later rounds (or simply by making the money "on paper", by marking up their books to the value of the new round). The companies can, in theory, "become self-sustaining" at any time, but in practice, rarely do, until they're absolute behemoths. (Think "The Social Network".) If you believe the thing you do has impact on the world, and you believe that impact is positive, (and if money is very, very cheap,) then it makes sense to spend other peoples' money to maximize the impact.

• You imagine that "whoever owns the company" is essentially some big investors, VCs, wealthy angel investors, etc. and it is. But it's also founders, employees and operators. These people often prefer getting paid some of what they earn in "ownership" (equity) over "salary", because it offers them the opportunity for a liquidity event that will compensate them more than anyone will ever agree to salary them for. This makes sense, because salary is a recurring cost, that the company has to budget for in perpetuity, and buying someone's ownership of a valuable thing is a one-off event. It's hard to make $50mm in salary, it's "easy" (relatively speaking) to make $50mm by owning 0.5% of business valued, by someone, at $10bb. People value that opportunity to potentially make life-changing money, when they know they're doing great work on a world-class product that they believe in. It's like the lottery, but you can control the odds. It motivates people to work very, very hard, and it's a very valuable carrot to be able to offer someone, that is "free" to use for the company from a cashflow perspective, aligns incentives between employer and employee, and is matched in magnitude to the performance of the company and its ability to pay it (since they don't pay it, investors do).

• Profitability is a good yardstick to ensure something is sustainable long-term because it's impartial, it's directly related to sustainability (producing more than you consume means you're generating excess value for someone else), and it forces people to make hard decisions, since it aligns incentives toward sustainability and away from sentimentality. The economy operates in credit cycles. It's a company's job to be able to navigate these cycles, and survive the deleveraging part of the cycle. Part of its ability to do this can stem from its ability to access capital markets to generate liquidity when it needs to. It's harder and much more expensive to borrow money if you don't have equity value. It's also very easy to spend the excess generated during the leveraging part of the credit cycle, and mistake it for durable "growth" (just look at the budgets of any government).

That doesn't seem entirely fair.

What they discovered is that LLMs are extremely capital intensive, and you can only tap investors for money (and attract top-talent as an employer) for so long before they expect some kind of return on their investment, so it's either "make substantive progress" or "operate as a non-profit", but it can't be both for very long, or you eventually become unproductive and lose to an organization that has a profit-center, like Google.

So now they've found a way to continue their work by partnering with a company (Microsoft), where that company has access to a bunch of capital necessary to build better models, and a bunch of ideas about how to commercialize OpenAI's existing progress, by integrating it into their own product stack.

It's an amazing deal for both sides, seemingly, because Microsoft takes money out of its left pocket to give to OpenAI, and OpenAI puts most of it right back into Microsoft's right pocket, by renting their Azure services, which simultaneously improves the economics of that business unit, and also likely gives them amazing insight into how to be a world-class service-provider for SOTA "AI companies", in terms of hardware and software needs and optimization.

Similarly, OpenAI gives Microsoft some ownership, but they're so confident they can make them all of their money back that, if they do, they get the equity "back", which they can use to incentivize world-class engineers and academics to keep building. Since they're confident about their ability to make progress, they just get to make that progress "for free", without giving up much of anything to do it.

Luckily for OpenAI and other non-conglomerated AI startups, in the last few decades, we created a world where renting computing resources is a mature, commodified business, with a bunch of massive companies competing to drive the prices to the bare minimum.

The layoffs have nothing to do with AI, they’re just pruning their headcount in response to the last few quarters of financial results and projections about the next few years.

They want to be prudent about how they manage headcount to preserve their share price, because a large part of SWE compensation is equity. If the share-based compensation packages are worth substantially less, they lose headcount anyway, but instead of losing people they think are potentially redundant or replaceable on their own terms, they might lose top performers to other companies when they can’t afford to.

This is one of those "be careful what you wish for" kinds of posts.

I'll settle for narrow oracle AIs that perform seemingly mundane, but transformative, work in physics, mathematics, materials science, biotechnology, etc.

I'm not particularly interested in AGI, and I frankly hope it's 5-10 years away, so we don't have to grapple with alignment problems before the majority are even aware of the existential risks posed by those problems.