> the textual nature of prompts leads us to take the intentional stance towards systems which aren’t conscious, and thus miss the essential nature of their non-meaning
I see LLMs as being capable of making useful distinctions and having a rich action space. They are widely used because their operation is useful, and that can only happen when semantics work well in practice. But useful things that pay for themselves don't need our "essential nature" blessing, they already have persistence by mutual entanglement with us.
You can't host the internet on your machine, or download a "Google" or "Meta" but you can download a local model. So I don't think AI will lead to more centralization. Besides local models, the interface itself - prompt based steering - is eminently more open ended that apps or websites ever were. Cloud or local, you get more control now as a user.
But you cannot train an open weight model by yourself, and maybe there could some distributed training net ran by thousand of volunteers around the world in the future, but until we get there we will still be tied to BigCo (American, Chinese or wherever in the world) decisions.
You would need institutions to do this? What do we have besides global megacorps with the resources for this? Universities perhaps training models for their local catchment?
We'll probably see a few middle-tier countries like France or Japan training their own national LLMs that they can control. Even if it's expensive, this will eventually be just as critical to national sovereignty as having nuclear weapons. They don't want to be completely dependant on the USA or China.
For decades, people ran word processors and graphics editors locally.
Now it’s in the cloud on a damn webpage.
With recent RAM and storage price disruptions, I feel like we’ll go to go back to an equivalent of everyone having a dumb Wyze terminal with the real “computer” in the cloud… Only difference is the dumb terminal will have color, be wireless, and portable.
I’d like to think that people will move back to native, but sadly there’s only one viable platform for native app dev, Apple. I base that statement on the highly fragmented Linux ecosystem and the fact that Microsoft are appear to be actively hostile to windows users. I leave Android out of the picture because of the continual and legally proven trust breaches of the owner.
It does suck. Though Android’s owner does appear to be afraid of Graphene OS so maybe there’s hope.
Linux is not really fragmented anymore, there is a lot of distros but they're all compatible with each other. Flatpak solved app distribution and systemd standardized the lower levels. If you build something on Ubuntu it will almost definitely work on Fedora and Arch with no code changes. 10 years ago that was impossible.
Yep, this is because for decades Apple has looked for ways strong local clients can add value in the form of bespoke hardware and software capabilities, while Microsoft has looked for ways commoditised generic clients can extract value.
Fragmented doesn't mean you can't make native apps.
GTK, QT, FLTK or TK apps coexist peacefully on linux the same way apps made with Cocoa, JUCE or whatever own internal proprietary toolkit apps run and coexist on MacOS.
This is exactly why I believe LLMs are fundamentally divergent from the trajectory of internet service path we have been on. It allows us to create space and disconnection from an otherwise tethered experience.
It is possible that models are only the byproduct of what frontier labs are doing. In other words, what you call AI is just a log, a trace of what the real AI, for which only frontier labs have access to, did.
> You can't host the internet on your machine, or download a "Google" or "Meta" but you can download a local model.
You can very much host websites on your machine (not the whole internet, but then again you can’t host the most powerful models either) and you can very much download a Wikipedia or Internet Archive.
But most people never will. This idea that everyone will soon be using local models while frolicking happily in the fields is a pipe dream. Besides, local or not, these models can already be harmful in ways the internet took decades to approach.
A local model has more data than Wikipedia and the Internet Archive is way too large to download on a consumer machine.
A local model works even if the Internet is perfectly firewalled or straight-up down. However, you still need electricity and eventually replacement parts, so in an apocalypse may not work either.
You do understand that was an example, right? Did you know there are other websites you can download? Anna has one that is pretty big and full of information. I’m pretty sure the AI labs know about it too.
> the Internet Archive is way too large to download on a consumer machine.
The Internet Archive isn’t a single zip download. You understand you can pick and choose what you want to download, right? And also, that it was just another example of many?
But the Internet and IA aren't controlled by you. They may be censored or taken down without anyone going near you, but not the local model (as long as you have access to electricity and your hardware doesn't break before you have access to replacements).
A local model has more data than Wikipedia and the Internet Archive is way too large to download on a consumer machine.
A local model works even if the Internet is perfectly firewalled or straight-up down. However, you still need electricity and eventually replacement parts, so in an apocalypse may not work either.
Not every problem will receive $20M in funding to be solved by AI; for the rest, good human guidance will have to suffice. Labs only pulled this stunt because they wanted to show investors how powerful their models are on their own. But look again at the cost of that army of 10,000 SOTA agents. At the very least, I foresee a need for humans to decide when costly AI resources should be committed to a specific search plan. Grant review remains irreducibly human because it involves choosing which directions to fund and weighing opportunity costs: taking one path forecloses others.
A big part of that responsibility can be put in code tests.
Ensuring good test coverage and quality is how you purchase trust in the work agents do. This also reduces the context problem - a test collection has no recall issues, it just runs every time you call it, the whole battery, checks all the things we could check by code in one fast tool call. For the rest, the things we can't test by code, I use manual testing.
A large class of problems are intent divergence, when the model passes tests but it didn't do what I asked. For that I keep a log of all user messages in the project history and review it with agents. This intent alignment is repeated from time to time to catch drift.
So I see the "why should I trust the work agent did?" problem as a combination of 1. ensure good testing 2. review intent alignment.
I just rely on a log of user messages, all messages the user typed in a project as raw data and do a pass with agents to synthesize intent. Then use this for planning and validation of code. I think the user messages are the most valuable data in a project for this reason. Doing this reflection pass on messages takes just a few minutes even for thousands of messages. It keeps global perspective which is often lost in local work.
You don't need to describe it; just show samples of the style you want to achieve. Of course it's not perfect, but it's easier than describing it.
I have my own theory about why it's impossible to remove the human from the loop:
1. Any task emerges from a need, from a human context. We need the human to pay and assume the risks and costs of using the model. So intent emerges from context.
2. While the task is being worked on, constant interaction with the context is needed, for action, for feedback, and for steering.
3. At the end of a task, consequences accumulate in the context, they don't fly to the model provider. The cost, risk, liability, gains and losses remain there.
So the LLM is great except for the start, middle and end of a task. Contexts are humans, teams, projects, and they are maximally distributed, you can't copy a context, it is indexical and relational, just as you can't copy my phone number or eat for me.
reply