There are "humanizer" scripts that these people like to use. This one is not the typical style I see in my online moderator adventures, which is usually "lowercase everything and remove all punctuation except periods", as if we are repelled by em-dash use and not the tortured prose.
I like the removal of a random definite article, to simulate an ESL writer and garner sympathy for using LLMs as a "tool to clean up posts". Heavens, they would never even think of using their Claude subscription to be a slop factory, that would cross a line.
You can definitely offload n-gram embeddings to storage; they're very sparsely used (only a few KB fetched per token) so this is quite effective. Loading to DRAM only becomes necessary if they are a bottleneck to overall performance (which might happen if you're doing very wide batches and everything else uses super fast VRAM/HBM).
I was looking at the qwen-next-flash, and the weights would fill my OEM Spark on their own, before the n-gram. I'm unclear if offloading to disk can work here, is that what you are implying is possible?!
I'd recommend pointing your agent at it (after installing sparkrun), and asking it to research the absolute latest in TP=1 Flash-Next - mine grabbed particular vLLM nightlies and mods to improve performance, and it was well worth it.
I have a quirky vLLM on k8s on 2x OEM sparks setup with 9 models available to me. I'm not keen to run nightly vLLM, too many issues with it in the past. Going the qwen-next path means displacing things I use daily :/
I have a watchful eye on the diffusion ~ Jev/Kev PR
For what it's worth, Flash Next outperforms every other model that is available to us on the GB10 in all of my testing; though if you have two sparks then the TP=2 version is even better and easier (I don't think you'll need the nightly for that at all, just use the recipe)
Nah, I’m streaming ngrams off NVMe on my Spark-alike right now. Works surprisingly well (except for when I accidentally bottlenecked it through my NAS)
GB10 boxes have way more compute than they have memory bandwidth, which nicely fits medium sized MoE models with speculative execution (MTP, DSpark/DFlash, etc)
Qwen 3.8 Flash Next (what I'm running basically entirely now) sees 30 / 35.0 / 45 tk/s for prose, analysis and code respectively for actual use (not short context benchmarking) with Pi. Thinking blocks are ~35tk/s or so.
The GB10 having so much compute is great for prefill too, 2000-3000/s for 14k to 64k token prompts (cold cache too) in the quick benchmark I did. 3500tk/s for warm cache which is nice :)
When I accidentally streamed my ngrams over the 2.5Gb/s network, it cut all the throughput down in half basically. Especially notable for the time-to-first-token, which is what clued me in that I'd messed up somehow!
For Qwen 3.8 27B, I got it up to a consistent 20tk-25tk/s but 27B thinks so much that it was honestly too painful: Flash Next is as smart, as useful, but much faster for real agentic dev usage IMO
Laguna S 2.1 saw similar numbers to Flash Next if I remember right, but their latest updates means it doesn't quite fit a GB10 128GB anymore at full context which is a shame.
Note: these are all NVFP4 quants (usually a dynamic one where some tensor layers are left at full precision though)
Yeah, I agree it's probably fixable, but I think a naive interpretation of the metadata is probably the source of this bug (which HF has had for as long as I can remember)
Perhaps not 1:1, but, personally, I can read for much longer than I can write. And so I'd argue that this duration increase matches any deeper thought that comes with shorter writing bursts.
That should tell you something about how much effort (thinking) is required for the activity. The fact you writing is t something you can do as much as reading means you use your brain a bit more. I would claim that consuming information always uses less thought then generating information. For a human brain at least.
Most people[1] interacting with chatbots don't have a paid subscription and they do interact with the free-tier LLMs that are Luna and Haiku, so I still think it's relevant.
[1]: not on HN obviously, but IRL, and probably among FT's readership as well.
Come on… It's been known for more than a century that bartering has never been the default mean of exchange among humans, can we kill this myth eventually?!
This is obviously not an attempt at anything remotely resembling documentary history. It’s more of a just so story explaining what problem each step solved.
Unless of course one thinks that smooth gray stones from a riverbed two miles away reflects some historically significant point in money’s evolution.
I'm not going to explain David Graeber's Debt to you, but yes, this exact 18th century fantasy of the emergence of money in caveman times contains the same kind of misunderstandings that also leads to most people having an incredibly misguided view of how the contemporary global financial system operates.
I certainly don't think anything like this actually happened. And the linked article doesn't really do so either. I do think it can be a useful illustration of what problem each technology solves.
In another thread you talk about how the internet really is a series of tubes. We all know that "tubes" can be a useful analogy for understanding certain things about circuits and things built upon them. But we also don't think that tubes are more than a very basic analogy. Tubes don't really do packet switching.
By leading with the improvement-order-barter story, the article announces a bias that borders on propaganda. It's maybe worth objecting to even if the goal isn't to be historically accurate.
The problem that was actually being solved when money was invented was how to get recently conquered villagers to support the soldiers that recently conquered them. (Demand taxes in the coinage that you pay your solders with, then they can live off the local economy and you don't have to feed them directly, which might be hard if they're very far away--tax collectors can take years to show up, but the demand for the new currency is instant, so your soldiers get fed right away).
The system of interpersonal debt that predated money likely outperformed money in a variety of ways from the perspective of the users. It was only for the conquerors that the invention of money solved any problem at all. It's hard to keep reading the article when it starts with the story that is used to distract from this.
There's a central problem, a blind spot, that plagues economics: because the fundamental credo of the discipline is that it's all about free people engaing in mutually beneficial trade to maximize their respective well-being, it completely neglects how important armed force and violence are in shaping the economic system we know.
Agreed. We should acknowledge that it has historically been about ensuring that conquerer grandpa and conquered grandpa end up with different qualities of life even after they're too old to swing a sword. I.e. extending violence begotten structure into places where violence is impractical.
The problem is the story is wrong, and that leads to misunderstandings of what money is and how economic institutions arose in fact vs mythology.
For starters, debt records came before money basically everywhere we have evidence for the invention of money.
i would very strongly suggest reading David Graeber. It completely revolutionized my understanding of how human social and economic institutions emerge and evolve in actual fact vs imagined myth.
We can just tell the real story, it's far more interesting and useful than the myth.
No, what I and the other commenter are saying is you don't understand why the article is misleading because you don't know the actual scholarly knowledge here.
In your other comment you use wording along the lines of "still useful to show what problem each technology solved" but the part you're missing is the explanations are largely wrong, not just in the sense of not telling actual history but you will come alway with mistaken conclusions because the imagined/example history is fundamentally wrong.
So, consider if you're getting this from multiple people, perhaps you're missing something on the topic.
There are two commenting critics. So "multiple" is true, I suppose, but only quite weakly so. And there are more than 4x that many upvotes on the top-level comment. So I don't think it is as simple as "everyone is saying you are wrong."
Not only bartering has never been the default mean of exchange, but also the idea that money were created to optimize the trading process is also wrong.
For an item to become currency in a society, it must already be something people value independently of its use as money, and people must expect others to accept it as well. Say the villagers live near the sea and find some beautiful shiny stones that everyone wants. Those stones can then become currency.
A good example of this is tobacco in colonial America. It came to be used as currency because it was already widely traded and valued.
It's not “just” that, because monetary systems don't work like that at all.
The point of GP is that while the myth says “in the beginning there was barter, and barter is inefficient so we created money to allow trading more efficiently” the reality is more “In the beginning there was trade, and some traded goods became money because they had the correct properties”.
reply