How do I make money in AI?
The answer is hard…ware

How do I make money in AI?

I spent the last four months learning the AI inference stack. I took three different GPUs and tried to understand them from first principles. An Apple M3 GPU (16 GB unified memory). An AMD Radeon RX 7900 XTX (24 GB VRAM). An NVIDIA RTX 5090 (32 GB VRAM). My repos are public.

I came to it as an auditor. I worked at PwC and KPMG. An auditor does not ask who is famous. An auditor asks one simple question.

Who has power?

The auditor's question

Every business has a limiting factor. One thing that, if you take it away, stops everything. Find that thing and you find who holds the power.

So what is the limiting factor for the AI companies?

If you guessed Sam Altman, you are wrong. If you guessed Dario, you are wrong. If you guessed data, you are wrong too. Data can be bought, scraped and generated.

Try a different test. Take away OpenAI's data centers. Can it survive? No. Take away Anthropic's. Same answer. They die.

Marketers in this space say AGI is here and coding is dead. People who say that do not understand the stack. If AI made software free, why am I paying for ChatGPT? Why are the model companies paying for anything? Ask what they spend their money on. The answer is chips, power and buildings.

The industry wants you to think the product is tokens. Tokens are a bill you pay every month. Hardware is something you own. Token pricing hides the hardware underneath it.

Who has the pickaxe?

The old line from Silicon Valley still holds. Sell the tools. Be the casino. In a gold rush you do not have to pick the miner who wins. You sell pickaxes to all of them. The house always wins.

I wrote about who prints the money in The Real Winners of the AI Wars. This paper asks the next question. If software is cheap and easy to copy, what about hardware?

That question stuck with me, because the answer is clear.

What I measured

I did not want to argue this from opinion. So I measured it.

When a model writes a reply, it writes one token at a time. For each token, the GPU reads the whole model out of memory. So the speed limit is simple. Memory speed divided by model size. The chip's math is mostly waiting. Little (1961) gives the rule for how much work must be in flight to keep a pipe full. Volkov (2016) shows why GPUs hide that wait. Both point at the same thing. Memory moves the work.

Here is what that looks like on two of my GPUs, from Chips and Speed. Model: Qwen3-8B, 4-bit, 4.68 GB of weights.

The limit is the measured memory speed divided by 4.68 GB. The published speed is the number on the box. No chip reaches it.

Same model. Same program. About 14 times the speed. The code did not change. The hardware did.

Nothing I ran went over its limit. Software can get you close to the wall. It cannot move the wall.

The bottleneck at every level

The wall moves as the buyer gets bigger. It stays hardware.

One GPU. Slow replies have two causes. The model is bigger than the GPU memory, so part of it sits in system RAM and is read from there, many times slower. Or the model fits and the memory is just slow. The table above is this case. The fix is more memory, or faster memory.

Two GPUs, or two machines. Now the model can be bigger than one card. How you split it decides what limits you.

• Split by layers. Each GPU holds half the layers and runs them in turn. Only a few kilobytes cross the link per token, so the link is rarely the wall. But the GPUs take turns, so one user gets about the speed of one card.
• Split every layer. Both GPUs work on every token at once. Now they must talk many times per token, and the wait on each message is the wall. A slow link loses the gain.

Thunderbolt used to be too slow for the second way. That changed in December 2025. macOS 26.2 lets Macs read each other's memory over Thunderbolt 5 (Gigazine, 2025). The wait per message fell from about 300 microseconds to under 10. Four Mac Studios ran Qwen3 235B at about 32 tokens per second, against 19 on one (Hardware Corner). Four machines, 1.7 times the speed. The link still sets the ceiling. A better link raised it.

A data center. Hundreds of GPUs. Now the wall is the network and the power. NVIDIA's answer is a rack of 72 GPUs wired so each one talks to the others at 1.8 TB per second, over 5,000 copper cables (NVIDIA, GB200 NVL72). That is more than 14 times a PCIe 5 slot.

At every size the fix is something you buy and wire. Memory, then the link, then the network and the power.

The economics

The second question is about the moat. If your software can be one shot by a model, how long does your business last? How hard is your product to copy?

Hardware is hard to copy. That is the point.

Here is the business, from my planning file. It is the whole stack, in one company:

1. Modify the GPU. Give a consumer card the memory its chip can already use.
2. Build the box. Several modified cards in one machine, tested before the buyer pays.
3. The first prompt. The model is installed and answering before the box leaves.
4. The network. Every computer in the office points at the box. Local inference, on site.

Each step exists somewhere. In the US, nobody sells all four as one package: modified GPUs that arrive as local inference, already set up. The first product is the RTX 4090. Shops in China already rebuild it from 24 GB to 48 GB of VRAM. They move the GPU chip onto a new board with memory on both sides, and add 12 more memory chips. A US shop, GPU Lab, sells them with a 90 day warranty (gpulab.net). Many US companies, labs and universities will not buy from China. They need a US invoice, a warranty and fast delivery.

Four of those cards in one box gives 192 GB of GPU memory. That is enough to run DeepSeek-V4-Flash, a 284 billion parameter model, on site (DeepSeek-V4-Flash-0731). A community vLLM fork documents exactly this four card setup (github.com/yhfgyyf/vllm-deepseek-v4-sm89).

So what do I sell? Four cards, or one box?

The box wins. A buyer next to eBay compares four odd graphics cards on price. Nobody buys four odd graphics cards. They buy a private AI system that works. So the box is the product. Single cards are the cash flow on the side.

The real margin sits on top of the box. Installation. Model setup. Remote support. Wiring it into the software the business already uses. That is where the hard skill gets paid.

And I do not gamble on stock. The buyer pays a deposit. I build to a signed parts list. The box passes tests we agreed on first: every card uses its full 48 GB, the model runs across all four, the temperatures hold. Then the buyer pays the rest.

The closest product on the market is TinyBox, from Tiny Corp, the company behind tinygrad. Its shop sells the RTX 5090 box (Tiny Corp shop, tinybox green v2) and the RTX PRO 6000 box (Tiny Corp shop, tinybox green v2 with RTX PRO 6000). The 5090 box is an older listing. It is still for sale, but the tinygrad.org home page no longer shows it.

Price per GB is not the question a buyer asks. The buyer asks: what is the cheapest way to run this model? The model needs 167 GB for its weights alone (DeepSeek-V4-Flash-0731). Four stock RTX 4090 cards have 96 GB. They cannot run it. The same four chips with 48 GB each have 192 GB. They can. The upgrade is the whole difference.

That upgrade is the skill I sell. NVIDIA already puts this chip, AD102, on a 48 GB card. It is the RTX 6000 Ada, a workstation card sold at a workstation price (NVIDIA, RTX 6000 Ada). The gaming version, the RTX 4090, ships with 24 GB (NVIDIA, RTX 4090). The chip is the same. The memory decides the price tier. NVIDIA and AMD are not paid to give a careful buyer the most memory per dollar. They are paid to sell the next tier up. A shop that moves the chip to a 48 GB board sells the buyer what the tiers hold back. The RTX 4090 is the first card, not the only one. Any card whose chip can use more memory than it ships with is a candidate. The skill is the same. Selling it as the full package is the business.

The hardware and parts for one box cost $23,967.70 in my working case. With labor, reserves and delivery it is $29,839.70. At $44,995 that leaves about $15,000 per box before overhead and tax. That is $5,005 less than the TinyBox with four RTX 5090 cards, which cannot run the model. It is under half the price of the TinyBox that can, and cheaper per GB of GPU memory: about $234 against $237.

Tokens are the service. Hardware is the tool.

Now the other side of the ledger. How much are people spending on tokens?

A lot. Enterprises spent $37 billion on generative AI in 2025. $12.5 billion of it went straight to model APIs (Menlo Ventures, 2025). Uber burned its whole 2026 AI budget in four months (Fortune, 2026). I covered this in How BoltBeam Works.

Look at who sells those tokens. OpenRouter runs about 1.5 quadrillion tokens a year for more than 8 million developers (Menlo Ventures, May 2026). OpenRouter owns no GPUs. It forwards the request and takes a cut. Modal owns no GPUs either. It rents an NVIDIA H100 for $3.949 an hour (Modal pricing, October 6, 2026). Run that all year and the rent is $34,593. For one GPU. Every year.

That is the tell. Tokens are the service. The obvious service of AI. Hardware is the tool. Everyone selling the service is standing on somebody else's tool, and paying rent on it.

So why rent when you can own?

Rent never stops. A tool gets paid off. And the moment you own the tool, you are not only a buyer. You can sell the service too. That is the entire business of every provider behind OpenRouter. Rent GPUs, load an open model, sell the tokens. Own the GPUs and you skip the first step.

And here is the question nobody is asking. What is a token worth after you spend it?

Nothing. A token is used the moment you buy it. It is gone. And the next token is better. Every few months a new model comes out that is smarter and cheaper per token. So the token you bought today was worth the most on the day you bought it. It only loses value from there.

That is the opposite of Bitcoin. Bitcoin is scarce, so holding it is the bet. Tokens get more plentiful and better every year. Better tokens only exist in the future. Every dollar spent on tokens today compounds negatively.

Now flip it. The box does not care which model it runs. When a better open model comes out next year, it runs on the same box. The tool gets better tokens for free. The token buyer pays again.

Bitcoin has a fixed supply, so over the years it has gone up. It went from $60,950 to $110,052 between November 2021 and November 2025, with a 66% fall on the way (Coinbase, BTC-USD daily closes). A token goes the other way. There is always a better model. In November 2021 a model at GPT-3's level cost $60 per million tokens. In November 2024 a model at the same level cost $0.06. That is 10 times cheaper every year (a16z, LLMflation). Epoch AI finds the same thing, 9 to 900 times a year depending on the task (Epoch AI). Money spent on tokens buys an answer that will soon be cheap.

Owning buys things a token price cannot. The data never leaves the building. The price cannot change next quarter. Nobody can switch the model off.

One honest check, because I am an auditor. Owning only pays when the tool is busy. So how busy is busy?

Here is my own usage. I am one person. I use AI to build deterministic software: kernels, a measurement tool, a training pipeline, this site. In 48 active days Claude Code processed 35.2 billion tokens for me, over 89 sessions (Claude Code stats, October 6, 2026).

That is one tool. I use Codex too. It shows another 34.2 billion tokens over 1,651 chats (Codex stats, October 6, 2026).

Together that is about 69 billion tokens, from one person, in a few months. That is the number that matters. AI is worth this much to me because it builds things that keep working, and my usage only goes up.

The models are getting more efficient. The same answer gets about 10 times cheaper every year (a16z, LLMflation). DeepSeek is the clearest case. DeepSeek-V4-Flash is a 284 billion parameter model that fits in the box's 192 GB, and on OpenRouter it costs $0.018 per million input tokens and $1.28 per million output tokens (OpenRouter models list, October 6, 2026).

That is why I think 192 GB is the sweet spot for value per token. It is the smallest box that fits this model: 167 GB of weights, with about 25 GB left for the conversation cache. Below it, 128 GB cannot run the model. Above it, 384 GB costs $91,000.

What is the 25 GB for? It is the working memory. Every conversation the model is in has a cache: what it has already read, kept so it does not read it again. The cache grows with the context, the length of the conversation. The 25 GB also has to hold the model's scratch space while it runs, so the real room for cache is less.

DeepSeek's newest models compress that cache hard. vLLM measured 9.62 GiB per conversation at a full one million tokens, and about half that when the cache is stored in 8 and 4 bits (vLLM, DeepSeek V4). That works out to at most about 4.9 KB per token. V4-Flash is the smaller model, so its real number is lower.

So the 25 GB holds about five conversations at a full million tokens, or about 39 at 128 thousand, before the scratch space. The fork for this four card setup tested 8, 32 and 128 thousand token inputs, and four 8 thousand token requests at once (vLLM DeepSeek-V4 fork).

That is enough for the conversations happening right now. Offloading is for all the others. My sessions run for hours, and each turn rereads the whole conversation. When a cache is thrown away, the next turn has to read the entire conversation again from scratch. Offloading keeps old caches on cheaper memory, so a conversation that comes back reloads its cache instead.

The RAM and SSD sizes are examples, not the box's final parts list. Three standard techniques do this:

• A smaller cache. Store it in 8 bits instead of 16, which halves it (vLLM, quantized KV cache). The four card fork already does this.
• Offload to system RAM. Move caches that are not in use to the CPU's memory. It is slower than GPU memory and far faster than reading the conversation again.
• Offload to SSD. LMCache moves caches through GPU memory, CPU memory, then local SSD (LMCache). SGLang's HiCache does the same (SGLang HiCache).

Total context is what matters for usage like mine. A box that can keep hundreds of millions of tokens of conversation warm on a cheap SSD can bring back yesterday's session without reading it all again.

Cheaper tokens make renting cheaper too. I am not hiding that. On price alone, one person can still rent today. But the box gets better with time, and rent does not. Every year a better model fits in the same 192 GB, and my token count keeps climbing. At some point I would rather own the hardware than keep paying rent on it. An office that works like me gets there first.

Why not just use ChatGPT?

The obvious question. ChatGPT already does this. Claude already does this. Why own a box?

Because I see the bigger picture. The model companies price their apps below what the hardware costs them. That buys two things. Your conversations, to train the next model. And your habits, so you reach for their app first.

The training part is in their own terms. On ChatGPT's consumer plans, your chats can train OpenAI's models unless you turn off "Improve the model for everyone" (OpenAI help). Claude's Free, Pro and Max plans, Claude Code included, ask you to choose. If you allow it, Anthropic keeps that data for five years (Anthropic, 2025).

The habit part is my view. They want you to depend on Claude or ChatGPT, not to learn. Each answer you get from them is one more thing you did not learn to do, and one more industry their models move into.

Here is the best way I know to think about it. Treat AI as a wish. The best wish is not to solve one problem. The best wish is more wishes.

A subscription gives you wishes on someone else's terms, at someone else's price, with someone else keeping the record. A box you own gives you unlimited wishes. You can go slower, make mistakes, and learn, and none of it is metered.

Who is doing this?

Every part of my plan already exists. I could not find anyone who sells all of it together.

Modding the cards. GPU Lab in Michigan turns a 24 GB RTX 4090 into a 48 GB card (GPU Lab). It sells single cards, not systems. Shops in Shenzhen move the 4090 chip onto new boards with 48 GB (Igor's Lab). One seller lists a single workstation with a modded 48 GB card (Jawa listing).

Selling boxes with several GPUs. Tiny Corp sells the TinyBox, from the table above. NVIDIA sells the DGX Spark and the DGX Station (NVIDIA, DGX Station). Lambda, Bizon, Puget Systems, Exxact and Comino sell workstations and servers (Lambda, Bizon, Puget Systems, Exxact, Comino). All of them use stock cards.

Private AI for small businesses. VRLA Tech ships servers with Ollama installed, ready when you power them on, with lifetime US support. It aims at teams of 2 to 15 people (VRLA Tech). Zylon sells private AI software to regulated industries (Zylon). Both use stock hardware.

That is the gap. The whole stack as one package, in the US: modified GPUs, four in one tested box, a large open model already installed, the office's computers already pointed at it, guardrails, and support. The open question is trust. Nobody selling boxes has yet asked a business to trust a modded card. That is what the tests before final payment are for.

Putting it all together

This is the end result of the business plan. The box is one central place for local inference, serving every computer in the office. It arrives ready. Plug it into the network, turn it on, and the inference is already set up for an enterprise, with its own guardrails.

The hardware is hard. The software should not be.

I can promise that because I have worked on every layer of this stack myself.

• Kernels. In my tinygrad fork I wrote the GPU code that runs the model (github.com/JulianAbeleda/tinygrad-arkey). One decode kernel ran on AMD, Apple and NVIDIA (A Universal Kernel).
• Measurement. BoltBeam checks each kernel against the speed limit of its chip (How BoltBeam Works).
• Training. DayCare teaches a small model one skill, without the usual black box (DayCare).
• The client. Arkey, below, is what a person actually touches.

The box boots into one menu. Arkey. It is open source (github.com/JulianAbeleda/arkey_v3). Pick the TUI and you are talking to the model in a terminal coding agent.

The agent is swappable. Arkey runs Codex, Kimi Code or Crush, and Claude Code is next. Here it is Crush, Charm's open source terminal agent (github.com/charmbracelet/crush). Arkey does not ship any of them. It copies the one you already installed into its own folder, then points it at the model you chose. Your normal install is left alone. Whichever agent your team already knows is the one it uses on the box.

Not everyone wants a terminal. For a chat window in the browser, the box runs DeepSeek Harness, DeepSeek's open source agent harness with a web UI (github.com/deepseek-ai/deepseek-harness). Pointing it at the box is one form: a custom provider whose base URL is the box's address on the office network.

Config has four choices. That is the whole setup.

• Local. Run the model on this machine.
• Server. Use a llama.cpp server on your network. That is the box. Every computer in the office runs Arkey and picks the same server.
• Frontier. Use a hosted provider when you need one.
• GPU Auto-scan. Find the GPU and set llama.cpp up for it.

Own the box. Point Arkey at it. Done.

Why most people will not do this

I am not hiding the plan. Anyone can do it. Most people will not, because hardware is hard.

Hard means three things. The skill only comes from hours at the bench. Every mistake costs a real card. And no model can do it for you.

Reballing memory on a GPU is hard. Moving a GPU chip onto a new clamshell board is hard. Building a four card machine that does not cook itself is hard. Networking and kernel engineering are hard, even though they are software.

This is my partner and me, at the bench. Before we touch a 4090, we practice on an AMD board, to test that we have the skill.

Watch it done. Gamers Nexus filmed an RTX 4090 being rebuilt from 24 GB to 48 GB, by hand, in Brother Zhang's repair shop in China.

You cannot prompt your way into a solder joint. You cannot one shot a thermal test. A model can write the plan. It cannot hold the hot air station.

Hard is the moat. It always was.

The end game

At the highest level the scarce thing is no longer only chips. It is power, and a place to connect it. Microsoft's CEO said in 2025 that he has chips in inventory he cannot plug in: "I don't have warm shells to plug into" (DCD, 2025). In the US a new power project waits about five years to connect to the grid (LBNL, 2025). The IEA says about one in five planned data centres could be delayed by grid limits (IEA, 2025).

China is building for this. In 2022 it started a national plan, East Data West Computing, to put data centers in its energy rich west (Sixth Tone, 2022). In 2025 it added more than 430 GW of wind and solar. BloombergNEF expects China to add more than six times as much generating capacity as the US over the next five years (Al Jazeera, 2026).

The US treats this as a race. Its 2025 AI Action Plan names China, and orders faster permits for data centers and the power plants behind them (White House, 2025).

Elon Musk is looking past Earth for the same reason. He plans to merge xAI into SpaceX. He said: "The lowest-cost place to put AI will be in space, and that will be true within two years, maybe three at the latest" (Fortune, 2026). In orbit the sun shines almost all the time. There are no neighbors and no grid queue. The power is not free. You pay for the launch, the panels and the radiators. A Georgetown researcher in the same article says it is "just technologically not feasible at the moment."

Power that does not wait in a queue is the last moat.

References

• Little, J. D. C. (1961). "A Proof for the Queuing Formula: L = λW." Operations Research 9(3), 383 to 387. doi.org/10.1287/opre.9.3.383
• Volkov, V. (2016). "Understanding Latency Hiding on GPUs." UC Berkeley, UCB/EECS-2016-143. eecs.berkeley.edu
• Abeleda, J. (2026). Chips and Speed. Arkey Research.
• Abeleda, J. (2026). The Real Winners of the AI Wars. Arkey Research.
• Abeleda, J. (2026). How BoltBeam Works. Arkey Research.
• Menlo Ventures (2025). 2025: The State of Generative AI in the Enterprise. menlovc.com
• Menlo Ventures (26 May 2026). OpenRouter now processes more than a quadrillion tokens a year. menlovc.com
• Fortune (26 May 2026). Uber burned through its entire 2026 AI budget in four months. fortune.com
• Modal. GPU pricing, checked October 6, 2026. modal.com/pricing
• OpenRouter. Models list, DeepSeek-V4-Flash-0731 prices, checked October 6, 2026. openrouter.ai/api/v1/models
• Gamers Nexus. Creating a 48GB NVIDIA RTX 4090 GPU | Brother Zhang's Repair Shop (ft. 张哥). YouTube. youtube.com/watch?v=TcRGBeOENLg
• Tiny Corp. tinybox green v2, 4 × RTX 5090, $50,000, checked October 7, 2026. tinycorp.myshopify.com
• Tiny Corp. tinybox green v2 with 4 × RTX PRO 6000 Blackwell, $91,000, checked October 7, 2026. tinycorp.myshopify.com
• GPU Lab. RTX 4090 48 GB upgrade and warranty terms. gpulab.net/warranty
• yhfgyyf. vLLM fork for DeepSeek-V4 on four RTX 4090 48 GB cards. github.com/yhfgyyf/vllm-deepseek-v4-sm89
• DeepSeek. DeepSeek-V4-Flash-0731 model card. huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731
• Gigazine (15 December 2025). macOS Tahoe 26.2 adds the ability to connect multiple Macs to create AI clusters. gigazine.net
• Hardware Corner. RDMA over Thunderbolt lets Mac Studios run huge local LLMs faster than expected. hardware-corner.net
• NVIDIA. NVIDIA contributes NVIDIA GB200 NVL72 designs to Open Compute Project. developer.nvidia.com
• NVIDIA. RTX 6000 Ada Generation. nvidia.com
• NVIDIA. GeForce RTX 4090. nvidia.com
• Coinbase Exchange. BTC-USD daily closes, November 1 of 2021 to 2025. api.exchange.coinbase.com
• Appenzeller, G. (2024). Welcome to LLMflation: LLM inference cost is going down fast. a16z. a16z.com
• Epoch AI (2025). LLM inference prices have fallen rapidly but unequally across tasks. epoch.ai
• Charm. Crush, a terminal coding agent. github.com/charmbracelet/crush
• International Energy Agency (2025). Energy and AI: executive summary. iea.org
• Lawrence Berkeley National Laboratory (2025). Queued Up: 2025 edition. emp.lbl.gov
• DatacenterDynamics (2025). Microsoft has AI GPUs sitting in inventory because it lacks the power necessary to install them. datacenterdynamics.com
• Sixth Tone (2022). Five things to know about China's mega East-West data center plan. sixthtone.com
• Al Jazeera (28 May 2026). China's secret weapon in AI race with US? Lots of cheap energy. aljazeera.com
• The White House (2025). Winning the Race: America's AI Action Plan. whitehouse.gov
• Fortune (19 February 2026). Can AI data centers really move to space? Experts say not for decades. fortune.com
• DeepSeek. DeepSeek Harness, an open source agent harness with a web UI. github.com/deepseek-ai/deepseek-harness
• Abeleda, J. tinygrad-arkey, a tinygrad fork. github.com/JulianAbeleda/tinygrad-arkey
• Abeleda, J. (2026). A Universal Kernel. Arkey Research.
• Abeleda, J. (2026). DayCare: Why Is Training So Complicated?. Arkey Research.
• vLLM. Quantized KV cache. docs.vllm.ai
• LMCache. Tiered KV cache offloading across CPU memory, local SSD and remote storage. github.com/LMCache/LMCache
• SGLang. HiCache, hierarchical KV caching. docs.sglang.ai
• vLLM (24 April 2026). DeepSeek V4 in vLLM: efficient long-context attention. vllm.ai
• OpenAI. How your data is used to improve model performance. help.openai.com
• Anthropic (2025). Updates to our consumer terms. anthropic.com
• GPU Lab. 48 GB RTX 4090 upgrades. gpulab.net
• Igor's Lab (2025). RTX 4090 with 48 GB VRAM and water cooling. igorslab.de
• Jawa. RTX 4090 48 GB workstation listing. jawa.gg
• NVIDIA. DGX Station. nvidia.com
• VRLA Tech (2026). On-premise AI for small business in 2026. vrlatech.com
• Zylon. Private AI for regulated industries. zylon.ai
• Lambda, Bizon, Puget Systems, Exxact, Comino. GPU workstations and servers. lambda.ai, bizon-tech.com, pugetsystems.com, exxactcorp.com, comino.com