In the fall of 2024, a component-by-component cost breakdown of a single Nvidia H100 server quietly made the rounds in semiconductor industry circles. Its central claim: HBM (High Bandwidth Memory) accounts for a bigger share of the cost than the GPU die itself. Earlier this year, a similar breakdown resurfaced with harder numbers attached. About 66%, or two-thirds, of what you pay for an AI chip goes not to the compute circuitry but to memory.
That same day, a post scored 121 points on Hacker News, the developer community's watering hole. Its title: "Don't Use Claude as Your Architect." The author described handing system architecture design to a large language model, then losing days untangling the structurally messy code that resulted. The comments filled up with similar war stories.
It might look like coincidence that a hardware supply-chain story and a developer's workflow complaint drew attention on the same day. But read the two signals side by side, and a set of decision criteria starts to take shape — one that solo entrepreneurs and small teams bringing AI tools into daily operations need right now.
What It Means That 66% of an AI Chip's Cost Is Memory
Unlike ordinary DRAM, HBM is built by stacking multiple layers of memory chips vertically and mounting them alongside the GPU die on a silicon interposer. That construction moves data more than 10 times faster than conventional memory — but it also comes with far greater manufacturing complexity and yield-management overhead. Only three companies can mass-produce HBM today: SK Hynix, Samsung Electronics, and Micron. For HBM3E and higher-spec products, SK Hynix alone holds more than 50% of the market.
That's why a strike at Samsung Electronics is more than a labor headline. When Samsung's semiconductor union staged its longest strike on record in 2024, industry insiders worried it could delay improvements to HBM yields. This isn't just one company's labor dispute — it's a structural chokepoint for the entire global AI infrastructure supply chain, since HBM supply volumes are a direct variable in how Nvidia schedules H100 and H200 shipments.
Look closer at what this number really means, and it points to something important: AI model performance isn't decided by algorithm design alone. Ever since the Transformer architecture went public, model architecture itself has effectively become open-source knowledge. GPT, Claude, and Gemini are all Transformer-based; what separates them is training-data quality, fine-tuning approach, and inference speed. Inference speed, in turn, comes down to chip performance — and chip performance is tethered to HBM supply. No matter how sophisticated your prompt engineering gets at the software layer, service quality shifts the moment the silicon supply chain underneath it wobbles.
For a solo entrepreneur or a small team, this dynamic usually becomes tangible the moment API prices rise or response times slow down. When ChatGPT introduced its paid tier in early 2023 and throttled response speeds for free users, many practitioners felt the underlying cost structure of AI infrastructure for the first time. As long as HBM supply constraints persist, AI API prices are likely to fall more slowly than most people expect.
Why the "Don't Use It as Your Architect" Warning Struck a Nerve
On Hacker News, 121 points is no small number. A score that high on a contentious topic, or a post carrying strong disagreement, signals that plenty of developers have lived through something similar.
Here's the gist of the post. Ask Claude or GPT-4 to "design the entire backend architecture for this service," and you'll get back a plausible-looking diagram and code structure. That's where the trouble starts. Often, that architecture reflects none of the real constraints — the actual deployment environment, the team's capacity to maintain it, dependencies on the existing codebase. Adopt it without realizing that, and the debugging and refactoring time down the road ends up longer than if you'd designed it yourself from scratch.
One commenter wrote, "An LLM is closer to a code-autocomplete tool than an architecture decision-maker." Another noted, "The real problem is that the model doesn't know when its answer is wrong." Those two lines capture the whole debate.
A separate story that broke the same day, about chatbot "personality hacking," reads as part of the same context. It described how a cleverly crafted system prompt could push a chatbot past its assigned role boundaries. In other words, both the developer community and everyday users are now in a hands-on phase of testing exactly how much authority to hand an AI agent.
Anthropic's decision to open-source "knowledge-work-plugins" around this same time reads like a response to that very debate. The design philosophy confines AI to a plugin that assists with specific tasks, rather than a controller that runs the entire system — which lines up with the community's hard-won conclusion that "Claude should be a plugin, not an architect."
The debate over how much scope to grant an AI agent, at bottom, is a question of setting trust boundaries. It echoes a principle found in practical guides across other fields: if you don't define clear role boundaries in the first 90 days — or in the earliest stage — of working with a new tool or partner, the cost of cleaning up later grows disproportionately. AI agents are no exception. Fail to narrowly define their scope early on, and plausible-but-off-context output piles up, until reversing it costs more time than doing the work yourself from the start.
Three Things Solo Entrepreneurs Should Check Right Now
Don't count on AI API costs falling fast. HBM supply constraints aren't going away soon. Samsung's HBM3E yield problems persisted through all of 2024, and even SK Hynix's new fabs meant to expand supply aren't coming online until sometime across 2025-2026. If your workflow currently treats AI API access as effectively unlimited, now's a good time to revisit the cost structure — especially for token-hungry work like image generation or large-document processing, which will likely feel any future pricing shifts the hardest.
Consciously flag the moments you're handing AI a "design" decision. For clearly bounded tasks — drafting copy, summarizing research, automating repetitive documents — AI is a genuine speed multiplier. But in areas that require context and judgment — strategic planning, service architecture, the direction of customer communication — adopting AI's output as-is risks following an answer that sounds plausible but doesn't fit your actual situation. "Don't use Claude as your architect" isn't just a developer's problem. Business strategy, content structure, and pricing design all sit inside the same boundary.
Write down the role boundaries for your AI tools, once. Spell out which model handles which task, and who makes the final call on the output. For a solo entrepreneur, this is really a promise to yourself — but skip it, and you'll drift into unconsciously rubber-stamping whatever AI produces. A simple operating rule, like "GPT-4o generates drafts; I personally review and send the final version of any client proposal," heads off the boundary-blurring problem before it starts.
Most of the friction solo entrepreneurs run into with AI tools today traces back to this exact point. The tools look impressive, so it's natural to hand over a lot at first. Then, at some point, the question arrives: "Is this actually the direction I wanted?" The later that question shows up, the bigger the cleanup bill.
The HBM supply chain sets the ceiling on both the quality and the price of AI services, while the developer community is narrowing the role boundaries of AI agents through direct, hands-on experimentation. At the point where these two currents cross, the decision of where and how deeply to bring AI into your business isn't a tool-selection question — it's a question of how you run your business. Keep following whatever design the tool proposes, and at some point you may find your business has quietly reoriented itself around the tool, without you ever deciding it should.
