Hacker News

Subscribe to Hacker News feed
Hacker News RSS
Updated: 7 min 9 sec ago

Show HN: Thaw – Git branch for a running LLM (fork agents, skip prefill)

Sat, 05/30/2026 - 6:07pm

I built thaw because forking an LLM agent is absurdly wasteful today. When an agent explores N branches — RL rollouts, best-of-N, parallel coding attempts — each branch re-runs prefill over the same shared context. You pay for the same prompt N times.

thaw snapshots a live inference session — weights, KV cache, scheduler state, and the prefix-hash table — and hydrates N children that diverge from the fork point without re-prefilling. It's `git branch` for a running model.

The receipt (H100 80GB, Llama-3.1-8B, real hardware): a pre-warmed pool boots once in 22.3s, then each fork round of 4 branches × 64 tokens runs in 0.88s median. Cold-boot equivalent would be ~340s/round — ~400× amortized. All rounds bit-identical at the fork boundary. Full JSON receipt + reproducer in the repo, nothing hand-waved.

NVIDIA shipped Dynamo Snapshot last week for fast pod cold-starts — and they free the KV cache before checkpoint, by design. thaw is the opposite bet: preserve the KV cache so a fork is near-free. Different problem, opposite mechanic.

pip install thaw-vllm. Works with vLLM and SGLang, Apache-2.0.

https://github.com/thaw-ai/thaw

I'm a solo dev and this is the thing I most want feedback on: is the fork primitive the right shape, or do people want it wrapped in a framework(LangGraph/TRL) node instead? Happy to go deep on the KV-restore internals.

Comments URL: https://news.ycombinator.com/item?id=48341069

Points: 2

# Comments: 0

Categories: Hacker News

Show HN: Phive

Sat, 05/30/2026 - 5:43pm

In 2025, my family and I had a long streak of playing a Gomoku / Go Bang / five-in-a-row based game called OK Play. I built a web version so that we could play any time we wanted (i.e. on our phones after kiddos went to sleep).

The first player to get five-in-a-row (horizontally, vertically, or diagonally) wins. In the first phase of play, players take turns placing their pieces next to existing pieces (always edge-to-edge; you can't place a piece with only a corner-to-corner connection). After players exhaust their pieces, play moves into the movement phase: you pick up an existing piece you own and place it according to the previous placement rules. During the movement phase, you cannot move a piece that would leave other pieces disconnected. Play continues in player order until someone wins.

I wrote the app using Elixir's Phoenix framework with Daisy UI / Tailwind CSS for styling. The app is deployed on Gigalixir via its generous free plan. I am by no means a frontend developer / designer, so there's for sure better ways to implement things than what I have here. I mostly focused on making it mobile friendly and getting it to support light and dark mode. There likely exists browser / device specific bugs, since we've only tested it out on our phones (iPhone 13 Pro, Safari / Chrome) and my computer (MacBook Pro, Safari). Happy to hear any suggestions, frontend or otherwise, if you have them!

Developing this has been a real journey. Highlights have included learning about Gomoku and its variants, articulation points (and Trajan's algorithm for strongly connected components), and the Monte Carlo tree search algorithm (for the intermediate level "AI" mode I've recently added for single-player use). Lowlights have all been CSS related.

I'd love to add a "matchmaking" mode in the future. I haven't really looked too much into the mechanics for how that's usually done though - it'll be a great learning opportunity!

Comments URL: https://news.ycombinator.com/item?id=48340891

Points: 1

# Comments: 0

Categories: Hacker News

Why the Brain Cannot Be a Computer

Sat, 05/30/2026 - 5:42pm

Article URL: https://arxiv.org/abs/2503.10518

Comments URL: https://news.ycombinator.com/item?id=48340887

Points: 1

# Comments: 0

Categories: Hacker News

10g Upgrade

Sat, 05/30/2026 - 5:32pm
Categories: Hacker News

OpenLogi – Rust Alternative to Logi Options+

Sat, 05/30/2026 - 5:25pm

Article URL: https://github.com/AprilNEA/OpenLogi

Comments URL: https://news.ycombinator.com/item?id=48340757

Points: 1

# Comments: 2

Categories: Hacker News

Pages