2026
5 篇文章用 Amazon Bedrock prompt caching 把重複的 context 成本壓低 90%
Amazon Bedrock 的 prompt caching 讓重複輸入的 token 成本最多降 90%，同時縮短首字延遲，適合多輪問答與 agent 工作流。
閱讀文章 ↗把模型快取放進節點：HyperPod 推論冷啟動的實務取捨
Amazon SageMaker HyperPod 推出模型快取，把權重與容器映像預載到節點 NVMe，讓擴容從數十分鐘縮到數秒。
閱讀文章 ↗前綴感知路由：讓 KV cache 不再被隨機打散
SageMaker Inference 新增前綴感知路由，把相同 prompt 開頭固定送到同一台機器，讓 prefix caching 真正累積成可重用的 KV cache。
閱讀文章 ↗用 Zstandard 與 Pingora 節省 PB 級快取儲存：Cloudflare 的 Cache Transcoding 原型
Cloudflare 工程實習生打造 Cache Transcoding 原型，在 Pingora 快取內以 Zstandard 壓縮文字資產，用少量 CPU 換取可觀的儲存與跨資料中心頻寬節省。本文解析其取捨、設計與測試結果。
閱讀文章 ↗HTTP QUERY 成為 RFC 10008：讀取型查詢的新標準
HTTP QUERY 方法正式標準化為 RFC 10008，讓讀取查詢可以帶請求主體又維持 safe 與 idempotent 語意，一舉解決 GET 網址過長與 POST 語意錯置這兩個 API 設計的老問題。
閱讀文章 ↗
2026
5 ARTICLESPrompt Caching on Bedrock: Where the 90% Input Savings Actually Come From
Amazon Bedrock prompt caching cuts repeated-context input costs up to 90% and lowers TTFT, but only if you place cache points and TTLs deliberately.
READ POST ↗Model Caching on HyperPod: What Changes When Weights Live on the Node
HyperPod model caching pre-loads weights and images to local NVMe, cutting scale-out cold starts from tens of minutes to seconds.
READ POST ↗Prefix-Aware Routing on SageMaker: What Changes When Your Prompt Starts the Same Way
SageMaker's new routing strategy sends identical prompt prefixes to the same instance so KV cache actually gets reused.
READ POST ↗How Cloudflare Could Save Petabytes of Cache Storage with Zstandard and Pingora
Cloudflare's Cache Transcoding prototype compresses cache entries with Zstandard inside Pingora, trading a small CPU increase for significant storage and bandwidth savings.
READ POST ↗HTTP QUERY Becomes RFC 10008: What APIs Gain
The HTTP QUERY method is now RFC 10008 — safe, idempotent requests that carry a body, fixing GET's URL-length limits and POST's wrong semantics for read-only API queries.
READ POST ↗