https://www.bobhuang.com/series/ai-frontiers/Reading the research and the engineering that turned language models into products: one paper, technique, or production problem per part, from the transformer paper to what I would do differently shipping AI systems.en17 Sep 2026 12:00:00 GMThttps://www.bobhuang.com/blog/979/ai-frontiers-part-65-field-guide-to-shipping-ai-systems/https://www.bobhuang.com/blog/979/ai-frontiers-part-65-field-guide-to-shipping-ai-systems/02 Jul 2026 12:00:00 GMTThe last entry in the series: the ten practices I would keep, the five things I got wrong, and what five years of watching this stack get built actually taught me about shipping software that thinks.https://www.bobhuang.com/blog/978/ai-frontiers-part-64-economics-of-applied-ai-where-the-margin-is/https://www.bobhuang.com/blog/978/ai-frontiers-part-64-economics-of-applied-ai-where-the-margin-is/04 Jun 2026 12:00:00 GMTToken prices fell by orders of magnitude and most AI products still do not have software margins. The full cost stack, the measured productivity evidence, and the four things that actually protect an applied AI business.https://www.bobhuang.com/blog/977/ai-frontiers-part-63-interoperability-standards-and-portable-context/https://www.bobhuang.com/blog/977/ai-frontiers-part-63-interoperability-standards-and-portable-context/07 May 2026 12:00:00 GMTMCP, A2A, OpenAPI, OpenTelemetry: the agent stack is acquiring protocols faster than it is acquiring practice. What to standardize, what to keep proprietary, and why your evaluation set is the most portable asset you own.https://www.bobhuang.com/blog/976/ai-frontiers-part-62-human-in-the-loop-review-queues-trust-calibration/https://www.bobhuang.com/blog/976/ai-frontiers-part-62-human-in-the-loop-review-queues-trust-calibration/09 Apr 2026 12:00:00 GMTA human reviewer is a system component with a latency budget, an error distribution, and a fatigue curve. Designing the queue, the interface, and the escalation rules is where human-in-the-loop systems succeed or quietly become rubber stamps.https://www.bobhuang.com/blog/975/ai-frontiers-part-61-retrieval-at-scale-sharding-and-freshness/https://www.bobhuang.com/blog/975/ai-frontiers-part-61-retrieval-at-scale-sharding-and-freshness/05 Mar 2026 12:00:00 GMTRetrieval systems look elegant at a million vectors and get difficult at a billion. Approximate indexes, shard routing, the freshness problem that vector stores handle badly, and the invalidation rules that decide what your users actually see.https://www.bobhuang.com/blog/974/ai-frontiers-part-60-continuous-evaluation-in-production-drift/https://www.bobhuang.com/blog/974/ai-frontiers-part-60-continuous-evaluation-in-production-drift/05 Feb 2026 12:00:00 GMTA pre-launch evaluation set is stale the day it ships. What to monitor, how to get labels cheaply from production, how to detect drift without drowning in false alarms, and how to make rollback a one-line change.https://www.bobhuang.com/blog/973/ai-frontiers-part-59-model-routing-choosing-a-model-per-request/https://www.bobhuang.com/blog/973/ai-frontiers-part-59-model-routing-choosing-a-model-per-request/08 Jan 2026 12:00:00 GMTOnce you have more than one model, every request is a decision. How routers are built, why offline accuracy numbers overstate them, and how to evaluate a policy you can only observe one arm of.https://www.bobhuang.com/blog/972/ai-frontiers-part-58-data-residency-and-sovereignty-inference/https://www.bobhuang.com/blog/972/ai-frontiers-part-58-data-residency-and-sovereignty-inference/04 Dec 2025 12:00:00 GMTModel inference creates data flows that no cloud-region dropdown covers. A practical map of the legal stack, the deployment options, and the surprising places your prompts actually travel.https://www.bobhuang.com/blog/971/ai-frontiers-part-57-security-of-agentic-systems-permissions-and-sandboxes/https://www.bobhuang.com/blog/971/ai-frontiers-part-57-security-of-agentic-systems-permissions-and-sandboxes/06 Nov 2025 12:00:00 GMTAn agent is a program that takes instructions from its input, holds credentials, and can act. That combination is a new security model, and the defenses that work come from 1975, not from prompt engineering.https://www.bobhuang.com/blog/970/ai-frontiers-part-56-science-with-models-from-conjecture-to-verification/https://www.bobhuang.com/blog/970/ai-frontiers-part-56-science-with-models-from-conjecture-to-verification/09 Oct 2025 12:00:00 GMTWhere models genuinely accelerate science — search with a verifier, fast surrogates, closed experimental loops — and why every real result still ends at an experiment that a model cannot run.https://www.bobhuang.com/blog/969/ai-frontiers-part-55-biology-and-protein-models-what-alphafold-changed/https://www.bobhuang.com/blog/969/ai-frontiers-part-55-biology-and-protein-models-what-alphafold-changed/04 Sep 2025 12:00:00 GMTProtein structure prediction was the first scientific field where a model genuinely changed the daily work. A tour of AlphaFold, protein language models, generative design — and the limits that wet-lab validation keeps rediscovering.https://www.bobhuang.com/blog/968/ai-frontiers-part-54-robotics-and-embodied-models-sim-to-real-gap/https://www.bobhuang.com/blog/968/ai-frontiers-part-54-robotics-and-embodied-models-sim-to-real-gap/07 Aug 2025 12:00:00 GMTVision-language-action models turned robot demos into something that looks like general manipulation. The distance between a demo and a deployment is physics, data, and reliability — and it is measured in nines, not benchmarks.https://www.bobhuang.com/blog/967/ai-frontiers-part-53-reasoning-distillation-thinking-into-small-models/https://www.bobhuang.com/blog/967/ai-frontiers-part-53-reasoning-distillation-thinking-into-small-models/03 Jul 2025 12:00:00 GMTDeepSeek-R1's release made chain-of-thought distillation a production technique: a small model can learn to reason from a large model's traces. What transfers, what only looks like it transferred, and how to run the loop without inflating your latency budget.https://www.bobhuang.com/blog/966/ai-frontiers-part-52-long-context-retrieval-position-and-recency/https://www.bobhuang.com/blog/966/ai-frontiers-part-52-long-context-retrieval-position-and-recency/05 Jun 2025 12:00:00 GMTA million-token window did not replace retrieval. Why models lose information in the middle of long inputs, how context windows were extended, and the engineering pattern that actually works at scale.https://www.bobhuang.com/blog/965/ai-frontiers-part-51-evaluation-of-agents-traces-and-cost-aware-scoring/https://www.bobhuang.com/blog/965/ai-frontiers-part-51-evaluation-of-agents-traces-and-cost-aware-scoring/24 Apr 2025 12:00:00 GMTAgent benchmarks told a story of rapid progress that mostly reflected measurement choices. What to hold out, what to count, why pass^k beats pass@k, and how to build an agent evaluation you can trust.https://www.bobhuang.com/blog/964/ai-frontiers-part-50-inference-cost-engineering-unit-economics/https://www.bobhuang.com/blog/964/ai-frontiers-part-50-inference-cost-engineering-unit-economics/03 Apr 2025 12:00:00 GMTInference cost is not a line item, it is an architectural property. A cost model for LLM systems, the four levers that actually move the number, and the unit economics that decide whether an AI feature is a business.https://www.bobhuang.com/blog/963/ai-frontiers-part-49-agent-observability-tracing-and-replay/https://www.bobhuang.com/blog/963/ai-frontiers-part-49-agent-observability-tracing-and-replay/06 Mar 2025 12:00:00 GMTAn agent that fails in production tells you nothing unless you captured the trajectory. What to record, how to make runs replayable, and why the trace is the most valuable artifact an AI system produces.https://www.bobhuang.com/blog/962/ai-frontiers-part-48-fine-tuning-versus-prompting-the-decision/https://www.bobhuang.com/blog/962/ai-frontiers-part-48-fine-tuning-versus-prompting-the-decision/06 Feb 2025 12:00:00 GMTFine-tuning and prompting are not competitors; they change different things. A decision procedure for choosing between them, and the evidence on when each one actually wins.https://www.bobhuang.com/blog/961/ai-frontiers-part-47-multi-agent-systems-when-decomposition-helps/https://www.bobhuang.com/blog/961/ai-frontiers-part-47-multi-agent-systems-when-decomposition-helps/19 Dec 2024 12:00:00 GMTBetween 2023 and 2024 the field tried to turn one model into a team. The results split cleanly: decomposition wins when the subproblems are genuinely separable and loses when the bottleneck is verification.https://www.bobhuang.com/blog/960/ai-frontiers-part-46-small-models-on-device-npus-and-privacy/https://www.bobhuang.com/blog/960/ai-frontiers-part-46-small-models-on-device-npus-and-privacy/27 Nov 2024 12:00:00 GMTOn-device inference is a memory-bandwidth problem wearing a TOPS costume. What fits on a phone, why NPUs are less magic than the spec sheet suggests, and why privacy regulation may be the strongest argument for local inference.https://www.bobhuang.com/blog/959/ai-frontiers-part-45-structured-output-schemas-constrained-decoding/https://www.bobhuang.com/blog/959/ai-frontiers-part-45-structured-output-schemas-constrained-decoding/07 Nov 2024 12:00:00 GMTAsking a model to return JSON politely is not a contract. A tour of constrained decoding — regex, grammars, logit masking — and the harder question of whether valid output is actually correct output.https://www.bobhuang.com/blog/958/ai-frontiers-part-44-synthetic-preference-data-rlaif-flywheel/https://www.bobhuang.com/blog/958/ai-frontiers-part-44-synthetic-preference-data-rlaif-flywheel/10 Oct 2024 12:00:00 GMTHuman preference labels cost dollars each and hours of latency; model-generated ones cost fractions of a cent. How RLAIF, UltraFeedback, and self-rewarding pipelines work — and the four ways the flywheel spins itself into a wall.https://www.bobhuang.com/blog/957/ai-frontiers-part-43-embeddings-as-infrastructure-search-clustering-drift/https://www.bobhuang.com/blog/957/ai-frontiers-part-43-embeddings-as-infrastructure-search-clustering-drift/05 Sep 2024 12:00:00 GMTThe unglamorous layer under every retrieval system: how embedding models are trained, why dense similarity is not relevance, what breaks at index scale, and how embedding drift quietly rots a search product.https://www.bobhuang.com/blog/956/ai-frontiers-part-42-evaluation-driven-development-build-your-own-test-set/https://www.bobhuang.com/blog/956/ai-frontiers-part-42-evaluation-driven-development-build-your-own-test-set/01 Aug 2024 12:00:00 GMTThe one practice that separates teams that improve from teams that churn: a small, private, versioned evaluation set built from your own traffic, with a written rubric and a cost column.https://www.bobhuang.com/blog/955/ai-frontiers-part-41-voice-interfaces-asr-tts-latency-budget/https://www.bobhuang.com/blog/955/ai-frontiers-part-41-voice-interfaces-asr-tts-latency-budget/11 Jul 2024 12:00:00 GMTSpeech recognition is largely solved and speech synthesis is largely solved. The hard part is the hundred milliseconds of turn-taking between them, which is why good voice interfaces are rarer than good speech models.https://www.bobhuang.com/blog/954/ai-frontiers-part-40-data-curation-deduplication-clean-beats-big/https://www.bobhuang.com/blog/954/ai-frontiers-part-40-data-curation-deduplication-clean-beats-big/06 Jun 2024 12:00:00 GMTThe part of pretraining that decides the model's ceiling and gets the least attention. Deduplication, quality classifiers and the biases they encode, multilingual coverage, and the provenance problem underneath it all.https://www.bobhuang.com/blog/953/ai-frontiers-part-39-gpu-supply-chain-cost-of-a-training-cluster/https://www.bobhuang.com/blog/953/ai-frontiers-part-39-gpu-supply-chain-cost-of-a-training-cluster/02 May 2024 12:00:00 GMTAccelerators, high-bandwidth memory, advanced packaging, interconnect, power and permits. Why the constraints on frontier training are industrial rather than algorithmic, and what that does to the market.https://www.bobhuang.com/blog/952/ai-frontiers-part-38-training-infrastructure-flash-attention-fsdp/https://www.bobhuang.com/blog/952/ai-frontiers-part-38-training-infrastructure-flash-attention-fsdp/04 Apr 2024 12:00:00 GMTTraining a frontier model is a distributed systems problem wearing a machine learning costume. Data, tensor, pipeline and expert parallelism, activation recomputation, and the failure rates that decide whether a run finishes.https://www.bobhuang.com/blog/951/ai-frontiers-part-37-long-tail-of-licensing-what-open-weights-permit/https://www.bobhuang.com/blog/951/ai-frontiers-part-37-long-tail-of-licensing-what-open-weights-permit/07 Mar 2024 12:00:00 GMTTwo models both described as open can carry obligations that differ in every dimension a business cares about. Reading the licenses, the training data provenance problem, and the rights questions courts had not yet answered.https://www.bobhuang.com/blog/950/ai-frontiers-part-36-benchmarks-and-contamination-measuring-honestly/https://www.bobhuang.com/blog/950/ai-frontiers-part-36-benchmarks-and-contamination-measuring-honestly/08 Feb 2024 12:00:00 GMTA benchmark that becomes a training target stops measuring capability. How contamination happens, how to detect it, and what a private evaluation set buys you that no leaderboard can.https://www.bobhuang.com/blog/949/ai-frontiers-part-35-mistral-mixtral-efficiency-first-open-ecosystem/https://www.bobhuang.com/blog/949/ai-frontiers-part-35-mistral-mixtral-efficiency-first-open-ecosystem/04 Jan 2024 12:00:00 GMTA 7B model with sliding-window attention, then a sparse mixture-of-experts that beat much larger dense models at a fraction of the active compute. Why efficiency became the open ecosystem's organizing principle.https://www.bobhuang.com/blog/948/ai-frontiers-part-34-prompt-injection-vulnerability-nobody-can-patch/https://www.bobhuang.com/blog/948/ai-frontiers-part-34-prompt-injection-vulnerability-nobody-can-patch/07 Dec 2023 12:00:00 GMTInstructions and data share one channel, so text a model reads can steer it. Why remote code execution is the right frame, what the research established in 2023, and the defenses that reduce rather than eliminate the risk.https://www.bobhuang.com/blog/947/ai-frontiers-part-33-mechanistic-interpretability-inside-the-black-box/https://www.bobhuang.com/blog/947/ai-frontiers-part-33-mechanistic-interpretability-inside-the-black-box/02 Nov 2023 12:00:00 GMTInduction heads, superposition, sparse autoencoders and circuits that can be edited: what a decade of trying to read a neural network actually produced, and why it matters more than a visualization.https://www.bobhuang.com/blog/946/ai-frontiers-part-32-serving-is-the-product-paged-attention-batching/https://www.bobhuang.com/blog/946/ai-frontiers-part-32-serving-is-the-product-paged-attention-batching/05 Oct 2023 12:00:00 GMTvLLM's memory manager took an operating-system idea and made inference several times cheaper. Why the serving layer, not the model, decides a product's unit economics.https://www.bobhuang.com/blog/945/ai-frontiers-part-31-autogpt-and-the-first-agent-wave/https://www.bobhuang.com/blog/945/ai-frontiers-part-31-autogpt-and-the-first-agent-wave/07 Sep 2023 12:00:00 GMTIn spring 2023 a script that looped a language model against itself became the fastest-growing repository in GitHub history. Why the autonomous agent dream arrived early, failed publicly, and still changed how we build.https://www.bobhuang.com/blog/944/ai-frontiers-part-30-tool-use-arrives-function-calling-plugins/https://www.bobhuang.com/blog/944/ai-frontiers-part-30-tool-use-arrives-function-calling-plugins/10 Aug 2023 12:00:00 GMTIn June 2023 models learned to ask for a function by name with typed arguments. Plugins tried to build an app store on top, and failed. What survived was the primitive: a model that can act rather than only answer.https://www.bobhuang.com/blog/943/ai-frontiers-part-29-open-weight-leap-llama-alpaca-ecosystem/https://www.bobhuang.com/blog/943/ai-frontiers-part-29-open-weight-leap-llama-alpaca-ecosystem/06 Jul 2023 12:00:00 GMTMeta released LLaMA to researchers in February 2023 under a gated license, and the weights circulated within a week. What followed — Alpaca, Vicuna, RedPajama, Dolly, StarCoder — set the template for open models for years.https://www.bobhuang.com/blog/942/ai-frontiers-part-28-diffusion-models-how-image-generation-works/https://www.bobhuang.com/blog/942/ai-frontiers-part-28-diffusion-models-how-image-generation-works/15 Jun 2023 12:00:00 GMTFrom nonequilibrium thermodynamics to latent diffusion and ControlNet: the mechanics of denoising, classifier-free guidance, why latent space made image generation affordable, and the licensing mess underneath.https://www.bobhuang.com/blog/941/ai-frontiers-part-27-alignment-before-the-assistant-era-rlhf/https://www.bobhuang.com/blog/941/ai-frontiers-part-27-alignment-before-the-assistant-era-rlhf/18 May 2023 12:00:00 GMTReinforcement learning from human feedback is why language models became assistants. It also optimizes against a proxy, which is why reward hacking, sycophancy and overoptimization were predicted before they were observed.https://www.bobhuang.com/blog/940/ai-frontiers-part-26-hallucination-failure-mode-that-will-not-go-away/https://www.bobhuang.com/blog/940/ai-frontiers-part-26-hallucination-failure-mode-that-will-not-go-away/13 Apr 2023 12:00:00 GMTWhy a model that has read the internet invents citations: the sources of hallucination, the calibration evidence, and the mitigations that actually work — retrieval, verification, and knowing when to abstain.https://www.bobhuang.com/blog/939/ai-frontiers-part-25-instruction-tuning-flan-to-alpaca/https://www.bobhuang.com/blog/939/ai-frontiers-part-25-instruction-tuning-flan-to-alpaca/30 Mar 2023 12:00:00 GMTA 137B model that had been instruction-tuned beat a 175B model that had not. How FLAN, T0, InstructGPT and the Flan Collection turned a base model into an assistant — and why Alpaca cost $600.https://www.bobhuang.com/blog/938/ai-frontiers-part-24-prompt-is-not-the-product-in-context-learning-limits/https://www.bobhuang.com/blog/938/ai-frontiers-part-24-prompt-is-not-the-product-in-context-learning-limits/16 Feb 2023 12:00:00 GMTIn early 2023 prompt engineering looked like a discipline and a moat. The research said otherwise: random labels work, order matters more than wording, and in-context learning looks like task location rather than instruction following.https://www.bobhuang.com/blog/937/ai-frontiers-part-23-where-the-frontier-goes-next/https://www.bobhuang.com/blog/937/ai-frontiers-part-23-where-the-frontier-goes-next/17 Sep 2026 12:00:00 GMTThe last entry in the series: the four resources that bound progress, why evaluation is now the binding constraint, what the open-weights diffusion means for moats, and what I would build next.https://www.bobhuang.com/blog/936/ai-frontiers-part-22-constitutional-ai-scalable-oversight/https://www.bobhuang.com/blog/936/ai-frontiers-part-22-constitutional-ai-scalable-oversight/16 Jul 2026 12:00:00 GMTRLHF does not scale, and the field knows it. From preference learning to AI feedback, debate, self-critique and deliberation training — the methods for supervising systems that humans can no longer fully check.https://www.bobhuang.com/blog/935/ai-frontiers-part-21-synthetic-data-model-collapse-debate/https://www.bobhuang.com/blog/935/ai-frontiers-part-21-synthetic-data-model-collapse-debate/21 May 2026 12:00:00 GMTIf models train on model output, do they degrade? The recursion argument, the accumulation counterargument, verification as the real fix, and the distillation economy that made synthetic data unavoidable.https://www.bobhuang.com/blog/934/ai-frontiers-part-20-rag-to-agentic-rag-evolution/https://www.bobhuang.com/blog/934/ai-frontiers-part-20-rag-to-agentic-rag-evolution/19 Mar 2026 12:00:00 GMTRetrieval-augmented generation grew from one embedding lookup into a planning loop with query rewriting, self-critique and graph structure. What changed, what it costs, and when long context beats retrieval outright.https://www.bobhuang.com/blog/933/ai-frontiers-part-19-kv-cache-engineering-memory-wall/https://www.bobhuang.com/blog/933/ai-frontiers-part-19-kv-cache-engineering-memory-wall/15 Jan 2026 12:00:00 GMTLong context is not a model feature, it is a memory budget. The arithmetic of the KV cache, grouped-query attention, latent attention, paging, prefix caching and eviction — and why serving is where context length is actually decided.https://www.bobhuang.com/blog/932/ai-frontiers-part-18-speculative-decoding-faster-inference-for-free/https://www.bobhuang.com/blog/932/ai-frontiers-part-18-speculative-decoding-faster-inference-for-free/20 Nov 2025 12:00:00 GMTDraft, then verify: the only widely deployed trick that makes inference two to three times faster without changing the output distribution at all. How it works, where it stops working, and what it reveals about serving.https://www.bobhuang.com/blog/931/ai-frontiers-part-17-quantization-running-70b-on-a-laptop/https://www.bobhuang.com/blog/931/ai-frontiers-part-17-quantization-running-70b-on-a-laptop/18 Sep 2025 12:00:00 GMTFrom GPTQ and AWQ to GGUF k-quants and 1.58-bit BitNet: how weight quantization actually works, what it costs in quality, and the memory arithmetic that decides what fits on your desk.https://www.bobhuang.com/blog/930/ai-frontiers-part-16-test-time-scaling-reasoning-models/https://www.bobhuang.com/blog/930/ai-frontiers-part-16-test-time-scaling-reasoning-models/17 Jul 2025 12:00:00 GMTDeepSeek-R1 published its reinforcement-learning recipe in January 2025 and the reasoning-model playbook stopped being a secret. What actually works, where the compute goes, and when thinking longer is a waste.https://www.bobhuang.com/blog/929/ai-frontiers-part-15-computer-use-gui-agents/https://www.bobhuang.com/blog/929/ai-frontiers-part-15-computer-use-gui-agents/15 May 2025 12:00:00 GMTWhen there is no API, the screen becomes the interface. How screenshot-driven agents perceive, ground and act — and why per-step reliability, not intelligence, is the wall they hit.https://www.bobhuang.com/blog/928/ai-frontiers-part-14-agentic-coding-turn/https://www.bobhuang.com/blog/928/ai-frontiers-part-14-agentic-coding-turn/20 Mar 2025 12:00:00 GMTSWE-bench went from 4% to over 60% in eighteen months and coding assistants grew hands. What actually changed in the agent loop, what the productivity evidence supports, and why review is now the bottleneck.https://www.bobhuang.com/blog/927/ai-frontiers-part-13-deepseek-v3-cost-efficiency-shock/https://www.bobhuang.com/blog/927/ai-frontiers-part-13-deepseek-v3-cost-efficiency-shock/16 Jan 2025 12:00:00 GMTDeepSeek-V3 reported a frontier-class training run for about $5.6M of GPU time. Reading the technical report: MLA, fine-grained experts, auxiliary-loss-free balancing, FP8, and what the headline number really measures.https://www.bobhuang.com/blog/926/ai-frontiers-part-12-model-context-protocol-plumbing-agent-era/https://www.bobhuang.com/blog/926/ai-frontiers-part-12-model-context-protocol-plumbing-agent-era/05 Dec 2024 12:00:00 GMTAnthropic published the Model Context Protocol on 25 November 2024. Why a JSON-RPC standard for tools was overdue, what it specifies, and why its security model is the interesting part.https://www.bobhuang.com/blog/925/ai-frontiers-part-11-openai-o1-inference-time-compute/https://www.bobhuang.com/blog/925/ai-frontiers-part-11-openai-o1-inference-time-compute/19 Sep 2024 12:00:00 GMTOn 12 September 2024 OpenAI shipped o1 — a model that spends more compute thinking before it answers. The research behind it: chain of thought, process reward models, and the economics of buying capability at inference time.https://www.bobhuang.com/blog/924/ai-frontiers-part-10-multimodal-turns-one-vision-audio-gpt-4o/https://www.bobhuang.com/blog/924/ai-frontiers-part-10-multimodal-turns-one-vision-audio-gpt-4o/18 Jul 2024 12:00:00 GMTGPT-4o landed a year after GPT-4 first accepted an image. How CLIP, Flamingo and LLaVA taught language models to see, why audio is the harder modality, and what multimodality changes in production.https://www.bobhuang.com/blog/923/ai-frontiers-part-9-small-language-models-phi-gemma-desktop-class/https://www.bobhuang.com/blog/923/ai-frontiers-part-9-small-language-models-phi-gemma-desktop-class/16 May 2024 12:00:00 GMTBy mid-2024 a 3.8B model could match GPT-3.5-class benchmarks. Microsoft's Phi line, Google's Gemma, and the data-quality thesis behind them — plus what small models change for cost, privacy, and architecture.https://www.bobhuang.com/blog/922/ai-frontiers-part-8-mixture-of-experts-memory-feasible-frontier/https://www.bobhuang.com/blog/922/ai-frontiers-part-8-mixture-of-experts-memory-feasible-frontier/21 Mar 2024 12:00:00 GMTMixtral made mixture-of-experts famous in December 2023: more total knowledge, same per-token compute. The 1991 idea behind it, what Mixtral actually is, and why MoE trades your memory bill for your GPU-hours bill.https://www.bobhuang.com/blog/921/ai-frontiers-part-7-context-window-wars/https://www.bobhuang.com/blog/921/ai-frontiers-part-7-context-window-wars/18 Jan 2024 12:00:00 GMTContext windows grew 100x in three years. Why position encodings were the bottleneck, how long-context models actually extend, and why bigger windows changed RAG economics without ending it.https://www.bobhuang.com/blog/920/ai-frontiers-part-6-vector-databases-infrastructure-layer/https://www.bobhuang.com/blog/920/ai-frontiers-part-6-vector-databases-infrastructure-layer/16 Nov 2023 12:00:00 GMTIn 2023 an infrastructure category appeared, raised hundreds of millions, and became the default neighbor of every LLM deployment. What a vector database actually does, why approximate nearest-neighbor search is the interesting part, and whether the category survives Postgres.https://www.bobhuang.com/blog/919/ai-frontiers-part-5-rag-before-it-was-everywhere/https://www.bobhuang.com/blog/919/ai-frontiers-part-5-rag-before-it-was-everywhere/21 Sep 2023 12:00:00 GMTRAG spent 2023 going from an Oxford paper to the default enterprise pattern. Where it came from, why it beats fine-tuning for knowledge, and what the first year of production deployments got wrong.https://www.bobhuang.com/blog/918/ai-frontiers-part-4-llama-2-open-weights-turning-point/https://www.bobhuang.com/blog/918/ai-frontiers-part-4-llama-2-open-weights-turning-point/20 Jul 2023 12:00:00 GMTJuly 2023: Meta ships Llama 2 with a license businesses can actually sign. What the release contained, why the license terms were the real story, and how a chaotic month of benchmarks reshaped the open-model ecosystem.https://www.bobhuang.com/blog/917/ai-frontiers-part-3-lora-and-qlora-fine-tuning-single-gpu/https://www.bobhuang.com/blog/917/ai-frontiers-part-3-lora-and-qlora-fine-tuning-single-gpu/01 Jun 2023 12:00:00 GMTTwo papers turned fine-tuning from a datacenter-only activity into something a person with one GPU can do: LoRA (2021) and QLoRA (May 2023). What they changed, how they work, and what the open-model ecosystem looked like the week Guanaco shipped.https://www.bobhuang.com/blog/916/ai-frontiers-part-2-gpt-4-emergent-abilities-debate/https://www.bobhuang.com/blog/916/ai-frontiers-part-2-gpt-4-emergent-abilities-debate/04 May 2023 12:00:00 GMTGPT-4 arrived in March 2023, and two months later a paper argued its boldest abilities were a mirage. This article works through both sides of the emergence debate and what it means for people building on these models.https://www.bobhuang.com/blog/915/ai-frontiers-part-1-attention-is-all-you-need-five-years-later/https://www.bobhuang.com/blog/915/ai-frontiers-part-1-attention-is-all-you-need-five-years-later/19 Jan 2023 12:00:00 GMTA working engineer re-reads 'Attention Is All You Need' in 2023: what the transformer actually changed, what it merely enabled, and why the paper's 2017 vocabulary makes it harder to read today than it should be.