cuda

Gemma 4 on an 2021 4 GB Laptop GPU: QAT Takes It From 9.5 GiB to 1.6

Quantization-aware training shrinks Gemma 4 memory to run on 4 GB laptops. · for ml engineers

Do Drafters Work on Small Models? Gemma 4 E2B and E4B on Two Small GPUs

Benchmarks show Gemma models run on small GPUs, guiding low-cost deployment decisions. · for ml engineers, platform teams

A 4 GB Laptop GPU vs a 6-Core CPU on Gemma 4, Re-Measured in ABBA Order: 4.1x

Benchmark compares 4 GB GPU vs 6-core CPU on Gemma 4 using llama.cpp and CUDA. · for ml engineers, performance testers

Can a QAT 12B Fit an 8 GB GPU? Gemma 4 12B on an RTX 3050

Shows 12B model can run on 8 GB GPU, guiding resource planning for ML workloads. · for ml engineers, devops

Gemma 4 on a Tesla T4, Part 3: Int4 Embeddings Serve E2B in 2.86 GiB at 2.30x bf16

Gemma 4 on T4 delivers int4 embeddings at 2.30× bf16 speed in 2.86 GiB. · for ml engineers

CUDA for AMD on Windows

AMD GPUs now support CUDA on Windows, affecting developers using NVIDIA-centric code. · for GPU developers

Hot Chips 2026: CUDA targets RISC-V

CUDA adding RISC-V support expands GPU compute to open-source hardware. · for gpu developers, hardware engineers

The Cheapest CUDA GPU on AWS Has an Arm CPU — and You Probably Want the Intel One

Cheapest AWS CUDA GPU uses Arm CPU, but Intel may be preferable for performance. · for ml ops

Geolocating Random Islet Image Using Geometry & CUDA GPU Programming

Demonstrates CUDA geometry for geolocating island images, useful for GPU-accelerated spatial analysis. · for GPU programmers, data engineers

Gemma 4 in Pure JAX: What Changes Between Turing and Ada, and What Doesn't

Gemma 4 on JAX highlights differences between Turing and Ada GPUs. · for ml engineers

Gemma 4 on a 4 GB GPU in Plain PyTorch: From Out of Memory to Faster Than llama.cpp

Gemma 4 runs on 4 GB GPU with PyTorch quantization, reducing hardware needs. · for ml engineers, AI startups

g5g vs g6 for LLM Serving: the Same Code, and 3.7x the Throughput

g5g vs g6 comparison shows 3.7x throughput boost for LLM serving on same code. · for ml engineers

Advanced GPU Optimization: How to tech an LLM with CUDA and ROCm? - Part 5 (Final Part)

Guide to accelerate large language models using CUDA and ROCm GPU stacks. · for AI engineers, ML researchers

Can a 26B MoE Run on an 8 GB GPU? Gemma 4 26B A4B on an RTX 3050

Gemma 26B runs on an RTX 3050 via llama.cpp and CUDA, showing consumer-grade inference is feasible. · for ML engineers, AI researchers

A Dense 31B on an 8 GB GPU: Gemma 4 31B on an RTX 3050

Running a 31B Gemma model on consumer GPUs shows feasibility of low-cost inference. · for ml engineers, hardware developers

Hot Chips 2026: CUDA Targets RISC-V – By Chester Lam

CUDA adds support for RISC-V, expanding GPU acceleration to open-source CPU architectures. · for chip designers

Geolocating a random island using geometry and CUDA programming

Demonstrates using CUDA for island geolocation, useful for high-performance spatial calculations. · for geospatial engineers

Is &p->member well-defined for a device pointer p in CUDA host code?

Clarifies the semantics of device-pointer member access in CUDA host code. · for cuda developers

JAX FFI handler does not find my CUDA function exported with nanobind

JAX FFI cannot locate CUDA functions exported via nanobind, blocking GPU acceleration. · for ml engineers

How to get the address of a CUDA/C++ struct member without dereferencing the member

Explains how to obtain a CUDA/C++ struct member address without dereferencing. · for systems programmers, GPU developers