Create a repeatable Vast.AI environment. Install build tools, CUDA/CUTLASS, SentencePiece, and HF libs. Run a reference model to capture tokens/s, latency, VRAM usage, and profiling artefacts for the baseline.

Investigate industry metrics (MLPerf, service SLIs), define the measurements to track, and build scripts that execute baselines across GPUs and workloads with structured output for later comparison.

Define boundaries between orchestrator, kernel modules, and telemetry. Stand up request queues, loader hooks, and a first-cut prefill path that reuses reference kernels while enabling swap-in of custom implementations later.

Add contiguous KV cache, greedy/top-k/top-p/temperature sampling, and end-to-end decode. Measure throughput and latency; validate against HF on short prompts.

Implement shared-memory tiled softmax/attention; test on 128-4096 tokens. Benchmark vs week-4 and capture Nsight metrics (occupancy, memory bandwidth).

Design a paged allocator (vLLM-style) handling variable seq lengths and in-flight batching. Add CUDA Graphs for the decode chain and micro-bench scheduling overhead.

Implement group-wise quantization with on-the-fly dequant in GEMM, compare AWQ/GPTQ/NF4 behavior and regressions.

Build prefill/decode queues with separate CUDA streams, overlap tokenization and H2D copies, and benchmark mixed prompt lengths.

Shard QKV/MLP across GPUs with NCCL all-reduce; add activation checkpointing for long contexts and measure scaling.

Enable a lookahead path with rollback, add an optional Medusa multi-token predictor, and log speedup ratios on long outputs.

Implement HTTP endpoints (/v1/completions and /v1/chat/completions), add batching inside the server, metrics, and KV eviction.

Run numerical parity checks, benchmark against llama.cpp/vLLM/TensorRT-LLM, stress test long prompts, and publish a thorough write-up with results and lessons learned.