Writing & Research

Technical Articles

Technical writing on large language models, model architectures, PyTorch implementations, GPU computing, and machine learning systems. These articles document concepts I explore, implement, and learn.

Selected articles

Understanding DeepSeek-V3: Architecture and Design

An exploration of the architecture behind modern language models, including Mixture of Experts, attention design, and efficient model computation.

Technical deep dive Read article ↗

Multi-Token Prediction in DeepSeek-V3

Understanding the MTP training objective, its transformer modules, shared embeddings, projection layers, and contribution to language-model training.

Architecture & implementation Read article ↗

Building Transformer Models from Scratch

A practical look at transformer components, attention mechanisms, normalization, embeddings, and training a model using PyTorch.

Implementation guide Read article ↗

Understanding Tensor Parallelism

How tensor parallelism distributes linear layers across GPUs, including row-parallel and column-parallel computation and communication between devices.

Systems engineering Read article ↗

A Practical Guide to LLM Training

A hands-on explanation of dataset preparation, tokenization, training configuration, loss tracking, and evaluating language-model experiments.

Practical guide Read article ↗

No articles in this category yet.