DeepSeek-V3-Inspired Implementation
Exploring Mixture of Experts (MoE), Multi-head Latent Attention (MLA), multi-token prediction, and language model training.
View repository ↗Research & Engineering
I'm interested in large language models, efficient model architectures, training systems, and machine learning infrastructure. I use this space to share my implementations, experiments, open-source contributions, and technical writing.
Implementations, experiments, and engineering work.
Exploring Mixture of Experts (MoE), Multi-head Latent Attention (MLA), multi-token prediction, and language model training.
View repository ↗Add a concise description of your project, the problem it solves, its technical approach, and your experimental results.
View repository ↗Contributions to machine learning libraries and infrastructure.
Explore my pull requests, bug fixes, implementation improvements, and technical investigations. Each contribution should link directly to the relevant issue or pull request.
View GitHub activity ↗Articles and explanations about LLMs, research, and engineering.
Replace this description with the actual topic and key takeaway from one of your published articles.
Read article ↗Browse my writing on attention mechanisms, model architectures, training, and machine learning systems.
Read all articles ↗My interests include LLM architecture design, efficient training, GPU computing, distributed systems, and open-source machine learning. I enjoy understanding systems from first principles, implementing technical ideas, and investigating performance bottlenecks.