San Jose, CA
Kunj Shah

Large Language Models: architecture, pre-training, and alignment safety.

I am a deep learning engineer focused on optimizing model capability, safety frameworks, and scaling efficiency. I research transformer pre-training architectures and develop safety-critical alignment frameworks to prevent prompt injections and tool exploitation.

Core Areas

Pre-training & Adaptation

Pre-training medical and domain-specific transformers using SwiGLU, Grouped Query Attention, and Rotary Embeddings. Architecting memory-efficient adaptations using LoRA/QLoRA on cloud clusters.

Alignment & Model Safety

Implementing alignment feedback loops (SFT, DPO, GRPO) for safety and instruction compliance. Developing inline BERT-based firewalls with ~20ms latency to filter prompt injections.

© 2025 Kunj Shah · Built with curiosity · San Jose, CA