AI Distillation Engineer vs AI Model Compression Engineer

A AI Distillation Engineer Only

• Knowledge distillation theory and implementation (logit-based, feature-based, relation-based)
• PyTorch and Hugging Face Transformers for model training and evaluation
• Quantization techniques (GPTQ, AWQ, GGUF, INT8/INT4) and calibration data selection
• Model architecture analysis - attention mechanisms, MoE routing, layer redundancy
• Synthetic data generation using teacher models for curriculum-based distillation
• Evaluation methodology - benchmarking distilled models across perplexity, task accuracy, latency, and throughput
• ONNX export and TensorRT / vLLM inference optimization
• LoRA, QLoRA, and parameter-efficient fine-tuning as complementary techniques