Technical Support Specialist

Can a small model running locally help an engineer interpret messaging-system logs without recommending unverified or destructive changes? We are testing training and evaluation methods for that task.

Enterprise message brokers involve complex failure cascades — thread pool exhaustion, broker store limits, corrupted persistence indices, and cascading timeouts. Small language models tend to hallucinate plausible-sounding configurations or suggest dangerous administrative operations without isolating the root cause.

The research investigates a dual-tier knowledge architecture. Stable middleware concepts and structured diagnostic trees are trained directly into the student weights via sequence distillation, while volatile environment state, active logs, and site-specific configurations are handled by an isolated local retrieval system. Critical safety verifiers enforce strict evidence quotes and automatically quarantine any recommendation involving unsafe file operations.

We evaluate candidate models on an independent, engineer-curated held-out gold benchmark covering real incident categories: broker deadlocks, consumer starvation, persistence store exhaustion, and memory leaks. Every response is judged on whether it correctly isolates the primary bottleneck, provides exact grounding for every claim, and rejects unverified corrective actions.

The pipeline for data generation, verification, fine-tuning, quantization, and held-out evaluation is implemented. Whether the student achieves the required domain gain is still under evaluation.

Generalization across arbitrary legacy configurations and unseen custom network topologies remains an open research challenge. Synthetic and curated gold evaluations do not fully capture the noisy, ambiguous log inputs found in diverse production incidents. Distinguishing between model confidence and actual correctness remains under active investigation.