Câu 37: NCP-AAI: Agentic AI

You are rolling out a multimodal conversational agent on NVIDIA’s stack: the model is containerized as a TensorRT-LLM engine, served via Triton Inference Server behind NIM microservices for routing and scaling, and protected by NeMo Guardrails for safety and compliance. During early testing, end-to-end latency exceeds your target budget, and you need to tune batching, model precision, and guardrail checks while maintaining both throughput and enforcement of safety policies. Which configuration change is most effective for reducing latency under these constraints while still enforcing NeMo Guardrails policies?

1. A. Quantize the TensorRT-LLM engine to FP16, tune Triton’s dynamic batching, and integrate NeMo Guardrails alongside inference to run policy checks in parallel. — đáp án hiện tại
2. B. Quantize the TensorRT-LLM engine to INT8, disable dynamic batching, and invoke Guardrails checks synchronously within the inference path.
3. C. Deploy separate Triton servers for model inference and guardrail validation, routing requests sequentially and merging outputs at the application layer.
4. D. Keep FP32 precision, increase batch size aggressively, and perform Guardrails checks in a downstream microservice after inference.