- Compatibility: This version requires EVA App v2.7.0 or higher.
Performance Enhancements
- TorchServe log profile setting function added: Separate development and operational Log4j2 profiles were added to maintain full visibility of models and metrics in the development environment and reduce model log output to warning/error level in the production environment.
- Model log observability improvements: Added Python logger and model name metadata to TorchServe MODEL_LOG output to allow administrators to identify the handler that created each model.log entry.
- Maintain access and metrics visibility for latency analysis: Maintain ACCESS_LOG, TS_METRICS, MODEL_METRICS in both log profiles so you can compare HTTP latency to TorchServe HandlerTime, Queue, and System metrics.
- Add log profile support when starting a service: Add
-log-profile dev|prod option to run.sh, default to development logging, include production environment Docker startup arguments, per-log retention cleanup, and safer shell execution behavior.
- Proxy logging adjustments per profile: Allows you to control proxy INFO log sampling at startup, resulting in full INFO logging in development environments and sampled INFO logging in production environments.
- Explicit use of RT-DETR V2 fast processor: Explicitly load RTDetrImageProcessorFast with option
use_fast=True to avoid processor fallback ambiguity.
Bug Fixes
- TorchServe handler log provenance fix: TorchServe reformatted Python worker log records before capturing them, fixing an issue where model.log was merging all handler output into an untraceable MODEL_LOG stream.
- IG processor device placement fix: Fixed passing
device=self.device when calling the IG processor so that the tensors generated in the embedding and inference flow match the handler device.
- Fix TorchServe pynvml warning repeat output: TorchServe system metrics pynvml deprecation Suppressed FutureWarning from repeatedly polluting errors.log.
- Fix archive/runtime packaging of logging utilities: Included shared logging utilities in eva_ts/utils in model archive extra files, keeping the handlers directory to contain only handler files.