- Compatibility: This version requires EVA App v2.7.0 or higher.
Performance Enhancements
- Helm-based TorchServe configuration flow organized: Helm ConfigMap's config.properties.template has been mounted to the /app/eva_ts/config.properties.template path, so that a runtime configuration file is created based on the deployed template.
- Helm's model and worker settings are reflected at runtime: The load model, number of workers, batch size, and batch delay time set in Helm values.yaml have been improved to be reflected in the generated config.properties.
- Added TorchServe runtime control options: Added number_of_netty_threads and worker_retry_timeout_sec entries to the TorchServe settings template rendered in Helm.
- Improved Kubernetes service defaults: Changed the default service type from NodePort to ClusterIP, and modified nodePort to be rendered only when the service type is NodePort.
- Adjusted basic serving resources and model parameters: For more stable runtime configuration, the default memory request/limit was increased to 32Gi, the number of workers for some models was adjusted to 1, and the placement settings of RT-DETRV2 were changed.
Bug Fixes
- Fixed an issue where Helm values settings were not reflected in TorchServe runtime settings: Fixed an issue where model and worker settings in Helm values.yaml could be ignored due to a mismatch between the deployed settings file path and the runtime settings creation flow.
- Modified torch.compile behavior for dynamic input in RT-DETRV2: Applied the dynamic=True option when compiling RT-DETRV2 to support various input sizes more reliably.
- Fix missing IG model archive dependency: Included interest_zone.py in IG model archive settings to enable interest-zone logic at runtime.