• compatibility: This version targets EVA App v3.0.0 and above.

Performance Enhancements

• Add server-side ensemble prediction: Add /predictions/ensemble and support model-specific threshold, vote-based detection aggregation, and current-model result reporting.
• Added ROI-specific detection operation: Only boxes within the set interest area are returned as responses, improving the downstream VLM flow to receive only area-related detections.
• Improved EVA App synchronization flow: Changed POST /sync to be used as the source of truth for camera count, health snapshot, and dynamic worker scaling.
• Proxy module structure reorganization: Proxy was separated into route module, Pydantic schema, TorchServe service, health service, camera registry, and scaling policy module, and the wrapper for existing import compatibility was maintained.
• Added background health checker and probe: Tracks the latest health state based on FastAPI, TorchServe inference/management endpoint, model worker readiness, optional inference probe, accelerator detection, and system resource threshold.
• Added dynamic worker scaling policy: Camera count-based worker scaling, model-specific resource profile, min/max worker constraint, and optional GPU memory validation were introduced.
• Added CUDA MPS deployment support: Added GPU_MPS_ENABLED, MPS pipe/log directory handling, daemon start/stop in run.sh, Helm values and volume mount for MPS environments.
• Added MIG GPU deployment support: Added Helm configuration to set MIG resource name, GPU count, and CUDA visible devices.
• MMGDino and LLMDet inference optimization: Iterative zero-shot inference performance was improved by applying fixed image preprocessing bucket, GPU-side image preparation, cached text feature, and compiled model submodule.

Bug Fixes

• Fixed the issue of ensemble fallback to unusable models: Modified to exclude models that are not registered in TorchServe or do not have READY workers from ensemble candidates.
• Modified handling of non-serving models in Ensemble prediction: Check the TorchServe management status before sending an Ensemble request to prevent prediction flow from failing due to an unavailable model.
• MIG metric permission error handling modification: If the MIG memory/utilization metric cannot be viewed, it has been improved to handle it as missing optional data instead of resource failure.
• MIG UUID handling modification: Use visible MIG UUID so that GPU discovery operates correctly in a MIG partition environment.
• Fixed handling of missing MIG utilization metric: Allowed resource schema or health response to not fail even if there is no utilization field.
• Fixed minimum worker maintenance issue during sync: Fixed so that the set minimum worker count is maintained when EVA App sync updates the camera count.
• SYNC API request field name modification: Changed id in Sync request format to eva_id.
• RT-DETRV2 person target labeling fix: Modified RT-DETRV2's person-only detection label to be handled consistently.