MLCosting

This project aims to improve the estimation of computational costs for DSS requests at ECMWF. Currently, cost estimation relies on manual or rule-based processes that may not reflect the true complexity of requests, leading to inefficient resource allocation and potential QoS issues. The proposed solution is MLCosting, a machine-learning-based API plug-in for DSS cost estimation. It will analyze request parameters (JSON-based) and metadata (user, queue status, etc.) using tabular ML models trained on historical data. MLCosting will predict execution time and dataset size, classifying requests into load tiers (light/medium/heavy/denied) to help prioritize processing.

Follow the developments on Github.