Мониторинг задач HTCondor и памяти GPU с помощью Prometheus и Grafana

Оригинал

Monitoring HTCondor Jobs and GPU Memory with Prometheus and Grafana

Part 5 of 6 in Building and Operating an HTCondor Pool for Research Computing. One RTX 3060 on our lab dashboard showed 4.18 GiB of free VRAM. On the same worker, the unclaimed HTCondor slot advertised 0 GPUs. A job requesting one GPU could not match that slot at the time of the snapshot. To investigate an idle job, I check its scheduler first, then the resources the worker advertises, and finally the physical card. What each measurement covers Question Source Scope Is the jo