Get Available Resources

Get Available Resources is a community skill for discovering and listing available system resources programmatically, covering compute inventory, storage capacity, network endpoints, service discovery, and resource monitoring for infrastructure management.

What Is This?

Overview

Get Available Resources provides patterns for querying and inventorying available computational resources across local and cloud environments. It covers CPU, memory, and disk capacity detection on local machines, GPU availability and properties for compute workload planning, cloud resource listing for instances, storage buckets, and managed services, service endpoint discovery for locating running APIs and databases, and resource health monitoring that tracks utilization and availability. The skill enables DevOps engineers and developers to build tools that dynamically discover and allocate resources based on current availability.

Who Should Use This

This skill serves DevOps engineers building infrastructure discovery and inventory tools, developers writing applications that adapt to available compute resources, and platform teams creating resource monitoring dashboards.

Why Use It?

Problems It Solves

Applications that hardcode resource assumptions fail when deployed to environments with different capacities. Manual resource inventory becomes outdated as infrastructure scales and changes. Workload scheduling without resource awareness leads to overcommitment and performance degradation. Service discovery in dynamic environments requires automated endpoint resolution.

Core Highlights

System profiler detects CPU cores, memory capacity, and disk space across operating systems. GPU detector identifies available accelerators with driver and memory information. Cloud lister queries provider APIs for instances, storage, and services. Health monitor tracks resource utilization over time.

How to Use It?

Basic Usage

import os
import platform
import shutil

def get_system_resources() -> dict:
    cpu_count = os.cpu_count() or 1
    try:
        import psutil
        mem = psutil.virtual_memory()
        memory_gb = round(mem.total / (1024**3), 1)
        memory_used_pct = mem.percent
    except ImportError:
        memory_gb = 0
        memory_used_pct = 0
    disk = shutil.disk_usage("/")
    return {
        "platform": platform.system(),
        "cpu_cores": cpu_count,
        "memory_gb": memory_gb,
        "memory_used_pct": memory_used_pct,
        "disk_total_gb": round(
            disk.total / (1024**3), 1),
        "disk_free_gb": round(
            disk.free / (1024**3), 1)}

def get_gpu_resources() -> list[dict]:
    gpus = []
    try:
        import torch
        for i in range(torch.cuda.device_count()):
            props = torch.cuda.get_device_properties(i)
            gpus.append({
                "name": props.name,
                "memory_gb": round(
                    props.total_mem / (1024**3), 1),
                "compute": f"{props.major}.{props.minor}"})
    except ImportError:
        pass
    return gpus

sys_info = get_system_resources()
print(f"CPUs: {sys_info['cpu_cores']}")
print(f"RAM: {sys_info['memory_gb']}GB")

Real-World Examples

import os
import shutil

class ResourceMonitor:
    def __init__(self):
        self.snapshots = []

    def snapshot(self) -> dict:
        resources = get_system_resources()
        resources["gpus"] = get_gpu_resources()
        self.snapshots.append(resources)
        return resources

    def check_capacity(self,
                        min_memory_gb: float,
                        min_disk_gb: float,
                        need_gpu: bool = False
                        ) -> dict:
        current = self.snapshot()
        checks = {
            "memory_ok": current["memory_gb"]
                >= min_memory_gb,
            "disk_ok": current["disk_free_gb"]
                >= min_disk_gb}
        if need_gpu:
            checks["gpu_ok"] = (
                len(current["gpus"]) > 0)
        checks["all_ok"] = all(checks.values())
        return checks

monitor = ResourceMonitor()
capacity = monitor.check_capacity(
    min_memory_gb=8, min_disk_gb=50,
    need_gpu=True)
print(f"Capacity check: {capacity}")

Advanced Tips

Run resource checks at application startup to adapt configuration like batch sizes and thread counts to available hardware. Collect resource snapshots periodically to track utilization trends over time. Use capacity checks as gate conditions in CI/CD pipelines to prevent running resource-intensive jobs on undersized runners.

When to Use It?

Use Cases

Build a workload scheduler that assigns tasks based on available CPU, memory, and GPU resources. Create an infrastructure inventory tool that reports current capacity across a fleet of servers. Implement a pre-flight checker that validates resource requirements before launching data processing jobs.

Related Topics

System monitoring, resource management, infrastructure automation, capacity planning, and compute resource discovery.

Important Notes

Requirements

Python with os and shutil modules from the standard library. Optional psutil for detailed memory and CPU monitoring. Optional PyTorch or pynvml for GPU detection and querying. System administrator privileges may be required for some hardware inventory operations.

Usage Recommendations

Do: cache resource detection results when hardware does not change frequently to avoid repeated system calls. Use capacity checks before allocating work to prevent out-of-memory failures. Log resource snapshots alongside job results for debugging.

Don't: assume resource availability is constant in dynamic cloud environments where instances scale. Use GPU detection libraries without fallback handling for CPU-only environments. Poll system resources at very high frequency, which adds unnecessary overhead.

Limitations

Resource detection accuracy depends on the operating system and installed monitoring libraries. Cloud resource listing requires provider-specific SDKs and authentication. Containerized environments may report host resources rather than container limits without cgroup-aware detection.