Senior Ml Engineer

Senior ML Engineer automation and integration for advanced machine learning tasks

Category: productivity Source: alirezarezvani/claude-skills

Senior ML Engineer is a community skill for applying experienced machine learning engineering practices, covering system design, experiment management, production deployment, monitoring, and team collaboration patterns.

What Is This?

Overview

Senior ML Engineer provides patterns that reflect production ML engineering experience across the full model lifecycle. It covers ML system architecture design, reproducible experiment management, feature store integration, model serving infrastructure, production monitoring with drift detection, and cross-team collaboration on ML projects. The skill enables engineers to apply battle-tested practices that avoid common pitfalls in ML system development.

Who Should Use This

This skill serves ML engineers transitioning from research prototypes to production systems, teams establishing ML engineering standards and best practices, and technical leads making architecture decisions for ML platform infrastructure.

Why Use It?

Problems It Solves

Models that work in notebooks fail in production due to environment differences and missing dependencies. Experiment results cannot be reproduced because configurations and data versions were not tracked. Deployed models degrade silently when input data distributions change over time. Teams duplicate effort building similar ML infrastructure without shared patterns and abstractions.

Core Highlights

System design patterns structure ML applications with clear boundaries between data, training, and serving components. Experiment tracking captures all parameters, data versions, and metrics for complete reproducibility. Production monitoring detects data drift, prediction quality degradation, and infrastructure issues. Feature management ensures consistent feature computation between training and serving.

How to Use It?

Basic Usage

from dataclasses import dataclass, field
from datetime import datetime
import json
from pathlib import Path

@dataclass
class Experiment:
    name: str
    model_type: str
    hyperparams: dict = field(default_factory=dict)
    metrics: dict = field(default_factory=dict)
    data_version: str = ""
    timestamp: str = ""

    def __post_init__(self):
        if not self.timestamp:
            self.timestamp = datetime.now().isoformat()

class ExperimentTracker:
    def __init__(self, log_dir: str):
        self.log_dir = Path(log_dir)
        self.log_dir.mkdir(parents=True, exist_ok=True)
        self.experiments: list[Experiment] = []

    def log_experiment(self, experiment: Experiment):
        self.experiments.append(experiment)
        path = self.log_dir / f"{experiment.name}.json"
        path.write_text(json.dumps({
            "name": experiment.name,
            "model": experiment.model_type,
            "params": experiment.hyperparams,
            "metrics": experiment.metrics,
            "data": experiment.data_version,
            "time": experiment.timestamp
        }, indent=2))

    def best_experiment(self, metric: str) -> Experiment:
        return max(self.experiments,
                   key=lambda e: e.metrics.get(metric, 0))

Real-World Examples

from dataclasses import dataclass, field

@dataclass
class ModelMonitor:
    model_name: str
    baseline_metrics: dict = field(default_factory=dict)
    alerts: list[str] = field(default_factory=list)

    def check_drift(self, current_metrics: dict,
                    threshold: float = 0.1) -> dict:
        drifted = []
        for metric, baseline_val in self.baseline_metrics.items():
            current_val = current_metrics.get(metric, 0)
            change = abs(current_val - baseline_val) / max(
                abs(baseline_val), 1e-6)
            if change > threshold:
                drifted.append({"metric": metric,
                    "baseline": baseline_val,
                    "current": current_val,
                    "change": round(change, 4)})
        if drifted:
            self.alerts.append(
                f"Drift detected in {len(drifted)} metrics")
        return {"drifted": drifted, "healthy": len(drifted) == 0}

class MLSystemDesign:
    def __init__(self, project_name: str):
        self.project_name = project_name
        self.components: dict[str, dict] = {}

    def add_component(self, name: str, role: str,
                      dependencies: list[str]):
        self.components[name] = {
            "role": role, "dependencies": dependencies}

    def validate_dependencies(self) -> dict:
        missing = []
        for name, comp in self.components.items():
            for dep in comp["dependencies"]:
                if dep not in self.components:
                    missing.append(f"{name} needs {dep}")
        return {"valid": len(missing) == 0, "missing": missing}

Advanced Tips

Implement feature computation as shared libraries used by both training pipelines and serving infrastructure to prevent train-serve skew. Set up automated retraining triggers based on drift detection alerts rather than fixed schedules. Use shadow deployments to compare new model versions against production before switching traffic.

When to Use It?

Use Cases

Design a production ML system with clear separation between feature engineering, training, and serving components. Establish experiment tracking standards that enable any team member to reproduce previous results. Build a monitoring dashboard that alerts on model performance degradation and data distribution shifts.

Related Topics

MLOps practices, feature store design, model serving architectures, experiment tracking systems, and ML system reliability engineering.

Important Notes

Requirements

An experiment tracking system for logging parameters, metrics, and artifacts. Model serving infrastructure with versioning and rollback capability. Monitoring and alerting tools configured for ML-specific metrics.

Usage Recommendations

Do: version everything including data, code, configurations, and model artifacts for complete reproducibility. Establish baseline metrics before deploying new models to enable meaningful comparison. Document data assumptions and preprocessing steps alongside model code.

Don't: deploy models without monitoring that detects performance degradation in production. Skip offline evaluation assuming that training metrics predict production performance. Build custom infrastructure for common ML operations when established tools are available.

Limitations

Best practices add process overhead that may slow rapid prototyping phases. Not all ML engineering patterns apply equally to every project scale and maturity level. Infrastructure investment in monitoring and tracking requires ongoing maintenance resources.