Model Merging

Model Merging is a community skill for combining multiple fine-tuned language models into a single model, covering merge strategies, weight interpolation, task vector arithmetic, and quality evaluation of merged outputs.

What Is This?

Overview

Model Merging provides patterns for combining the capabilities of multiple fine-tuned models without additional training. It covers linear weight interpolation, SLERP merging for smoother weight traversal, task vector arithmetic that adds or subtracts capabilities, and TIES merging that resolves parameter conflicts. The skill enables practitioners to create combined models from specialized fine-tunes.

Who Should Use This

This skill serves ML engineers combining domain-specific fine-tuned models into unified deployments, researchers exploring model combination techniques without expensive retraining, and teams creating versatile models by merging specialized adapters trained on different tasks.

Why Use It?

Problems It Solves

Deploying multiple specialized models requires proportionally more infrastructure. Multi-task training from scratch requires collecting data from all target domains simultaneously. Fine-tuning on one domain degrades performance on other domains. Retraining a model every time a new capability is needed is costly and time-consuming.

Core Highlights

Linear interpolation blends weights from two models using a mixing ratio to combine capabilities. SLERP merging traverses the weight space along a spherical path for smoother combinations. Task vector arithmetic computes the difference between fine-tuned and base weights, enabling addition and subtraction of learned behaviors. TIES merging resolves sign conflicts between task vectors to produce cleaner merged parameters.

How to Use It?

Basic Usage

from dataclasses import dataclass, field
import math

@dataclass
class ModelWeights:
    name: str
    params: dict[str, list[float]] = field(default_factory=dict)

class WeightMerger:
    def linear_merge(self, model_a: ModelWeights,
                     model_b: ModelWeights,
                     alpha: float = 0.5) -> ModelWeights:
        merged = ModelWeights(name=f"{model_a.name}+{model_b.name}")
        for key in model_a.params:
            if key in model_b.params:
                a_vals = model_a.params[key]
                b_vals = model_b.params[key]
                merged.params[key] = [
                    a * (1 - alpha) + b * alpha
                    for a, b in zip(a_vals, b_vals)
                ]
        return merged

    def task_vector(self, base: ModelWeights,
                    finetuned: ModelWeights) -> ModelWeights:
        vector = ModelWeights(name=f"tv_{finetuned.name}")
        for key in base.params:
            if key in finetuned.params:
                vector.params[key] = [
                    f - b for f, b in zip(
                        finetuned.params[key], base.params[key])
                ]
        return vector

Real-World Examples

from dataclasses import dataclass, field
import math

class AdvancedMerger(WeightMerger):
    def slerp_merge(self, model_a: ModelWeights,
                    model_b: ModelWeights,
                    t: float = 0.5) -> ModelWeights:
        merged = ModelWeights(name=f"slerp_{model_a.name}_{model_b.name}")
        for key in model_a.params:
            if key not in model_b.params:
                continue
            a = model_a.params[key]
            b = model_b.params[key]
            dot = sum(x * y for x, y in zip(a, b))
            norm_a = math.sqrt(sum(x ** 2 for x in a))
            norm_b = math.sqrt(sum(x ** 2 for x in b))
            cos_angle = dot / max(norm_a * norm_b, 1e-8)
            cos_angle = max(-1.0, min(1.0, cos_angle))
            angle = math.acos(cos_angle)
            if angle < 1e-6:
                merged.params[key] = a
                continue
            sa = math.sin((1 - t) * angle) / math.sin(angle)
            sb = math.sin(t * angle) / math.sin(angle)
            merged.params[key] = [
                sa * x + sb * y for x, y in zip(a, b)
            ]
        return merged

    def apply_task_vector(self, base: ModelWeights,
                          vector: ModelWeights,
                          scale: float = 1.0) -> ModelWeights:
        result = ModelWeights(name=f"merged_{base.name}")
        for key in base.params:
            if key in vector.params:
                result.params[key] = [
                    b + scale * v for b, v in zip(
                        base.params[key], vector.params[key])
                ]
        return result

Advanced Tips

Sweep the alpha parameter from 0.1 to 0.9 to find the optimal blend ratio. Use SLERP instead of linear interpolation when models have different weight magnitudes. Combine multiple task vectors with individual scaling factors.

When to Use It?

Use Cases

Merge a coding-focused fine-tune with a writing-focused fine-tune into a single versatile model. Create a bilingual model by combining monolingual fine-tunes without multilingual training data. Remove unwanted behaviors from a model by subtracting the corresponding task vector from the model weights.

Related Topics

Weight interpolation methods, task arithmetic for neural networks, model soup techniques, LoRA adapter merging, and multi-task model combination.

Important Notes

Requirements

Models sharing the same base architecture and tokenizer for compatible weight merging. Sufficient disk space and memory to load multiple model checkpoints simultaneously. Evaluation datasets for assessing merged model quality on target tasks.

Usage Recommendations

Do: evaluate merged models on benchmarks from each source model to verify capability retention. Start with linear interpolation at alpha 0.5 as a baseline before trying advanced methods. Keep the base model checkpoint for comparison and as a fallback.

Don't: merge models with different architectures or tokenizer vocabularies, as the weights are incompatible. Assume that merging always improves quality without running evaluation benchmarks. Apply task vectors with scales larger than 1.0 without careful testing, as this amplifies learned patterns beyond stable ranges.

Limitations

Merged models may not achieve the same quality as multi-task training on combined datasets. Merging more than two models increases the chance of destructive interference between parameters. No merging strategy guarantees preservation of all source model capabilities.