Constant Time Testing

Constant Time Testing is a community skill for empirically testing whether code executes in constant time, covering statistical timing measurement, test harness creation, dudect methodology, result interpretation, and regression testing for timing side-channel prevention in security code.

What Is This?

Overview

Constant Time Testing provides patterns for statistically verifying that code execution time does not depend on secret inputs. It covers statistical timing measurement that collects high-precision timing data across many executions with different inputs, test harness creation that sets up controlled environments for accurate timing measurement, dudect methodology that applies the fix-vs-random approach comparing timing distributions for fixed and random secret inputs, result interpretation that uses statistical tests to determine if timing differences are significant, and regression testing that monitors for constant-time property violations across code changes. The skill enables empirical verification that cryptographic code resists timing attacks by catching leaks that static analysis cannot reliably identify.

Who Should Use This

This skill serves cryptographic library developers testing implementations against timing leakage, security teams validating constant-time properties in production code, and embedded systems engineers testing timing-sensitive security functions such as key derivation and signature verification routines.

Why Use It?

Problems It Solves

Static analysis cannot detect all timing variations introduced by hardware and compilers. Source code review misses microarchitectural timing effects. Compiler optimizations may introduce branches not visible in source code. Proving constant-time properties requires empirical measurement on target hardware.

Core Highlights

Timing harness collects nanosecond-precision measurements across thousands of executions. Dudect tester compares timing distributions using Welch t-test statistics. Result analyzer reports whether timing differences are statistically significant. Regression runner integrates timing tests into CI pipelines.

How to Use It?

Basic Usage

import time
import random
import numpy as np
from scipy import stats

class TimingTester:
  def __init__(
    self,
    func,
    input_size: int = 32,
    iterations: int\
      = 10000
  ):
    self.func = func
    self.input_size =\
      input_size
    self.iterations =\
      iterations

  def measure(
    self, data: bytes
  ) -> float:
    start = time\
      .perf_counter_ns()
    self.func(data)
    end = time\
      .perf_counter_ns()
    return end - start

  def run_dudect(
    self
  ) -> dict:
    fixed_input = bytes(
      [0] * self.input_size)
    fixed_times = []
    random_times = []

    for _ in range(
        self.iterations):
      random_input =\
        random.randbytes(
          self.input_size)
      if random.random()\
          < 0.5:
        fixed_times.append(
          self.measure(
            fixed_input))
      else:
        random_times.append(
          self.measure(
            random_input))

    t_stat, p_value =\
      stats.ttest_ind(
        fixed_times,
        random_times)
    return {
      't_statistic':
        float(t_stat),
      'p_value':
        float(p_value),
      'constant_time':
        abs(t_stat) < 4.5}

Real-World Examples

import hmac

def test_hmac_compare():
  key = b'secret_key_here'
  msg = b'test_message'
  expected = hmac.new(
    key, msg,
    'sha256').digest()

  def compare(data):
    hmac.compare_digest(
      expected, data)

  tester = TimingTester(
    compare,
    input_size=32,
    iterations=50000)
  result = tester\
    .run_dudect()

  assert result[
    'constant_time'], (
    f'Timing leak detected: '
    f't={result["t_statistic"]'
    f':.2f}')

Advanced Tips

Run timing tests with at least 100000 iterations for reliable statistical power to detect small timing differences. Disable CPU frequency scaling and turbo boost during measurements for consistent results. Use percentile-based cropping to remove outliers caused by context switches and interrupts before statistical analysis. When testing on Linux systems, setting the CPU governor to performance mode and isolating cores with the isolcpus kernel parameter can significantly reduce measurement variance.

When to Use It?

Use Cases

Verify that a password comparison function shows no timing difference based on where characters diverge. Test that an AES implementation executes in constant time across all key values. Add timing regression tests to a cryptographic library CI pipeline to catch compiler-induced leaks.

Related Topics

Timing side channels, dudect, constant-time verification, cryptographic testing, and security testing.

Important Notes

Requirements

High-precision timer access for nanosecond measurement. scipy library for statistical testing. Controlled test environment with minimal background process interference. Root or administrator access for disabling CPU frequency scaling during measurement collection.

Usage Recommendations

Do: run tests on the actual target hardware where the code will execute in production. Collect enough samples for statistical significance before drawing conclusions. Test after each compiler or optimization flag change.

Don't: run timing tests on shared or virtualized hardware where other processes introduce noise. Conclude constant-time behavior from a small number of measurements. Skip testing on different CPU architectures where behavior may differ.

Limitations

Statistical testing can only provide confidence rather than mathematical proof of constant-time execution. Operating system interrupts and scheduling introduce timing noise that reduces measurement precision. Results on one CPU model may not transfer to different architectures, particularly when moving between x86 and ARM platforms. Virtual machine and container environments introduce additional timing jitter that makes measurements less reliable than bare-metal testing.