Property-Based Testing: Beyond Unit Tests in Elixir

The Limits of Example-Based Tests

Traditional unit tests verify behavior for specific inputs: "given input X,

expect output Y." This works well for known edge cases but leaves vast regions

of the input space unexplored. Property-based testing inverts the approach:

define properties that must hold for all valid inputs, then let the

framework generate hundreds of random inputs to verify them.

The Prismatic Platform's TACH doctrine mandates property-based tests for all

pure, stateless modules -- validators, parsers, encoders, and type constructors.

ExUnitProperties Basics

Elixir's property testing library is ExUnitProperties, backed by StreamData

for data generation:

defmodule PrismaticDD.Scoring.ValidatorPropertyTest do
use ExUnit.Case, async: true
use ExUnitProperties


property "confidence scores are always between 0.0 and 1.0" do
check all score <- float(min: 0.0, max: 1.0),
label <- string(:alphanumeric, min_length: 1) do
result = Validator.validate_confidence(%{score: score, label: label})
assert {:ok, validated} = result
assert validated.score >= 0.0
assert validated.score <= 1.0
end
end
end

The check all macro generates random values matching the generators and

runs the block for each combination. By default, it runs 100 iterations,

configurable via max_runs.

Writing Good Generators

StreamData provides primitive generators that compose into complex structures:

# Primitive generators
integer()                    # any integer
float(min: 0.0, max: 1.0)  # bounded float
string(:alphanumeric)        # alphanumeric string
binary()                     # random binary data


# Composite generators
list_of(integer())           # list of integers
map_of(atom(:alphanumeric), string(:utf8))  # map with atom keys


# Custom generators for domain types
def entity_generator do
gen all name <- string(:alphanumeric, min_length: 1, max_length: 100),
type <- member_of([:person, :company, :domain, :address]),
confidence <- float(min: 0.0, max: 1.0) do
%{name: name, type: type, confidence: confidence}
end
end

The gen all macro (from StreamData) creates a generator that produces

maps matching your domain model. These generators are reusable across tests.

Properties Worth Testing

Roundtrip Properties

Encoding followed by decoding should return the original value:

property "JSON roundtrip preserves data" do
check all data <- map_of(string(:alphanumeric), string(:utf8)) do
assert data == data |> Jason.encode!() |> Jason.decode!()
end
end

Idempotency Properties

Applying an operation twice should produce the same result as applying it once:

property "normalizing a slug is idempotent" do
check all input <- string(:alphanumeric, min_length: 1) do
once = Slug.normalize(input)
twice = Slug.normalize(once)
assert once == twice
end
end

Invariant Properties

Certain conditions must always hold regardless of input:

property "validated entities always have a non-empty name" do
check all entity <- entity_generator() do
case Validator.validate(entity) do
{:ok, validated} -> assert String.length(validated.name) > 0
{:error, _} -> :ok
end
end
end

Shrinking: Finding Minimal Failing Cases

When a property test fails, StreamData automatically shrinks the failing

input to the smallest value that still triggers the failure. This makes

debugging dramatically easier.

For example, if a parser fails on a 200-character string, shrinking might

reduce it to a 3-character string that exposes the same bug. Custom generators

inherit shrinking behavior from their component generators.

TACH Doctrine Integration

The TACH doctrine classifies modules by their testing requirements:

Module TypeRequired TestsExample

|-------------|---------------|---------|

Pure/statelessProperty tests (ExUnitProperties)Validators, parsers, encoders Stateful (GenServer)Unit tests + state transition testsRegistries, caches LiveView/ControllerUnit + E2E (Wallaby)Page components Cross-appIntegration tests (@tag :integration)Pipeline coordinators

Pure modules are identified by having no side effects: no database calls, no

PubSub broadcasts, no file I/O. These are the best candidates for property

testing because they are deterministic and fast.

Performance Considerations

Property tests run 100 iterations by default. For computationally expensive

properties, reduce iterations:

property "expensive validation holds", max_runs: 25 do
check all input <- complex_generator() do
assert expensive_validation(input)
end
end

For CI, the platform runs property tests with max_runs: 200 to increase

confidence. Local development uses the default 100 for faster feedback.

Real Impact

Property-based testing has caught bugs in the Prismatic Platform that

example-based tests missed entirely: Unicode normalization edge cases in

slug generation, floating-point precision issues in confidence score

aggregation, and off-by-one errors in pagination boundary calculations.

These bugs would have reached production without property testing.