Flags and experiments — Voidhash Docs

These tools let an agent create and roll out feature flags, fix them for one person, and run A/B tests from draft to a concluded winner.

Early access

Feature flags and A/B tests are in early access. Ask us on Discord, or email support@voidhash.com on a paid plan, to turn them on for your team.

Feature flags

Key access: Feature flags.

Tool	What it does
`list_flags`	Flags with their state in production, whether test follows it, their variants, and the experiment that sets each.
`get_flag`	One flag with who gets it in production and in test: overrides, a running experiment, rules, then everyone else.
`create_flag`	Adds a flag, off unless given a rollout.
`update_flag`	Turns a flag on or off or changes its rollout, in production or in test, or its key or variants. Asks first when people lose it.
`set_flag_override`	Fixes a flag, or an experiment arm, for one person, in production or in test.
`set_flag_rules`	Sets an environment's rules and what everyone else gets. A rule matches people by name or by conditions on their traits and app, and says what share of them get the flag. Asks first in production.
`evaluate_flags`	What one person gets from each flag in production or in test, and why; `context` says what their app would report. It only looks: it puts nobody in an experiment.
`archive_flag`	Archives a flag. Asks first when it's on.
`restore_flag`	Restores an archived flag as it was.

A flag decides separately in production and in test, which apps using a test key read. Test follows production until a tool changes something in it with environment: "test"; update_flag with followProduction: true drops test's own configuration again.

A rule in set_flag_rules matches the people it names in people, and everyone all of its conditions hold for. A condition compares a trait of the person (by key), or what their app reports: platform, app_version, os_version, country or locale, with in, not_in, gte, lt, is_set or is_not_set. "Give the new checkout to half of the people in Germany on version 2.4 or newer" becomes one rule with two conditions at 50%. evaluate_flags reads a person's traits itself; pass context for conditions on the app, which otherwise count as not set.

Experiments

Key access: Experiments.

Tool	What it does
`list_experiments`	A/B tests with their status, arms, split and flag values.
`get_experiment`	One experiment in depth, with what can be done next.
`create_experiment`	Sets up a draft experiment.
`update_experiment`	Changes its setup. Asks first when changing arms after it started.
`start_experiment`	Starts or resumes it. Asks first.
`pause_experiment`	Pauses it: everyone sees the control arm.
`conclude_experiment`	Ends it for good with a winner. Asks first.
`get_experiment_results`	Each arm's conversions and revenue against the control, with a verdict.
`archive_experiment`	Archives a draft or concluded experiment.
`restore_experiment`	Restores an archived experiment.

An arm shows a paywall at a location, gives feature flags a value, or both. create_experiment and update_experiment take an arm's flag values as flags, each naming the flag by key with isEnabled and, optionally, one of its variants; a flag one arm names, every arm must. An experiment can test a flag alone, with no paywall. While it runs, the flag takes the value of the person's arm, after its switch and overrides and before its rules, and concluding leaves the winner's value for everyone the flag's own rules and overrides don't name.

create_experiment and update_experiment take the goal event as primaryMetric. Have the agent set it: left empty, results count purchase_completed, an event Voidhash never sends, so every arm shows 0 conversions. See Conversions for which built-in event to use.

Example

Test whether showing the annual plan first at onboarding raises trial starts.

The agent calls list_paywalls, then create_experiment for a draft with primaryMetric set to $subscription.created, which also fires when a trial starts. Then it calls start_experiment, which asks first. Later, get_experiment_results reports each arm against the control.

Next steps

How a flag decides: environments, overrides, rules and rollouts.
A/B test results: conversions and verdicts.
Analytics and ads: measure the results.