Skip to main content

Run and evaluate an A/B test

Split a page's traffic between variants, read which one is leading, and decide when to pause, promote, or keep testing.

Written by Ryan

Every page in a funnel can run several variants at once. granvl splits the page's visitors between its live variants by weight and reports results per variant, so you can see which version earns more leads, and better leads.

How the traffic split works

  • Only live variants share traffic. Archived variants get none and show a dash in the Traffic column.

  • The weights of a page's live variants must total 100%. A variant at 0% is paused: it stays on the page with its history but gets no new visitors.

  • A visitor keeps seeing the same variant of a page for 30 days, as long as that variant still gets traffic. If it's paused or archived, they're assigned again.

  • Archive takes a variant out of the split and sets its weight to 0. Its numbers are kept. A variant with 20 page views or fewer is deleted instead, since there is no history worth keeping.

  • A duplicated variant starts at 0%, so it never takes traffic before you've reviewed it.

  • A page can hold up to 15 variants, paused ones included. Archived variants don't count.

Set up a test

  1. Open the funnel. On Overview, pick the page in the page tabs above the variants table.

  2. On the variant you want to beat, click Duplicate. The copy is named "(Copy)" and starts at 0%.

  3. Change one meaningful thing in the copy: the headline and angle, the offer, or the form structure. Click Edit to open it in the page editor, or ask your agent.

  4. Click Preview to check the copy.

  5. Type each variant's share in the Traffic column. The footer shows the total; click Save weights once it reads 100%.

To rename a variant, click its name, type, and press Enter.

You can also hand the whole setup to your agent:

On my mortgage funnel's quiz page, create a variant with a question-style headline, split traffic 50/50 with the current variant, and give me a preview link.

The agent uses create_variant or duplicate_variant, then set_weights.

Protect the test

Editing a variant that already has results mixes old and new copy under one name, and the numbers stop meaning anything. When your agent tries to change the content of a variant that has at least one conversion and either 100 or more visitors, 5 or more conversions, or paid ad traffic since its last reset, granvl stops the edit until you confirm. The agent will offer to duplicate the variant instead. Prefer that.

After a big change you want to measure from a clean start, use Reset statistics in the funnel's ··· menu. Variant totals start from zero; the chart keeps its history.

Test before you send traffic

Ask your agent for a test link to run the live pages end to end without sending anything to your CRM or ad platforms:

Create a test link for this funnel so I can submit the form on the live pages.

The agent uses create_test_link. The link works for 24 hours. Forms submit and the whole flow runs, including validation, qualification and the thank-you redirect, but nothing is delivered to your destinations, no pixel or conversion event fires, every analytics row is tagged as test, and the SMS verification popup is skipped.

When you're done, ask the agent to run cleanup_test_data so test rows don't linger in your stats. A test link can't prove your CRM field mapping works; use send_test_lead for that.

Read the results

The variants table on Overview covers the date range picked above it.

The variants table with traffic split, visitors, conversions, qualified leads and conversion rate

Column

What it shows

Variant

The letter and name, with Edit, Preview, Duplicate and Archive. A Leading chip marks the current leader.

Traffic

The variant's share of the page's visitors.

SMS 2FA

Whether SMS phone verification is on for this variant. See Validate phone numbers and email addresses.

Visitors

Unique visitors in the range.

Conversions or Click-throughs

The page's goal: conversions on Quiz and Form pages, click-throughs on a Landing page in a multi-step funnel.

Qualified

Leads that passed your qualification rule. Shows a dash until the funnel has a rule.

CVR or CTR

Goal count divided by visitors. The leader's rate is highlighted.

Tick Show archived in the footer to see archived variants. The footer also shows the page totals and which metric picks the leader.

Conversions or qualified leads

Metric

What it counts

Conversions

Every submission (or converting click) on the variant.

Qualified

Leads that passed your qualification rule, or that your CRM or a teammate marked as qualified.

Booked, Held, Won

Leads your CRM reported at that stage, once your CRM sends stage updates.

A variant that wins on raw conversions can lose on qualified leads. Pick the measure that matches what you pay for in Analytics > Lead quality > Leading metric:

  • Automatic: uses the richest signal the funnel has. Won rate once any lead is won, Booked rate once your CRM reports bookings, Qualified rate once you have a rule, Conversion rate otherwise.

  • Conversion rate · filled the form

  • Qualified rate · passed your rule

  • Booked rate · appointment set, Held rate · appointment held, Won rate · deal closed: available once your CRM has reported those stages.

The Leading chip and the highlighted rate follow this choice. Setting up a rule and CRM stages is covered in Qualify leads and track what happens next.

When granvl calls a winner, and when it won't

The dashboard never declares a winner. Leading marks the live variant with the best rate on your leading metric right now. granvl labels it "an observed rate, not a confirmed test winner", because with few visitors the leader can change from one day to the next.

When you ask your agent what to do, it uses granvl's checks on the same data:

Signal

When it appears

Promote and pause suggestions (get_optimization_suggestions)

At least two variants in rotation each have 50 or more visitors, and at least one converts at under half the leader's rate.

Start a test (get_optimization_suggestions)

A Landing, Quiz or Form page has 100 or more visitors but only one variant in rotation.

Test is ready insight (list_insights)

Two variants each have 400 or more visitors and the difference is statistically significant (p < .05).

Scale variant insight (list_insights)

Two variants each have 500 or more visitors and the leader converts at 1.4 times the next best or more.

Test stuck insight (list_insights)

The test has run 14 days or more, two variants each have 200 or more visitors, and the difference still isn't significant.

When none of these apply, the answer is "keep testing". granvl reports that the test is still gathering data rather than guessing.

What to change and when

The granvl skill gives your agent these rules of thumb:

Decision

When

Any pause or promote call

Not before each variant has about 100 visitors.

Promote a winner to 100%

The winner has 500 or more visitors, converts at 1.4 times the next best or more, and the lead holds on mobile and desktop separately.

Archive a variant

After it has sat at 0% for at least two weeks and you no longer need it.

Before promoting, ask your agent to check the result by audience. A variant can win overall and lose on your biggest segment:

Before we promote the leader on my quiz page, check whether it still wins on mobile and desktop separately.

The agent uses get_cohort_comparison. To promote, set the winner to 100% and the others to 0%, then start the next test from the winner.

Change one big idea per test (the angle, the offer, the number of quiz steps, or the form) so you know what caused the difference.

Record what you learned

When a test ends, ask your agent to save the result:

Save what we learned from this test: the question headline beat the benefit headline on the quiz page.

The agent uses save_learning with the outcome (winner, loser or inconclusive), the hypothesis and the before and after values. Later, get_research_history returns past wins, dead ends to avoid, and open questions, so new tests build on old ones instead of repeating them.

Related articles

Did this answer your question?