AdPix lift test: how to measure whether a budget change worked

An AdPix lift test tells you whether one specific change, such as raising a channel’s budget on a given date, caused extra conversions. You enter the action and its date, AdPix fits a model on the days before that date, forecasts what would have happened if you had done nothing, and reports the gap with a 95% confidence interval and a plain verdict. A test needs about two months of history, runs for 7, 14 or 28 days and is included from the Starter plan up. This page covers setup, the settings that change accuracy, how to read the result and when a lift test is the wrong tool.

Lift test is a measurement of the incremental effect of a marketing change: the observed result after the change minus a counterfactual estimate of what the result would have been without it.

What does an AdPix lift test answer?

An AdPix lift test answers one question about one action: did this change, on this date, move the metric I care about, and by how much? The unit is an action, not a channel. “We raised the Google Ads budget by 30% on 3 March” is a lift question. “What does Google Ads return across the year?” is a marketing mix modeling question (AdPix docs: run a lift test).

The reason to run one is that attribution reports cannot answer it. Attribution assigns credit to the touches a buyer saw, so it tends to credit channels that catch demand which already existed. Gordon and colleagues compared 15 large advertising experiments at Facebook with the observational methods advertisers commonly use, and found the observational estimates often failed to match the experimental effect, even after controlling for extensive demographic and behavioral data (Gordon et al., 2019). The incrementality testing guide explains that gap in general terms; this page is about running the test in AdPix.

Who should run lift tests in AdPix?

Lift tests suit any team that changes budgets, launches or pauses campaigns, or swaps creative, and wants to know whether the change paid. Because a test needs only about two months of history, it is usually the first measurement study a site can run. The AdPix guide to choosing between the two studies answers “Almost always Lift” for teams starting out (AdPix docs: MMM or Lift).

It fits best when:

• You make one clear change at a time and know the date it took effect.
• The target metric happens daily, such as purchases, leads or sign-ups, so there is enough signal to forecast.
• The change is material. A 5% budget nudge is usually too small to separate from normal day-to-day swings.

How does the method behind a lift test work?

A time-based lift test builds a counterfactual: a forecast of the metric for the days after the change, based only on data from before it. The incremental lift is the sum of observed minus forecast over the measurement window. Brodersen and colleagues at Google formalized this approach with Bayesian structural time-series models, which combine trend, seasonality and control series that the change did not affect (Brodersen et al., 2015).

Two things make or break this kind of test in general. The pre-period has to be long and stable enough to learn the metric’s weekly rhythm, and something unaffected by the change should act as a control, so that a market-wide swing is not credited to your action. Randomized holdout experiments are still the stronger design where you can run them, because they do not depend on a forecast; a time-based test is the practical option when you cannot randomize who sees an ad.

How does a lift test work in AdPix?

In AdPix you open Lift (incrementality) under Marketing measurement and click + New test. All definitions below come from the AdPix documentation.

1. Define the action. Pick the action type: budget increase or decrease, campaign start or stop, channel added or removed, creative change, or other.
2. Action date. The day the change actually took effect. This is the only required field.
3. Channel and, optionally, Campaign. The channel sets the spend window used for cost per acquisition and incremental ROAS.
4. Measure. Choose a Target metric, any event your site records or revenue, and a Measurement period of 7, 14 or 28 days.
5. Create test. A test whose window has not ended waits as Scheduled and runs once the window plus two days of settling for late conversions has passed. Scoring an unfinished window would produce a false negative lift.

Which settings change the accuracy of the result?

Every new or re-run test uses a placebo-calibrated engine. AdPix runs placebo tests on dates through the past year, dates where nothing changed, to learn how much the method tends to see effects that are not there, and shows the calibration source on the result: the site’s own history, the pooled network, or the model alone. A result counts as significant only when p is at most 5% and the interval excludes zero.

How do you size a test before you change the budget?

Size it first with Check sensitivity. In the new-test drawer you enter the Planned spend change (%), and AdPix returns the minimum detectable effect (MDE), the statistical power for your planned change if a marketing mix model run has measured the channel, and the shortest of the 7, 14 or 28-day windows that reaches 80% power.

A hypothetical store shows why this step matters. It sells about 130 orders a day, and its search channel spends $400 a day. The team plans to raise that to $520 a day, a 30% increase, from 3 March.

• Suppose the check returns an MDE of 6% for a 14-day window. Over 14 days the store expects about 130 × 14 = 1,820 orders, so the test can only see an effect of roughly 0.06 × 1,820 ≈ 109 extra orders or more.
• The extra spend is $120 × 14 = $1,680. If each incremental order is worth $45 in revenue, 109 orders is about $4,900, an incremental ROAS of about 2.9 on the extra spend.
• If the team’s break-even ROAS is 2.0, a real but smaller effect, say 90 extra orders ($4,050, ROAS about 2.4), would still be profitable, yet the 14-day test might not see it. The fix is the 28-day window or a bigger change, not hoping.

The A/B test sample size guide explains MDE and power in more depth, and the same logic applies here.

How do you read an AdPix lift result?

Read three things in order: the verdict, the interval, then the quality grade. The verdict sits at the top of the result and follows the 95% interval (AdPix docs).

Next to it, a detectability label states the minimum detectable effect: High confidence under 5%, Medium under 10%, Low with wide bands under 20%, Underpowered over 20%. The quality grade runs from A (calibrated, little placebo bias, clean window, material spend change) to D (unreliable, never sent to the marketing mix model). A warning mark on a significant result means it fails a false-discovery check across the site’s tests from the last 180 days.

Back to the hypothetical store. After 14 days the counterfactual is 1,820 orders and the store recorded 1,960.

• Incremental orders: 1,960 − 1,820 = 140, a lift of 140 ÷ 1,820 ≈ 7.7%.
• Incremental cost per order on the extra spend: $1,680 ÷ 140 = $12.
• Incremental revenue at $45 an order: 140 × $45 = $6,300, so incremental ROAS on the extra spend is $6,300 ÷ $1,680 = 3.75.
• If the 95% interval on incremental orders runs from +30 to +250, the whole interval is above zero, so the verdict is Positive. Even the low end, 30 orders and $1,350, is short of break-even at ROAS 2.0, which is worth knowing before you scale further.

The result’s effect cards show the incremental lift, incremental revenue, ad spend in the window, incremental ROAS and CPA, and the chart plots observed against counterfactual with the 95% band. To check the ROAS line against your margin, the calculator below includes break-even ROAS; the ROAS formula guide explains the arithmetic.

What does the Incrementality page add to lift tests?

The Incrementality page measures spend changes you never set up as tests. With Detect spend changes automatically on, which is the default, AdPix compares each channel’s and campaign’s median daily spend in the two weeks before a day with the week after it. A change of at least 20% that holds for at least five of the next seven days becomes an activity, and activities are grouped into 14-day episodes, each measured with one calibrated lift test (AdPix docs: Incrementality).

Per channel it reports incremental conversions per day, average and marginal incremental CPA, and a K factor, attributed conversions divided by incremental ones. A K factor above 1 means attribution over-credits the channel. A channel stays in Learning until it has four measured episodes with spend levels at least 30% apart, so the docs’ advice is practical: change budgets in steps of 20% or more and hold them for at least two weeks. Missing cost rows count as missing data, not zero spend.

Which plan includes AdPix lift tests?

Campaign lift measurement is included from the Starter plan, 499,000 toman a month or 4.9 million toman a year, and the Pro plan adds true campaign impact (incrementality) analysis (AdPix pricing). On the Free plan the lift page shows an upgrade screen (AdPix docs: plans and entitlements). Plans attach to a website, not to the account, and a website’s first paid plan starts with a 14-day free trial.

AdPix is a web analytics platform that replaces Google Analytics 4: realtime, acquisition, engagement, conversion and e-commerce reports, plus attribution, lift testing and marketing mix modelling, with no data sampling and a free plan. The AdPix features overview shows where lift sits next to the other features.

What are the limits of a lift test, and when should you use something else?

A lift test cannot separate changes that happen at the same time. If you raise the search budget and launch a sale on the same day, the lift belongs to both, and the test cannot split it. The lift result has an Other spend changes around this test section that lists material changes, at least 10% of total daily spend, within the window or the 14 days before it, so you can see the overlap.

Use something else when:

• You need the whole budget split. Lift measures one change. Spreading a yearly budget across channels is what the marketing mix model is for, and lift results calibrate it through Calibration experiments.
• The change is small or the metric is rare. If the sensitivity check returns Underpowered, a bigger change or a longer window is the fix.
• You can randomize exposure. A user- or region-randomized holdout run by an ad platform does not depend on a forecast, and is the stronger design when it is available.
• Your goal is not on the site. Brand recall or offline sales are outside what a website metric can show.

A useful habit from the docs: build a test on a date when nothing changed. It should come back Neutral, and if it does not, the metric is too noisy for the window you picked. The performance marketing guide puts lift, attribution and MMM side by side for budget decisions.

How do you run your first AdPix lift test?

Start with the next budget change you were going to make anyway.

1. Make sure the AdPix tag is on the site and your conversion event fires, and import ad cost for the channel if you want incremental ROAS and CPA.
2. Open + New test, enter the planned spend change and run Check sensitivity.
3. If the MDE is larger than the effect you expect, raise the change or choose the 28-day window.
4. Make the change, enter the action date the day it takes effect, and leave everything else unchanged for the window.
5. Read the verdict with its interval and grade, then send a grade A or B result to the marketing mix model.

A/B test calculator

Sample size and duration for your next experiment. The free plan covers 100,000 events a month, no credit card. Setup takes under five minutes.

Start freeTry it free