Experiments

Adaptive experience vs. the static CTA

Half of visitors see the normal fundraising call to action. The other half see the intent-aware experience. This is how a team would read that comparison, and what it must not conclude too early.

Fictional data throughout. The figures on this page were written by hand to show the layout. They are not results and are not a forecast. This prototype makes no claim that adaptive giving raises more money.

Design

Adaptive experience vs. static fundraising CTA

Allocation: 50% control · 50% adaptive
Assigned by: A random id stored on the visitor’s device. Nothing about the person.
Primary metric: Donation conversion rate, completed gifts per exposed session
Window: Sample 21-day test window

Sample-ratio check

50.1%Control · 18,420 sessions49.9%Adaptive · 18,310 sessions

Split looks right z = 0.57. If traffic isn’t divided as configured, every other number is suspect.

Is the difference real?

Donation conversion

Control

0.96%176 of 18,420

Adaptive

1.11%203 of 18,310

Relative difference

+16.0%p = 0.15

Not conclusive (p = 0.15). A gap this size appears by chance about 15 times in 100 even when the two experiences perform identically. There is no winner yet, and this page does not suggest one.

Decide the sample size and primary metric before the test starts, and don’t stop early because a number looks good.

Two-sided two-proportion z-test on completed gifts per exposed session.

Metrics

Control vs. adaptive

Experiment metrics by arm, Sample data · fictional, for illustration only
Metric	Control	Adaptive	Note
Sessions exposed	18,420	18,310	Everyone assigned to the arm.
Impressions of the fundraising slot	18,420	12,480	Lower for adaptive: 5,830 sessions were deliberately left unchanged.
CTA engagement (any CTA)	552 · 3.0% of impressions	1,219 · 9.8% of impressions	Adaptive counts softer clicks (learn, action, help) too. Not comparable to a gift ask.
Donation starts	361	412
Donation completions	176	203
Donation conversion (of exposed)	0.96%	1.11%	The primary metric.
Revenue	$9,152	$10,962
Average gift	$52	$54
Recurring gifts	29 · 16% of gifts	41 · 20% of gifts
Action completions	84	197
Email sign-ups	141	268

Sample data · fictional, for illustration only.

Over time

Completed gifts per day

Day-to-day movement is noisy and both arms rise toward the campaign peak on day 21. Look for a consistent gap across many days, not a single good one. The same numbers are in the table below the chart.

Completed gifts by day

Control
Adaptive

View as a table

Completed gifts by day and arm (sample data)
Day	Control	Adaptive
Day 1	6	6
Day 2	5	5
Day 3	7	5
Day 4	6	7
Day 5	5	6
Day 6	7	7
Day 7	7	7
Day 8	7	8
Day 9	6	8
Day 10	7	7
Day 11	8	9
Day 12	7	10
Day 13	7	9
Day 14	7	11
Day 15	8	11
Day 16	8	10
Day 17	8	12
Day 18	10	12
Day 19	12	12
Day 20	10	14
Day 21	28	27

Funnel

From exposed session to completed gift

Control

Sessions exposed18,420 100.0%
Clicked a CTA552 3.0%
Started a gift361 2.0%
Completed a gift176 1.0%

Adaptive

Sessions exposed18,310 100.0%
Clicked a CTA1,219 6.7%
Started a gift412 2.3%
Completed a gift203 1.1%

Bars share one scale, so they can be compared across arms. Clicks include any CTA, not only gift asks.

Adaptive arm only

What each experience did

Adaptive arm performance by experience shown, Sample data · fictional, for illustration only
Experience shown	Shown	Share	Clicks	Click rate	Outcome	Count
Learn	5,264	42%	237	4.5%	Read the impact section	237
Take Action	3,010	24%	301	10.0%	Actions completed	197
Donate	1,840	15%	177	9.6%	Gifts completed	118
Subscribe	1,060	8%	94	8.9%	Email sign-ups	88
Receive Help	1,306	10%	410	31.4%	Fundraising asks shown	0
Left unchanged	5,830	Sessions where the model wasn’t confident enough. The normal page was shown and no ask was added.

Click rates differ by design: a learn prompt and a gift ask are different asks, so ranking experiences by click rate would mislead.

Guardrails

What must not get worse

Kill switch →

Holding
No fundraising asks to help-seeking sessions
0 asks shown across 1,306 help sessions. This is enforced by the policy, not just measured: service pages carry no asks in either arm.
By design
Unsure means unchanged
5,830 of 18,310 adaptive sessions (32%) were left alone. Low confidence never forces an ask.
Not in sample
Dismissals and unsubscribes do not rise
This is a configured guardrail (Unsubscribe / dismissal rate does not rise) but the sample data doesn't include it, so nothing is claimed. A real test must measure it before any rollout.
Control
Adaptive behavior can be turned off
The kill switch on the overview returns every visitor to the static CTA immediately.

Live from this browser

What you just did on the public site

No events yet. Open the red wolf page, interact with the adaptive box and page (add ?present=1 to the address for Demo controls). Impressions, clicks and simulated gifts show up here, stored only in this browser.