A/B test analysisData Analysis

A/B test results analysis with a clear verdict

Delivery from 1 dayRevisions includedQuality check
Packages
Payment protectionDetails
  • Revisions within the package are free.
  • If you do not respond within 3 days of delivery, the work is accepted automatically.
Order · $35

The money is reserved when you pay and released only once you accept the work.

A/B test results analysis with a clear verdict

Example result

Example for the Standard package

Data analysis: 9 months of bookings at a family-run guesthouse

Key numbers

MetricValue
Bookings (excl. cancelled)251
Room nights sold689
Revenue$78,540
Average rate per night$114
Cancellation rate11.6% (33 of 284)
Average stay2.7 nights

Summary by channel

ChannelBookingsShare of revenueAvg. rateCommissionNet per night
Booking platforms14254%$10915%$93
Direct (phone, email, website)8738%$1260%$126
Local travel agent228%$10210%$92

Direct guests pay more, stay longer (3.4 nights vs 2.3) and bring $33 more per night after commission than platform guests.

Trends and outliers

  • July and August made 41% of revenue; occupancy reached 93%.
  • March and April were weakest (occupancy 38%), but weekend nights still sold at 79%: the gap is midweek.
  • Cancellations were 19% for platform bookings with free cancellation and 4% for direct bookings with a deposit.
  • Outlier: 6 bookings in May at $45 per night. These are all one corporate group with a manual discount; they were kept in totals but flagged.

Findings

  1. Your most profitable guests are direct, returning guests, yet only 14% of platform guests were invited to book direct next time.
  2. Midweek in spring is the only real gap; weekends sell themselves.
  3. Free cancellation on platforms costs about 27 lost nights over 9 months.

Recommendations

  1. Add a "book direct" card at check-out with a 10% returning-guest rate: still $20 above platform net.
  2. Create a midweek spring offer (3 nights for the price of 2, Sun–Thu) for March–April.
  3. Switch platform listings to a non-refundable rate at 8% off alongside the flexible one.
  4. Track the booking channel for every guest from now on: 17 rows had no channel and were grouped as "direct".

All totals were cross-checked against the monthly sums in your sheet; the difference is $0.

Client: a 12-room guesthouse by a lake, run by two owners Data: 284 booking rows, January to September (date, room type, nights, channel, price paid, cancellation) Question asked: "Which channel and which months actually make us money, and where are we losing it?"

The client and all details are made up.

About this service

A variant that looks 12% better after a week may be pure chance, and a test stopped at the first good day is a common way to ship a change that does nothing. From your visitors and conversions per variant we calculate conversion rates, the lift, statistical significance and a confidence interval, and check whether the sample was big enough. You get a clear verdict: ship the variant, keep the original, or treat the result as inconclusive and run longer.

What is included

  • What you getA test report with conversion rates, lift, significance, confidence intervals, a sample size check, segment results in higher packages and a clear verdict, as formatted text with tables on the order page. You also get it as a Word (.docx) report with the tables in Excel (.xlsx).
  • Conversion and liftFor every variant
  • SignificanceAnd a confidence interval
  • Sample size checkWas the test long enough
  • SegmentsDevice, source or new users
  • Clear verdictShip, keep or run longer

How it works

  1. 1
    BriefYou paste or attach the numbers per variant
  2. 2
    AnalysisWe test significance and sample
  3. 3
    VerdictYou get the decision and why

What we need from you

Visitors and conversions per variant, ideally by day, the metric you test, how long the test ran and what changed between the variants.

We will ask you:

  • What was tested?
  • Which metric decides the winner?
  • How long did the test run?
  • Which tool ran the test?

Compare packages

PackagesBasicOne test, 2 variants$15StandardUp to 3 variants and 3 metrics$35PremiumUp to 4 variants with segments$69
Deadline1 day2 days4 days
Revisions123
Variants2up to 3up to 4
Metrics1up to 3up to 5
Segment breakdowns0up to 2up to 6
Order BasicOrder StandardOrder Premium

Payment protection

You pay when you order, but the money is released only for accepted work.

  • Fixed priceThe package price does not change after you pay.
  • The money is reservedWhen you pay, the amount is only reserved. It is released to the service once you accept the work.
  • Checked before deliveryEvery result goes through a quality check before it reaches you, and you see it on the order page before accepting it.
  • Free revisionsAsk for changes within your brief as many times as the package allows.
  • Acceptance after 3 daysIf within 3 days of delivery you neither accept the work nor ask for changes, it is accepted automatically. If you write in the order chat after delivery, the order waits; once we answer, we may start the 3 days again and will say so in the chat.
  • Redo or a full refundIf the result does not match your brief and the revisions did not fix it, or the work takes more than twice its deadline, write in the order chat: we will redo the work or refund the payment in full.
How it works →

Questions and answers

My tool already says the variant won. Why check?

Tools differ in method and some show "winners" too early. An independent check on the raw numbers tells you how sure you can be.

Can you tell me how long to run my next test?

Yes, from your traffic and current conversion we estimate the sample size needed to detect the lift you care about.

What if the result is inconclusive?

That is a valid answer. We say how much more traffic would be needed, or whether the difference is too small to matter.

Packages
Order · $35· 2 days