Computing A / B Test Setting with Stata

[konstantin roth]

Hi all,

I am fairly new to Stata. I am facing the following problem. I have multiple versions of add campaigns and want to compare them among each other. Data looks as follows:

Clicks and views correspond to cumulated values from the individual campaigns. How do I test, e.g., if the facebook campaign performed significantly different from the premium campaign? I feel like chi² test will be the right approach as we are comparing dichotomous values (success/failure facebook / premium) but I don't know how to approach the testing procedure in Stata as I have never worked with cumulated data. Does anyone have any hints on how to do this?

Kindest regards,
Lucas

[Carlo Lazzaro]

Konstantin:
you can create an unique categorical variable (let's say -i.add-) with as many levels as the different add campaings via -fvvarlist-.
The you can consider -poisson- (or possibly -regress-; I also assume that you don't have other predictors):

poisson clicks i.add

The same holds for -views- as dependednt variable.

[Dave Airey]

Is it the case that views are independent observations from which a click was recorded as a success? So in an expanded form you have Views-Clicks=Failures and Clicks=Successes for each ad campaign?

[Steve Samuels]

Welcome to Statalist, konstantin! Please read FAQ 12, about how to post good questions.. Your screen shot, like most, is too blurry and tiny to read, About this, FAQ 12.5 states:

In particular, please do not post screenshots. Many members will not be able to read them at all; they usually can't be read easily; and they do not allow copy and paste of data or code, which is highly desirable to allow experienced members to make precise suggestions for your questions.,

Use dataex, described in the FAQ to list a portion of the observations. Place code and output between CODE delimiters, also described in the FAQ.

[konstantin roth]

Hi all,

sorry for my late reply and many thanks for your comments. Your help is much appreciated.

Steve Samuels I installed dataex and will be using it from now on.

Dave Airey you are absolutely right. Views are idependent observations. Clicks is the only dependent variable. Originally, I had daily views and clicks for different campaigns with 0 or 1 indicating the campaign type. The data looked like this:

* Example generated by -dataex-. To install: ssc install dataex
clear
input byte(premium standard onsite premiumplus premiumdata standarddata mobiledata aepremium aestandard facebook) long views int clicks
1 0 0 0 0 0 0 0 0 0  46 0
1 0 0 0 0 0 0 0 0 0  60 0
1 0 0 0 0 0 0 0 0 0  97 0
1 0 0 0 0 0 0 0 0 0 102 0
1 0 0 0 0 0 0 0 0 0  82 0
1 0 0 0 0 0 0 0 0 0  51 0
1 0 0 0 0 0 0 0 0 0  54 0
1 0 0 0 0 0 0 0 0 0  86 0
1 0 0 0 0 0 0 0 0 0 120 0
1 0 0 0 0 0 0 0 0 0 199 0
end

I collapsed all 50000 rows to cumulate the clicks for each campaign ending up with:

* Example generated by -dataex-. To install: ssc install dataex
clear
input byte(premium standard onsite premiumplus premiumdata standarddata mobiledata aepremium aestandard facebook) double(views clicks)
0 0 0 0 0 0 0 0 0 1 13854422 109505
0 0 0 0 0 0 0 0 1 0  9073626   9053
0 0 0 0 0 0 0 1 0 0  6563055  12363
0 0 0 0 0 0 1 0 0 0   910939   4603
0 0 0 0 0 1 0 0 0 0  2202505   3078
0 0 0 0 1 0 0 0 0 0 11431155 103174
0 0 0 1 0 0 0 0 0 0 34699473 184254
0 0 1 0 0 0 0 0 0 0  1960630   4610
0 1 0 0 0 0 0 0 0 0  4184137   5779
1 0 0 0 0 0 0 0 0 0 24630280  50735
end
label var views "(sum) views"
label var clicks "(sum) clicks"

As already mentioned, each view can be regarded as one user/observation coming from one individual campaign (0 | 1 indicated which campaign). Ideally, I would find a possibility to transform all views into different observations holding values of 0 | 1 for campaign type and 0 | 1 for success or failure (click | no click). In this case, I would be perfectly able to work with the data.

Any idea how that might be possible? Or does anyone have an alternative suggestion?

Kindest regards,
Konstantin

[Dave Airey]

You might look at the prtest command or the prtesti immediate variant with the count option for a test of two proportions. You could also look at the ci command for proportion confidence intervals.