Testing

 Introduction:

•        Testing is the technique used to know the actual intent of the customers who are visiting a website

•        Though a website has a major goal like, content production or e-commence or engagement. There are micro elements that are also part of this. Ex: Looking for careers link in Amazon web site.

•        So it’s important to know how each group of users are responding to different group of people.

•        We can use testing to know the known- unknown elements

Types of tests

•        There are many types of testing techniques that are used. Most popular are,

                a. A/B testing or Split testing

                b. MVT testing or Multivariate testing or bucking testing

•        Any testing technique requires having the hypothesis ready and they have a clear definition of test elements.

•        By testing, you can’t derive new learning’s other than the elements that we have hypothesis for.

A/B Testing

1. A/B Testing, also known as split testing, is a method of website optimization in which the conversion rates of two versions of a page version A and version B are compared to one another using live traffic.

2. Site visitors are bucketed into one version or the other. By tracking the way visitors interact

With the page they are shown the videos they watch, the buttons they click, or whether or not they sign up for a newsletter you can determine which version of the page is most   effective.

You design two versions of a web page (A & B), divide the traffic between the two, and choose the one that gives you the maximum conversions.

How A/B Testing Works

In an A/B test, you take a webpage or app screen and modify it to create a second version of the same page. This change can be as simple as a single headline or button, or be a complete redesign of the page. Then, half of your traffic is shown the original version of the page (known as the control) and half are shown the modified version of the page (the variation).

As visitors are served either the control or variation, their engagement with each experience is measured and collected in an analytics dashboard and analyzed through a statistical engine. You can then determine whether changing the experience had a positive, negative, or no effect on visitor behaviour.

5 Simple Steps to Start A/B Testing Today

1.    Determine your goal

Your goal will vary based on your business.

For instance, a Business-to-Business company might be focused on generating more leads for their sales staff. On the other hand, an e-commerce Business-to-Consumer website might want to increase sales.

2.Decide what to test

Now that you know your goal and the page you’re going to test, it’s time to decide what element you’ll test.

Here are some options:

·         Your headline.

·         Your offer text.

·         Your button text.

·         Your form fields.

·         The colour of your form button

I could go on and on. There are so many things to test.

Note that some sites will see a dramatic improvement in their conversions just by changing the colour of their opt-in button. Other sites, however, will see little improvement from a button colour change. They’ll have to test bigger elements like their headline, offer, and USP (unique selling proposition).

Why? Well, one reason could be, if your website visitors don’t understand how you’ll help them, they won’t opt-in no matter what color the button is.

Quick Tip: If you’re having trouble deciding what to test, choose something anything. Let’s just get a test up and running. You can always test other elements later.

3. Create your test

There are two parts to this step.

• The Creative Part

The first is what you might call “the creative” or making another version of the element you’re testing.

For instance, maybe your headline doesn’t immediately tell your visitor “What’s in it for me?” Write another version of your headline, this time focusing more on your visitor.

• The Tech Part

The second part of creating your test is the technology you use to deliver each variation to your visitors.

If you’re intimidated by this step, fear not. Modern technology makes this super easy and fast.

4. Wait.

I find this step the most challenging of the five steps, mainly because waiting isn’t my strong suit. However, to get accurate results, we must wait.

Note: Go ahead, log in and check out your results during your test just do not (I repeat, DO NOT) stop, pause, or edit your test until it’s complete.

5. Determine your winner

When your test is over, it’s time to calculate your results or determine if your tests (and the results) are “statistically significant.”

Although it sounds complicated, we’re simply making sure our results will perform the same over the long haul. In other words, how sure are you that your test results are accurate?

A/B Testing Process

Example:

•        The current version of a company's home page might have in-text calls to action, while the new version might eliminate most text, but include a new top bar advertising the latest product. After enough visitors have been funneled to both pages, the number of clicks on each page's version of the call to action can be compared.

•        It's important to note that even though many design elements are changed in this kind of A/B test, only the impact of the design as a whole on each page's business goal is tracked, not individual elements.

•        A/B testing is also useful as an optimization option for pages where only one element is up for debate.

MULTIVARIATE TESTING

Multivariate testing uses the same core mechanism as A/B testing, but compares a higher number of variables, and reveals more information about how these variables interact with one another. As in an A/B test, traffic to a page is split between different versions of the design. The purpose of a multivariate test, then, is to measure the effectiveness each design combination has on the ultimate goal.

Once a site has received enough traffic to run the test, the data from each variation is compared to find not only the most successful design, but also to potentially reveal which elements have the greatest positive or negative impact on a visitor's interaction.

Getting Started With Multivariate Testing 

To create your first multivariate test, first choose a tool or framework that supports multivariate testing. You can use one of the tools listed in the section “Tools” in the end of this article. Please note that not all A/B testing tools support multivariate testing, so make sure your tool of choice allows it.

The following parts of a page (listed in order of importance) are typically included in a multivariate test:

·         Headline and heading,

·         Call-to-action buttons (colour, text, size, placement),

·         Text copy (content, length, size),

·         Image (type, placement, size),

·         Form length.

What to measure: defining test objectives

Before you start formulating a test hypothesis, or begin running tests, the first and most important step is to ensure that there are defined objectives for the website. You'll want to examine your marketing goals in order to determine the appropriate success factors that all of your organization's stakeholders can agree upon. Here are some typical measurable website goals:

Make money: sell product, generate leads, and advertising or promotional click-throughs.

Save money: enable users to adopt self-service features and/or answer product and service questions on their own (such as through online FAQs and documentation).

Examples of Multivariate Testing

 Common examples of multivariate tests include:

1. Testing text and visual elements on a webpage together

2. Testing the text and colour of a CTA button together

3. Testing the number of form fields and CTA text together

Using multivariate testing as a method of website optimization is a powerful method of gathering visitor and user data that gives detailed insights into complex customer behaviour. The data uncovered in multivariate testing removes doubt and uncertainty from website optimization. Continuously testing, implementing winning variations and building off of testing insights can lead to significant conversion gains.

Filters and Segments

Segmentation: It is the key to greater understanding of your Web Analytics data. You want to use a segment if you are selecting entire visits and you want to use a filter if you are looking at specific events, page views etc..

·           Custom Variables: Custom variables can be used to define additional segments to apply to your visitors other than the ones already provided by Analytics. It is a powerful technique as it can be tailored to a website and its idiosyncrasies but, on the downside, it depends on additional website coding.

 ·           Advanced Segments: Advanced segmentation the other hand, allow you to apply and remove segments without removing data. They are considered less effective for long term segmentation purposes.

·           Profile filters: In Google Analytics, Filters are used by Views to segment the data into smaller groups. Filters can be used to include only specific subset of traffic, exclude unwanted data, or to search and replace certain pieces of information. Belong to a long-term segmentation strategy. The data collected in a specific profile in the past cannot be changed or removed, so please be careful when applying filters.

     Filter types and uses

•                    Include and exclude filters

•                    Search-and-replace filters

•                    Advanced filters

•                    Exclude internal traffic

•                    Filter domain referrals

•                    Filter on geography

•                  Custom filter fields &Social Network filters

i.            Include and exclude filters

Use Include and Exclude Filters to eliminate unwanted hits.

If you apply an Exclude Filter and the pattern matches, the hit is thrown away and Analytics continues with the next hit. If the pattern does not match, the next filter is applied to that hit. You can create either a single Exclude Filter with multiple patterns separated by '|' or you can create multiple Exclude Filters with a single pattern each.

ii.            How search-and-replace filters work

Search-and-replace filters use regular expressions to find a search string in a filter field and replace it with a replacement string.

To create a search-and-replace filter:

1.      Follow the instructions to create a new filter for your view.

2.      Set the Filter Type to Custom.

3.      Click the Select filter type drop-down menu and select Search and Replace.

4.      Use the Filter Field drop down menu to select the field (dimension) you want search.

5.      Enter a regular expression in the Search String field.

6.      Enter the Replace String. To delete the search string entirely, leave this blank.

7.      Use the Case Sensitive checkbox if your regular expression is case sensitive. Otherwise, the search will be case insensitive.

iii.              Advanced filters

The Advanced filter lets you construct Fields for reporting from one or two existing Fields. Use expressions and corresponding variables to capture all or parts of Fields and combine the result in any order you wish. For general information on how filtering works, read About Filters.

iv.            Exclude internal traffic

Filter out traffic to your website from people on your corporate network.

Most of the time, Google Analytics is used to track how external customers and users interact with your website, since internal traffic patterns are typically different from external traffic patterns. When your reporting views contain hit data from both internal and external users of your website, it might become difficult to determine how your customers are actually interacting with your website.

v.            Filter domain referrals

Reduce spam traffic from your Google Analytics data.

Referral traffic is the segment of traffic that arrives on your website through another source, like through a link on another domain. Analytics automatically recognizes where traffic was immediately before arriving on your site, and displays the domain names of these sites as the referral traffic sources in your reports.

vi.            Filter on geography

Instructions for creating filters are in Create/manage view filters.

Here are two ways to use filters on geo-fields to track data based on geographical regions:

1. Consolidate countries into sales regions

2. Use Region-Specific Reporting

vii.  Custom filter fields

It Includes,

·         Content and traffic

·         Campaign or Ad group

·         E commerce

·         Audience /users

·         Location

viii.     Social Network filters

If you create a custom filter that uses any of the Social dimensions (Social Network, Social Action, Social Action Target), keep in mind that these dimensions apply only to Social interactions (social-hit data).

Create a filter at the account level

To create a filter at the account level:

•        Sign in to your Analytics account.

•        Select the Admin tab and navigate to the account in which you want to create the filter.

•        In the ACCOUNT column, click All Filters.

•        Click + New Filter.

•        Select Create new Filter.

•        Enter a name for the filter.

•        Select Predefined filter to select from the predefined filter types.

•        Select Custom filter to construct a custom filter from the options we provide. If you create a custom filter, consult our definitions of the filter fields.

•        From the Available views list, select the views to which you want to apply the filter and then click add.

•        Click Save.

To create a filter at the view level:

•        Sign in to your Analytics account.

•        Select the Admin tab and navigate to the view in which you want to create the filter.

•        In the VIEW column, click Filters.

•        Click + New Filter.

•        Select Create new Filter.

•        Enter a name for the filter.

•        Select Predefined filter to select from the predefined filter types.

•        Select Custom filter to construct a custom filter from the options we provide. If you create a custom filter, consult our definitions of the filter fields.

•        From the Available views list, select the views to which you want to apply the filter and then click add.

•        Click Save.

To change the filter order for a view

•        Sign in to your Analytics account.

•        Select the Admin tab and navigate to the view in which you want to create the filter.

•        In the VIEW column, click Filters.

•        Click Assign Filter Order, select the filter you want to move, then click Move up or Move down. Click save when you are finished.

•        If you want to remove a filter from the view, click remove in the row for that filter.

To add existing filters to or remove them from a view

•        Sign in to your Analytics account.

•        Select the Admin tab and navigate to the view in which you want to add or remove filters.

•        In the VIEW column, click Filters.

•        Click + New Filter.

•        Select Apply existing Filter.

•        Add or remove the filters as necessary.

•        Click Save.

Recommended Views

Unfiltered View: Always have a view called “Unfiltered View – Do Not Delete” defined to keep your raw data intact. This view is there to make sure if anything unexpected happens to the data at least you have a backup saved somewhere. You should never add any filters to this view.

Main View:  This view would be the one you use for reporting on a regular basis. Call it whatever you like, but make sure it’s communicated to your team which view should be used for reporting.

Test View: Apply your filters to this “Test View” first. Since the effects of applying a filter cannot be undone, it is recommended that you apply your new filters to a Test View first and let them run for a couple of days, to have enough time to verify the credibility of the results. If you have not already created and defined a “Test View”, it is a good time to create one now; your Test View should use the same exact settings as your “Main View” that you would normally use for your reporting purposes. If everything looks good after a few days, then apply to your Main View.

Filter Processing

Filters cannot be applied to your previous historical data; they can only be applied to your data moving forward. Filters are processed in order, so make sure you arrange the filters in the correct order.

Basic Filters

In this blog, I am going to cover some of the basic (as opposed to more customized) filters everyone can use to segment their data. We use these filters to make sure our data is valid, to clean up the data, and to find out where the data is coming from.

Data Collection

Exclude Internal IP
Include Internal IP
Include Specific Hostname
Include Specific HYPERLINK "http://www.lunametrics.com/blog/2015/12/10/basic-google-analytics-filters/"Subdomain
Exclude Dev Site Traffic

Introduction to segments

You can use any segment as a filter for your reports. Once you apply a segment, it remains active as you navigate throughout the reports until you remove it. You can apply up to four segments at a time, and compare results from each segment side by side in your reports.

The All Sessions segment is applied to all reports by default. This segment includes all data for every session in a date range, and lets you see performance for your entire user population.

Apply segments

To apply segments to a report:

•        Sign in to your Analytics account.

•        Open the View that includes the reports you want to use.

•        Click the Reporting tab, then open the report you want. This example uses the Audience Overview report.

•        At the top of your report, click + Add Segment...