Possibility to view specific version in Marketing A/B tests

Alf Nilsson
Member since: 2008
 

Currently the only way to look at a specific version of an A/B test is either to clear cookie and browes to the page until you get the version you want to look at, or to use the preview function in Edit mode.

Our problem is that we want to look at a specific version in a mobile device which gives us three issues

1) You can't preview the exact experience of this version in Episerver Preview

2) The Edit interface is not really good to use in a mobile device

3) The stakeholders that wants to preview the specific versions does not have access or interest accessing Episerver

I wish that it was possible to access a specific version in the A/B test. For example using a querystring or similar.

#203278
Apr 15, 2019 13:25
Jacob Pretorius
Member since: 2016
 

Hi Alf,

I think this would be a very nice feature to have in the system. I've had a quick go at getting a crude version of it on a test site (I used Quicksilver). I've made some progress getting it working with a ?version= query string simply running on the start page as you will see below. Obviously, it's just a testbed implementation, and once it's actually fully working it could be abstracted to an Attribute or helper.

public ViewResult Index(StartPage currentPage, string version = "")
{
    var viewModel = new StartPageViewModel()
    {
        StartPage = currentPage,
        Promotions = GetActivePromotions()
    };

    // Check for AB test override flag
    if (!string.IsNullOrWhiteSpace(version))
    {
        var testManager = ServiceLocator.Current.GetInstance<ITestManager>();
        var contentLoader = ServiceLocator.Current.GetInstance<IContentLoader>();

        var tests = testManager.GetActiveTestsByOriginalItemId(currentPage.ContentGuid);
        if (tests?.Count > 0)
        {
            // We found the test for this page, lets load the content versions of this page also
            var contentVersionRepo = ServiceLocator.Current.GetInstance<IContentVersionRepository>();
            var pageVersions = contentVersionRepo.List(currentPage.ContentLink);

            // Make sure we found page versions
            if (pageVersions?.Count() > 0)
            {
                if (string.Equals(version, "a", System.StringComparison.OrdinalIgnoreCase))
                {
                    // Look for the first 'A' test version of the page
                    var testItemVersion = tests[0]?.Variants[0]?.ItemVersion;

                    var pageVersionWeWant = pageVersions?.Where(w => w?.ContentLink?.WorkID == testItemVersion)?.First();
                    if (pageVersionWeWant != null)
                    {
                        viewModel.StartPage = contentLoader.Get<StartPage>(pageVersionWeWant.ContentLink);
                    }
                }

                if (string.Equals(version, "b", System.StringComparison.OrdinalIgnoreCase))
                {
                    // Look for the first 'B' test version of the page
                    var testItemVersion = tests[0]?.Variants[1]?.ItemVersion;

                    var pageVersionWeWant = pageVersions?.Where(w => w?.ContentLink?.WorkID == testItemVersion)?.First();
                    if (pageVersionWeWant != null)
                    {
                        viewModel.StartPage = contentLoader.Get<StartPage>(pageVersionWeWant.ContentLink);
                    }
                }
            }
        }
    }

    return View(viewModel);
}

It works perfectly if the user is logged in to the CMS (even in other tabs, so they have a session) but unfortunately, I can't get the ContentLoader to return the challenger (B) version of the content when logged out. The B version has a VersionStatus of Published (enum 4) instead of CheckedOut (enum 2) that usual content (and version A) has once actually published.

I took a quick peek at the ContentLoader/ContentRepositry code for some sort of a flag we can give it to get unpublished versions of pages using the Work ID in the ContentReference we already have here, but I could not find anything. I even gave the old DataFactory a try to no avail.

Perhaps someone else has a way to get the "Published" (not "CheckedOut") page content. Hypothetically you could use a logged in user account and cache it but that gets messy real quick.

Let me know if you have any thoughts or can think of a suitable way to get this going.

Cheers,

Jacob.

#203361
Apr 17, 2019 18:24
Jacob Pretorius
Member since: 2016
 

Part Two, the saga continues.

I've now got it working reliably with a different (somewhat more hackey) approach where I override the AB testing cookie based on the QS flag.

// Check for AB test override flag, accepts any string for A test, "b" for B test
if (!string.IsNullOrWhiteSpace(version))
{
    var testManager = ServiceLocator.Current.GetInstance<ITestManager>();

    var tests = testManager.GetActiveTestsByOriginalItemId(currentPage.ContentGuid);
    if (tests?.Count > 0)
    {
        // Get the AB cookie for this page
        var abCookie = HttpContext.Response.Cookies[$"EPI-MAR-{currentPage.ContentGuid.ToString().ToLower()}_{Thread.CurrentThread.CurrentUICulture}"];
        if (abCookie != null)
        {
            // Default to "A" test content
            var versionId = 0;

            // update to "B" if set
            if (string.Equals(version, "b", System.StringComparison.OrdinalIgnoreCase))
                versionId = 1;

            // Update the response cookie
            abCookie.Value = $"start={DateTime.UtcNow.AddDays(1).ToString("MM/dd/yyyy hh:mm:ss")}&vId{versionId}=&viewed=False&converted=False&k0=False";
            Response.SetCookie(abCookie);
        }
    }
}

Due to browser caching it does look like you need to do an extra page reload when switching between test versions (A to B or vice versa). Seems to work well otherwise.

Let me know if this solves your problem.

Cheers,

Jacob

#203373
Apr 18, 2019 11:47