Dynamic Testing Experience Results

The analytics for Dynamic Testing experiences focus on the performance of the goal metric and the current chance to win.

Example of the Experience Results page for a Dyanmic Testing experience

Dynamic Testing experience results are available as soon as sessions included in the experience end, which occurs after customers are inactive for at least 30 minutes. Updates are available after every minute by refreshing the page.

Goal Metric Breakdown

When you activate a Dynamic Testing experience, Monetate measures performance of the goal metric across all variants. When it determines that one variant is outperforming another, it adjusts the traffic distribution of the variants based on the cumulative goal performance. In the screenshot below, variant B has a 100% current chance to win.

Callout of the entry for variant B in the Goal Metric Breakdown table that shows its current chance to win is 100%

The Experience Results page displays goal performance for the current leader as well as the weighted average across variants in the Dynamic Testing experience. For Dynamic Testing experiences running for an extended time, the overall goal performance provides insight into how all variants have contributed to the goal if the leading variant has changed throughout the test.

The Goal Performance Over Time and Traffic Distribution Over Time widgets show how the Personalization engine updated traffic throughout the test as well as the resulting impact on the goal metric over time. In this screenshot, you can see the Add to Cart Over Time chart change to reflect the goal metric for the experience.

Example of a Goal Performance Over Time chart

For each widget you can toggle between a cumulative view and an interval view of the data presented. Click the settings cog for the widget, and then select the view you want.

The cumulative data view for the Traffic Distribution Over Time widget presents sessions over time.

Callout of the settings cog and VIEW DATA AS options for the Goal Performance Over Time widget

Place your mouse pointer on any section of either graph to view specific information about the performance of the variants. This screenshot shows the cumulative data view from the time the experience started to a specific point for the goal metric's performance over time.

Example of the pop-up window with a breakdown of variant performance over a specific time range

This screenshot shows the traffic distribution interval view for a specific date.

Example of the pop-up window with a breakdown of variant performance on a specific date

Performance of All Metrics

The All Metrics widget shows the performance of each metric included in the experience for each variant in the test. Click a metric on the left-hand side of the widget to see its performance charted.

Example of the All Metrics widget

You can toggle between a cumulative view and an interval view of the metric performance charted on the right-hand side of the widget.

Callout of the CUMULATIVE and BY INTERVAL toggle options

As with the Goal Performance Over Time and Traffic Distribution Over Time widgets, place your mouse pointer on any point on the graph to view its data breakdown.

Example of the cumulative data breakdown view for the New Visitor Conversion metric

The All Metrics widget provides you with insight into how the current leader impacts each tracked metric in the experience. With that information in mind, you shouldn't use the performance of any metric that isn't the goal metric to inform business decisions because the Personalization engine reallocates traffic based on how the goal metric performs and doesn't pay attention to the performance of any other metric when doing so.