PEGA Simulation - Test and Learn Approach to Maximize the Marketing KPIs

What is ‘test and learn’ marketing?

Test and learn marketing is a data-driven approach to constantly optimizing Strategies and to improve the KPIs performance of recommendations made from PEGA Decisioning Simulation capability through a series of ongoing tests. It allows organizations to convert insights into hypotheses and then test those theories to prove their value. Primarily to improve the take rate of recommendations made from PEGA OR it could be improving the revenue AND/OR margin for an Organization. It also could be a two different variant of the Treatment for the same action to the same channel. Eg. Having two variants of the Email treatments for the same Action. This is a perfect example of test and learn marketing.

Test: Taking the guesswork out of marketing

Test and learn principles remove this speculation by building a data-driven system that proves the success and failure of individual marketing strategies. Instead of relying on correlative insights, we can run the PEGA Simulation to attribute meaningful KPIs to each change, measure performance and test variations to determine which strategy is most effective.

Learn: Turn insights into better marketing decisions

The learn aspect can be as simple as finding out which hypotheses are correct and prioritizing them in order of value (which meets the Organisation KPIs objective already set). We can also learn from previous experiments (and their data) to create new hypotheses or determine which testing opportunities meeting the marketing KPIs.

We could also leverage an existing PEGA Decisioning Machine learning AI capabilities like Adaptive Modelling, Predictive Modelling to experiment to spot opportunities for testing, makes recommendations based on your results and predicts the outcome/value.

A good test and learn marketing principles has two main objectives:

1. Increase conversion rates through testing variations.

2. Learn from previous tests and apply findings to future marketing decisions.

We run a series of tests to determine the best outcome for specific scenarios and we collect all of this data to spot patterns and inform smarter marketing decisions.

By applying a test and learn model to all of our marketing campaigns, we can improve every aspect of brand performance with data-driven insights. The more tests you run and the more data we collect, the more reliable these insights become and the more robust our marketing decisions will be.

Some of the example of Goal for Test-and-Learn approach

• Measure the Impact of Decisioning Framework values (Real Time Controls) changes and to align Organizational KPIs

• Test the effectiveness of different treatments for the same Action for the same Channel Eg: Keep different Email Treatment (Title and/or content) for Control and Test to measure the impact

• Determine which Treatment Content generate the highest engagement.

• Test the potential value of a new Channel.

• Calculate the lifetime customer value of leads based on acquisition channel and their position along your marketing funnel.

• Identify negative friction that’s preventing leads from converting into customers.

• Optimize marketing funnel to reduce dropouts and maximize conversions.

• Optimize the customer experience to maximize repeat purchases and lifetime customer value.

A test and learn model can be applied to any marketing objective and campaign type where we want to maximize performance and collect insights to inform better marketing decisions in the future.

How to apply a test and learn model in PEGA Decisioning & Marketing

Some quick overview of the key steps to take in test and learn model are:

1. Define goal and KPIs: It’s crucial that Organisation should have a specific goal in mind and understand which KPIs truly measure success. It could be higher engagement reate, higher order rate, improving revenue, margin and reducing churn etc.

2. Develop hypothesis: This is the theory that your tests will prove or disprove – to achieve the same KPI, there could be different school of thoughts. Ideas will help us to develop hypothesis.

3. Create testing criteria: To get meaningful results from tests, weneed to define strict testing criteria and remove every possible variable that might skew the desired outcome.

4. Test ideas with the highest potential first: This will have the biggest impact on performance and collect the most valuable data early on – but it also ensures that our test and learn models achieve a positive ROI as soon as possible.

5. Run your tests until statistical significance: This ensures your results are reliable enough to inform decisions while mitigating for variables outside of your control.

6. Turn insights into action: Individual results help you make specific decisions but collecting data from all of your tests helps you to make informed predictions and decisions for future campaigns – for example, which ad format is most effective for your campaign message or how much to bid to achieve your target reach.

7. Measurable success post implementation: Once right Strategy has been identified, implement the hypothesis produced results closed to KPIs and make the BI tools in place to measure the success/failure of the recent change.

8. Apply Continuous Improvement Technique: By learning from successes and failures, we continue to do what works and constantly improve upon what doesn’t

This test and learn approach can be applied to any marketing strategy and, PEGA Customer Decision Hub Simulation capability would help to achieve the possibilities in a quick turn-around time.

Happy Learning!

Nanjundan Chinnasamy

PEGA Simulation - A Technique in the PEGA Marketing to test and Solve Problems in the Marketing KPIs

Today i would like to give some insights on The Use of PEGA Simulation in Solving the Problems in Business/Marketing KPIs. This article focuses on utilization of analysis of different possibilities using a Simulation feature made available in PEGA Customer Decisioning Hub. For a given business problem, there will be more than one solution. Each solution is analyzed with the help of simulation. They have been evaluated and the optimal solution was selected. 

The modern trend these days is to optimize everything that we can, especially in Marketing when PEGA Customer Decision Hub installed to drive the Marketing messages for an Enterprise. The result is the achievement of predetermined targets (KPIs), such as reducing Churn, Increase Revenue from Each customer, Increase Margin and increase the proposition value for each customer. Most of the Organizations are lagging in measuring & optimizing the Outcome of the Marketing messages against Organization KPIs. It is not all about recommending the ‘number’ of marketing messages at various customer touch points. Organizations could start exploring the ways to optimize the current logic and run the various simulation options to understand the Strategy which brings values close to the KPIs made available. 

Let's discuss one of the Telecom business problems and understand how PEGA Simulation will help Markets to test and Learn to maximize the business outcome. Aka. KPI

Problem:

Let's consider three huge companies in the telecommunications market in the US. Each company sold some of the four products (known as “quadruple play” when combined), including high speed Internet, cable TV, telephone, and cell phone services. Many households purchased individual services from the different companies. The aggressiveness of the players made “product bundling” a key marketing strategy, but there was not a clear understanding of how to use this strategy, or the impact it might have on the market.

Company A wanted to enter the markets that they hadn’t covered yet OR they wanted to cross -sell other products, but each one knew that penetrating the competitor’s niche would force them to reply in the same manner. 

The challenge was to build a Strategy (Including Product Price and Offer price etc.. ) that would allow the client to analyze several scenarios, taking into account the whole telecom market (with its 3 major players) and the “product bundling” effect. 

Solution:

After doing enough market study, the Marketing Team of Telecom company Arrived prices and offers for each product bundle combination. Marketing team arrived at the conclusion of “quadruple play” buble cost could be either $200 OR $220. They are not sure which price point would help to benefit both the customer and Organization to maximize their value.  

Organization A can leverage PEGA Simulation capability to understand the impact of business KPIs (Churn, Revenue and Margin) keeping the possible price combinations to determine the best pricing Strategy. Strategy which gives us closest KPI outcome can be selected for Implementation. 

Outcome:

It was the high flexibility of using PEGA Simulation capability to build, measure and Learn possible what-if scenarios. Outcome of the Simulation result can be discussed with Marketers explaining various organization specific KPIs. Marketing team could make Informed Decisions based on the simulation results produced for each what-if variance.  

Simulation is one great technique that allows us to have a reliable view of possibilities before getting into Production. So we have a simple way to test an infinite number of variants to solve the problem and for the actual implementation to choose the best possible solution to the situation.Simulations help to detect problems and complications that resulted in normal operation and save significant costs that we had to then spend on their removal. In practice it is impossible to try different alternatives, in terms of irreversible steps that are expensive. 

 

Conclusion:

By using simulation experiments, we can obtain results for different alternatives. Team can share the informed choices to the Marketing team before taking Decision to implement any Marking Strategies. There is a detailed knowledge base article made available in PEGA for managing Simulation tests in Customer Decisioning Hub. More details can be found here https://community.pega.com/knowledgebase/articles/decision-management/85/managing-simulation-tests-pega-customer-decision-hub