Beating controls --- Should you focus on the What or the Why?

There are two schools of thought about what to focus on to beat creative controls.

When reviewing a control for any medium, the creative elements one looks at closely include the offer, the main selling proposition and the creative execution. So you know that beating your control requires testing of one or all three of these elements are required to create a winner.

One school of thought advocates testing only one element at a time so you know why the new creative won. For example, test the offer while preserving the look, feel and main selling benefit of the control. In other words, don’t add other changes that pollute the purity of the offer test.

The other school on the other hand, takes a different position. Breakthroughs happen only by testing radically different creative executions. So test everything at once without worrying about what made the huge difference in response when the tested creative wins.

Both approaches offer advantages.

In your testing, do you work hard to figure out exactly “what” made the big difference? Or do you take your gloves off and concentrate on testing radically different creative without spending too much energy on analyzing exactly what elements of the creative made the big difference?