Multivariate Tests and the Creative Process: A Shout Out to the Number 2 Pencil

I recently overcame a design block with a quick, hard smack to my forehead. No, I’m not a freak — let me explain.

There I was, sitting in front of the computer, Adobe Illustrator on one screen, Photoshop on the other — both art board and canvas, completely blank. My task: come up with not one, but several design concepts for a Landing Page. An added twist, the design concepts were for a single element of a 4×4 Multivariate Test (MVT). Being the first round of MVT for this Landing Page, each concept had to be well differentiated in order for the data to be meaningful afterward.

The MVT was set to follow an A/B test that was already live and collecting data. Of course in my head, the growing A/B data was starting to feel more like a countdown to embarrassment. I needed ideas and my creative gears were locked up and starting to smoke. With several iTunes tracks yielding no inspiration I had to make a change — and fast! And so it was out of this desperation that it hit me — or I hit me — as the story goes. You see; I smacked my forehead when I realized I knew better. I just needed to find my sketchpad and pencil.

The Number 2 Pencil Saves the Day

OK, so perhaps I’m giving a little too much credit to an inanimate object. Truthfully, this should have read “Doris Harrison Saves the Day”, but the Number 2 pencil title had a little more edge to it and helps with my point. Doris Harrison was an instructor of mine at the Academy of Art University in San Francisco and she was relentless about following the creative process. Starting on the computer was a cardinal sin according to Doris, and every major assignment had stage-gates where she reviewed each step of the creative process as we progressed.  At the beginning it seemed overkill and felt like a terrible waste of graphite and paper. But as the class progressed and the assignments became more intense, I was converted. My final design project for Doris was selected from more than 3-dozen concept sketches, of which nearly a dozen were solid contenders, each of which was unique in how it communicated the message.

So why should this project be any different?

Had my time in the working world transformed me into a man with unstoppable creative genius that transcended the need for rudimentary tools? Was I above the law? Any evidence from previous lucky moments was now being burned out of my memory by the blank white computer screens glaring back at me. It was time to get back to basics.

The Creative Process — Approved for Use with Multivariate Tests

1. Research

Find out all you can about the product or subject — know what makes it different. Learn the history of the subject and its competitors. Understand the audience and market. Gather pictures of the subject or go give it a try — anything that will help to make it as tangible as possible. This step is also where the designer learns the context and constraints they will need to work within (i.e., size of the finished piece, download speeds, etc.)

2. Define the Challenge

What are you trying to communicate — what’s your point? In corporate speak — what’s the value add? In laymen’s terms — why should I care? During this step you’re building the success criteria for your design. “For this design to be successful, it must communicate X, Y, and Z.” These points can be a lot of things; building brand image, key product differentiator, conveyance of a feeling or level of sophistication. After Steps 1 & 2 are complete, I typically have a few pages of notes written out. These notes are not overly organized, but I do like to make sure that the challenge is well-identified and as concise as possible. If you’re working with other designers, consensus on the challenge is very important.

3. Attempt to Answer the Challenge/First Ideas

Brainstorm first solutions — get those first ideas you had down on a sketchpad. I use a battery of questions to help me get started. For instance:

• What would the world be like without the product or subject (your “Got Milk” solution)?
• How was the task done before the subject or product was invented?
• What does using the product or subject make you feel like?
• What action describes the product vs. its competitors (fast vs. slow, nimble vs. cumbersome)?
• How does the product make life simpler for the audience (is there a process “before and after” you can contrast)?
• Does the subject look like or act like something in nature?

These ideas should be sketched out on paper as quickly as possible. They don’t need to be neat or well articulated. We’re talking quick and dirty thumbnail sketches no larger than necessary — a typical thumbnail sketch for me is about 2×3 inches.

4. GET AWAY!

Perhaps the most important step, take time away from the design process to let your subconscious work on solving the problem. For some people this is taking a walk or a bike ride — for others, it’s working on something completely unrelated. Regardless, getting away from the project at this stage is essential.

5. Let the Ideas Flow

I’m always amazed when I come away from Step 4 at how quickly the ideas start flowing. Again, quick and dirty — get all the ideas down on paper.

6. Select and Refine the Best Ideas

Now you should have perhaps a few dozen concepts from the previous steps. It’s now your job to determine which of these concepts should move on to the next level of refinement. Depending on your own internal processes, at this point you should be willing (or it may be required) to share your concepts with members of your team or your client (again, depending on your process). At Closed Loop Marketing we have an internal review process at this point to determine which concepts will be taken to a “client-ready” level of refinement. We will typically present 3 to 4 unique concepts per element variation to the client. These are presented in rough form (sketches for illustrations). I prefer scanning sketches in and using the computer for any frame/line or text elements (though it’s not required).

7. Produce to the Appropriate Level

With approval to continue, you’re now ready to execute the design. Depending on the requirements of the project, you may deliver a Photoshop file, a Flash movie, or GIFs and JPGs. Regardless, you should feel confident that you have concepts that are well thought out and differentiated — you’ve got the best of the best to utilize during your MVT.

Prediction: Pencil shortage by 2010

As the creative medium becomes more and more online, the pull away from the creative process becomes greater and greater. After all, why not begin on the computer since that’s where you’re ending up? And, truth be told, there are plenty of successful designers who haven’t picked up a sketchpad in years. If you’re able to move as quickly as your brain does in this format — by all means, stylus/mouse away.  But as more and more marketers leverage MVT in their optimization efforts, the demands on creative resources will increase to match the resulting need. Consider that 49% of marketers surveyed at AdTech at the end of 2007 said they planned to use MVT during 2008 and we start to see how steep the demand curve is going to get. And this is a great thing for designers. Not only will there be more opportunities to exercise your creativity, there’s a direct (and real-life) feedback loop as a result of the platforms currently available (such as Google Website Optimizer, WideMile or Omniture’s Test&Target).

Putting Pencil to Paper

Knowing your own creative process and sticking to it is the first step — especially as the complexity of projects increases. The second step is getting buy-in with your team, management and/or client. I’m lucky — at Closed Loop Marketing we have a creative process that accommodates a high level of quality and we work with clients to ensure that the MVT’s are afforded the time needed to create compelling and unique variations. Our clients like the ROI from the MVT as well, since the best of the best variations rise to the top – rather than the best of the mediocre.

If you’re a designer, hopefully this has struck a chord or provided some inspiration to return to your creative beginnings. If you’re a manager, hopefully you’re building time into your projects to take full advantage of the creative process. And finally, if you happen to make #2 pencils – thanks for keeping graphite alive!