Smashing Magazine calls it “The Ultimate Guide,” which is a bit of an overstatement, however they have published a first-rate introduction to creating simple experiments to improve email subject lines and more. The secret is to create rigorously structured tests by showing random groups of users different content and watching how they respond — one group gets the “A” subject line, another gets the “B,” hence “A/B Testing.”

Direct mail marketers have long been doing this sort of testing. In fact, they’ve moved on to multi-variable testing and Taguchi-style experiments — but that is for those with huge audiences who can experiment in mass.

Simple A/B testing is easy and will generate surprising insights into your audience’s behavior. Plus, anyone with a website or a mailing list can get started.

What else can you test? Here’s Smashing Mag’s short list:

• The call to action’s (i.e. the button’s) wording, size, color and placement
• Headline or product description
• Form’s length and types of fields
• Layout and style of website
• Product pricing and promotional offers
• Images on landing and product pages
• Amount of text on the page (short vs. long)

via Smashing Magazine.