Bob Bly Direct Response Copywriter Official Banner

The flaw in the A/B split test challenge

September 29th, 2017 by Bob Bly

One of the most nonsensical — and fairly common —
conversations in marketing goes something like this….

The ad agency or copywriter submits a promotion.

The client says, “I don’t like it.”

The copywriter gets his dander up, bristles, and says

“It’s very strong. Test it.”

The client refuses.

The copywriter goes on Facebook and says:

“I have a stupid client who refuses to A/B split test.”

Other copywriters chime in and say the client is stupid.

But … in fact, he is not stupid.

A lot of the time, the client is well aware that the A/B split
test is the only legitimate way to test a promotion.

The reason this is true about A/B splits is because only a
marketing test definitively determines whether the copy is strong
based on results, and not on subjective judgment.

But despite this fact, subjective judgment will always be part of
the process of developing marketing campaigns. Reason:

Before your copy can be tested in an A/B split, the team has to
agree on pricing … offer … theme … bonuses … lists … the “big
idea” for the promotion … the headline … the lead … and more.

“Pre-test” and screening of what to ultimately prepare and test
for real with money has to come down to subjective judgment.

Now, you’ve heard the old expression, “Opinions are like
A-holes; everybody has one.”

But of those opinions, the one that counts the most in addition
to the copywriter’s is the client’s, because he is paying the bills
and putting his money on the line.

And having the final say does not make him an A-hole. Far from

Fact of the matter is, you may know more about copywriting than
your client (although in some cases, not).

But the client almost certainly knows more about his business
than you.

Therefore, his instincts and opinions should be considered
carefully, and never ignored or dismissed by you out of hand.

Now, if you feel strongly that nothing beats your copy, and
client criticism annoys you, start your own online information or
merchandise marketing business.

When it’s your business, you can run all your copy exactly as you
wrote it.

But if having the final say is paramount, you should only be an
entrepreneur who writes his own copy.

And not a freelancer writing for copy for clients.

A number of copywriters have made the transition from doing
client work to 100% writing copy for their own products.

Nothing wrong with that. More power to them. I know several, and
a few are almost militant about why theirs is the smarter
copywriting path.

As an ancillary income stream, I also write copy to sell info
products I publish online.

And it by itself generates a six-figure annual income we could
comfortably live on.

But that’s not the way I roll.

Writing copy for clients has pleasures and intellectual
challenges I maintain you simply cannot get from writing only
about your own products.

And as a contract copywriting freelance, I absolutely love the
wide variety of products, services, offers, industries, and
markets I get to write about and for.

It’s more fun than I can shake a stick at.

So overall, no complaints — though on rare occasion I may grumble
a bit.

But for nearly 4 decades, I have been primarily a traditional
copywriter working for clients.

And secondarily an info marketer, book author, consultant, and

That’s where I want to be in my copywriting business.

And as I have done what I wanted to, likewise, you should do what
works for you.

As the late, great David Ogilvy said, quoting an old Scottish

“Be happy while you’re living, for you’re a long time dead.”


This entry was posted on Friday, September 29th, 2017 at 9:35 am and is filed under General, Writing. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

68 responses about “The flaw in the A/B split test challenge”

  1. Donna Batone said:

    I like your writing style! I’m always enjoying reading your articles and notes. Will you be so honor and read my writing piece as well?

  2. Zay said:

    I have been primarily a traditional
    copywriter working for clients. geometry dash game

  3. Vernonk Mang said:

    John arnold is an academic writer of the Dissertation-Guidance. Who writes quality academic papers for students to help them in accomplishing their goals. Marketing Dissertation Writing Service

  4. sakib khan said:

    February 2018 Calendar word

  5. best tower fan in india said:

    Choose best tower fan in India to beat the summer and get super cool with best and branded tower fans and with Comparisons.

  6. vex 3 said:

    I have had a lot of harvest after watching this article from you! I feel it interesting, your post gave me a new perspective! I have read many other articles about the same topic, but your article convinced me! I hope you continue to have high quality articles like this to share with veryone!

  7. driving directions said:

    Thanks for sharing.I hope you continue to have such quality articles to share with everyone! I believe there will be many people who share my views when they read this article from you!

  8. Daniella said:

    Many thanks for this amazing post! It’s really helpful for me.

  9. ritariya said:

    To promote business marketing is the good thing. The products should be ideal and fresh. Good products should be sold. They should have proper idea about the business and the product. They should be well firm in selling the product.private tours washington dc

  10. Hard Work said:

    If you really need one of the best solution to access on ecommerce product upload outsourcing services so you may make use of Product Data Entry this hub and can easily move ahead towards your work. Have a visit for once.

  11. asianfanfics said:

    I have read through other blogs, but they are cumbersome and confusing more than your post. I hope you continue to have such quality articles to share with everyone! I believe a lot of people will be like to read this article!

  12. fuwone said:

    The burger is one of the most liked snacks items for many people. is the leader in making and selling such mount watery snacks item.

  13. deertipal said:

    Thank you for the free gift code generator excellent article.Great information, your post free gift code generator is really awesome!Thanks for free gift codes generator posting such a nice blog.

  14. metro pcs bill pay said:

    In this article, we are going to discuss various topics about MetroPCS with some information and instructions.

    metro pcs bill pay |
    metropcs payment |
    metro pcs payment |
    metropcs pay bill

  15. mi ka said:

    The articles you share, it is very detailed and meticulous, I hope you will have more interesting articles to share with readers next time.
    candy crush soda

  16. Alisa said:

    Thank you for posting such a great article! I found your website perfect for my needs. coolmath last horizon

  17. Elisa said:

    I think this is a really good article. You make this information interesting and engaging. cool math games parking

  18. mohit said:

    Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as
    best free sports streaming sites

  19. pranav said:

    bovaj mast blog chhe aa mane bau gamyu.

  20. run 3 said:

    I have read a lot of articles, but for your article, it left me a deep impression, thank you for sharing.

  21. jennifer Winget said:

    It’s a nice article. I really like it. I like your writing style. hostsailor I’m always enjoying reading your articles and notes. From this article, we can understand the flaw in the A/b split test. Thank you for share it with us.

  22. Patricia Hoover said:

    Thanks for sharing visit this website:

  23. juice said:

    helpful tutorial. directx-12-download-for-windows-10

  24. majece majece said:

    After visiting I know more about using spy apps. It’s really important to know in modern world

  25. said:

    What about indigo kids? What kind of phenomena is that and what they can offer to the world?

  26. Spy store said:

    If you are looking for best information on how to spy on your wife cell phone then visit here

  27. Anna said:

    I have read the article on The flaw in the test challenge. This is actually a conversation in the market. This is always a part of the development of the markets. The conversation here is really funny. The author explains about the market strategies.

  28. csub canvas login said:

    Having read this I believed it was really informative. I appreciate you spending some time and effort to put this content together. ttps:// I once again find myself personally spending a significant amount of time both reading and posting comments. But so what, it was still worth it!

  29. majece majece said:

    By the way, information from will help you to learn How to Calculate Grade Point Average. I had such experience few weeks ago

  30. said:

    Excellent and very informative article, thanks

  31. said:

    Do you know that nowadays you can really save your time and hire the expert writing services that will work with your papers.

  32. Matthew said:

    The best part in the run game series. Let’s play run 3 on Android and iOS.

  33. harold26 said:

    You must have a recent purchase receipt from with Survey invitation.
    You should have a basic understanding of either English, Spanish, or French.
    You Should Have a Personal computer, Laptop, Mobile with good internet access.

  34. juliannart said:

    Activate Commonwealth Debit Card is required if you received a Commonwealth Debit Card recently. If you want to use your Commonwealth Debit Card for the first time then you need to activate Debit card to activate my new commonwealth debit card

  35. watsonmeri567 said:

    Hollister Survey asks all participants to rate and review their visit experience with Hollister service, price, environment, staff, cleanliness, and many other points. Hollister Customer Survey takes all your feedback and will try to implement it to improve its guest satisfaction.

  36. Dennis Thomas said:

    The article is a good one. thomasdennis

  37. Wil Ronald said:

    Very good post. Lots of information mentioned in this blog. Thanks for sharing here. html

  38. Wil Ronald said:

    Very good post. Lots of information mentioned in this blog. Thanks for sharing here. html

  39. All Poetry Life said:

    Wow so nice Amazing post, thanks for sharing this article.
    sad poetry in urdu

  40. Wuxiaworld said:

    Pretty good post. I just happened to see your blog and wanted to say that I really enjoy reading your blog posts. I hope you will post again soon. Thank you very much for the useful information.

  41. Wuxiaworld said:

    You have created so many great articles here, that I have read yours a few times. Much of your opinion is consistent with my opinion. This is great content for your readers. Please continue to share!
    Xem thêm v? v?n b?n ngu?n nàyNh?p v?n b?n ngu?n ?? có thông tin d?ch thu?t b? sung

  42. IndigoCard said:

    Thanks for the information. Keep sharing this type of thread.

  43. kigowosds said:

    Floyd Joy Mayweather Jr. is an American professional boxing promoter mayweather vs logan paul purse and former professional boxer. He competed between 1996 and 2015, and made a one-fight comeback in 2017.

  44. alexandrajohn said:

    TalkToFridays: How about getting the chance to receive a TGI Fridays Coupon for Free Entree Meal next time you go to TGI Fridays outlet just by taking the TGI Fridays Survey?

  45. raleveyotah01 said:

    Are you ready to share your feedback with the Nebraska Furniture Mart to win a $1000 gift card? Cause these Nebraska Furniture Mart will get your feedback by

  46. One Dasty said:

    We are dedicated to the breeding of the finest quality, sweet-tempered rough-coated pug puppies at the most reasonable prices. We take great pride in the feedback received from purchasers of our puppies that their puppies have very social and friendly adult dogs. This provides us with the necessary incentive to continue breeding these wonderful, loving and loyal companions.

  47. McOrlando said:

    All of Pitbull dog and puppyflea treated, health certificate, genetic health guarantee, and AKC registration papers. All new families are required to sign a contract that includes a basic health guarantee and a 3 year genetic health guarantee. They have been raised in our home around small children, kittens and dogs of every size to help them develop well round personalities accustomed to daily home activates. We encourage families to ask question and to stay in contact with us.
    All puppies sold as pets will go home spayed-neutered! We also sell puppies along with breeding right. Note this will include an additional fee.

  48. ligabiskepak said:

    Love Books-A-Million books, toys & tech, including ebooks, movies, music & textbooks? How about the service you received during this visit? Books-A-Million looks forward to your feedback that will help them to keep improving.

  49. Every said:

    Every morning when I wake up, I play the cookie clicker game. Bake a cake and feel the morning! Have a good day!

  50. robert said:

    Throughout the years, I often had one, two, or three exotic kittens and the occasional litter of gorgeous Bengals, Servals, Savannah and Caracal kittens.

  51. ts epass said:

    The Department of Backward Class Welfare has started disbursing Pre-Matric and Post-Matric Scholarships online through the ePass initiative, ts epass which is one of the government of Telangana’s major welfare policies aimed at improving students’ educational opportunities. The system connects numerous welfare departments, the Treasury,Colleges, and Banks to grant scholarships.

  52. Wisdom Pharma said:

    Wisdom Pharma offers you all the best selfcare products in one convenient place, such as their best anti aging cream in Pakistan. So, buy from Wisdom Pharma and get high quality Anti-Aging Cream online. Wisdom Pharma brings the best Anti-aging creams in Pakistan. Prevent premature aging today.

  53. said: offers not only the greatest collection of rugs and mats in Pakistan but also a wide variety of home accessories. Gaba has a great collection of leather dustbins online. Bring your house together with our amazing dustbins, available online.

  54. MCA Leads said:

    MCA Live Transfer Leads professional guarantees to deliver Qualified MCA Aged Leads with precise and updated knowledge prepared for conversion. Qualified MCA Leads.

  55. Wisdom Pharma said:

    Wisdom Pharma offers you all the best self-care products in one convenient place, such as their Vitamin-C serums So, buy from Wisdom Pharma and get all the Vitamin C you need online. Wisdom Pharma brings the best Vitamin C serum prices in Pakistan Prevent ageing with Vitamin C today.

  56. Saba Khan said:

    I would thoroughly recommend this water filtration system completely.Absolutely awesome Article.l I could see how each stage works and its actually very straight forward.we are best industrial water softener and
    water filter cartridge suppliers in uae All in all a great.

  57. Sandra James said:

    Yes. The flaw is very apparent and some people don’t notice it. this really helps spread information. Des Moines wedding planner

  58. thanhtongnguyen said:

    run 3 , Thank you for taking the time to talk about it; I have strong feelings about it and want to learn more about it. Would you mind adding information to your blog post? It will really benefit all of us

  59. brianwilson said:

    Hello, I read this nice article. I think You put a best effort to write this perfect wordle 2 numberle article

  60. ipc section said:

    This will make not happen only to the extent same in as far many of years that ago is i would have like my to

  61. stevedaviss said:

    We understand how difficult it might be to complete an assignment. Maybe you’re trying to push into a new grade level ,maybe you’re having trouble figuring out how to organize your work or maybe you just have to speak about a subject that you’re having trouble grasping. When you are looking for online assignment writing services UK order with us, you’re working with some of the most experienced experts in modern academia. We have a huge team of professional writers who undergo rigorous tests before they start working for us

  62. roy smithh said:

    Statistical significance is the best evidence that Version A is actually better than Version B—if the sample size is large enough. Fifty percent statistical significance is a coin toss. If you’re calling tests at 50%, you should change your profession. And no, 75% statistical confidence is not good enough either.

    Any seasoned Online Book Writer has had plenty of experiences in which a “winning” variation at 80% confidence ends up losing badly after getting pushed live to a site and exposed to more traffic.

    What about 90%? That’s pretty good, right? Nope. Not good enough. You’re performing a science experiment. Yes, you want it to be true. You want that 90% to win, but getting the truth is more important than declaring a winner.

  63. Ndure Shoes said:

    There are always amazing sale on brands on the Ndure website, bringing amazing footwear and apparel.

  64. JordanSkemp said:

    Flaw-in-the-ab-split-test-challenge is a short story about a man who has been chosen to participate in a physical test that would be the first step towards an operation to correct his severe scoliosis. The procedure is considered a medical miracle, as it would have saved the man’s life. However, as he is led into the surgery room, he begins to doubt the treatment.

  65. wix logo maker said:

    Sharing Tips : The wix logo maker tools didn’t need registration and provided a free SVG vector file for future usage with no need to pay, also high-resolution PNG is free as well.

  66. nichika said:

    Your feedback helps me a lot, A very meaningful event, I hope everything will go well fnaf security breach

  67. gil12 said:

    Best Shisha in Ireland offers you a wide range of shisha with flavourful drinks to go with it. Bring in your friends and family and enjoy the secluded atmosphere we offer you. We are offering amazing deals on drinks and shisha combos throughout the week.

  68. papa's pizzeria said:

    Copywriting for clients is enjoyable and intellectually demanding. I contend that you cannot succeed by writing just about your own goods.

Leave a Reply