Bob Bly Direct Response Copywriter Official Banner

The flaw in the A/B split test challenge

September 29th, 2017 by Bob Bly

One of the most nonsensical — and fairly common —
conversations in marketing goes something like this….

The ad agency or copywriter submits a promotion.

The client says, “I don’t like it.”

The copywriter gets his dander up, bristles, and says

“It’s very strong. Test it.”

The client refuses.

The copywriter goes on Facebook and says:

“I have a stupid client who refuses to A/B split test.”

Other copywriters chime in and say the client is stupid.

But … in fact, he is not stupid.

A lot of the time, the client is well aware that the A/B split
test is the only legitimate way to test a promotion.

The reason this is true about A/B splits is because only a
marketing test definitively determines whether the copy is strong
based on results, and not on subjective judgment.

But despite this fact, subjective judgment will always be part of
the process of developing marketing campaigns. Reason:

Before your copy can be tested in an A/B split, the team has to
agree on pricing … offer … theme … bonuses … lists … the “big
idea” for the promotion … the headline … the lead … and more.

“Pre-test” and screening of what to ultimately prepare and test
for real with money has to come down to subjective judgment.

Now, you’ve heard the old expression, “Opinions are like
A-holes; everybody has one.”

But of those opinions, the one that counts the most in addition
to the copywriter’s is the client’s, because he is paying the bills
and putting his money on the line.

And having the final say does not make him an A-hole. Far from

Fact of the matter is, you may know more about copywriting than
your client (although in some cases, not).

But the client almost certainly knows more about his business
than you.

Therefore, his instincts and opinions should be considered
carefully, and never ignored or dismissed by you out of hand.

Now, if you feel strongly that nothing beats your copy, and
client criticism annoys you, start your own online information or
merchandise marketing business.

When it’s your business, you can run all your copy exactly as you
wrote it.

But if having the final say is paramount, you should only be an
entrepreneur who writes his own copy.

And not a freelancer writing for copy for clients.

A number of copywriters have made the transition from doing
client work to 100% writing copy for their own products.

Nothing wrong with that. More power to them. I know several, and
a few are almost militant about why theirs is the smarter
copywriting path.

As an ancillary income stream, I also write copy to sell info
products I publish online.

And it by itself generates a six-figure annual income we could
comfortably live on.

But that’s not the way I roll.

Writing copy for clients has pleasures and intellectual
challenges I maintain you simply cannot get from writing only
about your own products.

And as a contract copywriting freelance, I absolutely love the
wide variety of products, services, offers, industries, and
markets I get to write about and for.

It’s more fun than I can shake a stick at.

So overall, no complaints — though on rare occasion I may grumble
a bit.

But for nearly 4 decades, I have been primarily a traditional
copywriter working for clients.

And secondarily an info marketer, book author, consultant, and

That’s where I want to be in my copywriting business.

And as I have done what I wanted to, likewise, you should do what
works for you.

As the late, great David Ogilvy said, quoting an old Scottish

“Be happy while you’re living, for you’re a long time dead.”


This entry was posted on Friday, September 29th, 2017 at 9:35 am and is filed under General, Writing. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

98 responses about “The flaw in the A/B split test challenge”

  1. Donna Batone said:

    I like your writing style! I’m always enjoying reading your articles and notes. Will you be so honor and read my writing piece as well?

  2. Zay said:

    I have been primarily a traditional
    copywriter working for clients. geometry dash game

  3. Vernonk Mang said:

    John arnold is an academic writer of the Dissertation-Guidance. Who writes quality academic papers for students to help them in accomplishing their goals. Marketing Dissertation Writing Service

  4. sakib khan said:

    February 2018 Calendar word

  5. best tower fan in india said:

    Choose best tower fan in India to beat the summer and get super cool with best and branded tower fans and with Comparisons.

  6. vex 3 said:

    I have had a lot of harvest after watching this article from you! I feel it interesting, your post gave me a new perspective! I have read many other articles about the same topic, but your article convinced me! I hope you continue to have high quality articles like this to share with veryone!

  7. driving directions said:

    Thanks for sharing.I hope you continue to have such quality articles to share with everyone! I believe there will be many people who share my views when they read this article from you!

  8. Daniella said:

    Many thanks for this amazing post! It’s really helpful for me.

  9. ritariya said:

    To promote business marketing is the good thing. The products should be ideal and fresh. Good products should be sold. They should have proper idea about the business and the product. They should be well firm in selling the product.private tours washington dc

  10. Hard Work said:

    If you really need one of the best solution to access on ecommerce product upload outsourcing services so you may make use of Product Data Entry this hub and can easily move ahead towards your work. Have a visit for once.

  11. asianfanfics said:

    I have read through other blogs, but they are cumbersome and confusing more than your post. I hope you continue to have such quality articles to share with everyone! I believe a lot of people will be like to read this article!

  12. fuwone said:

    The burger is one of the most liked snacks items for many people. is the leader in making and selling such mount watery snacks item.

  13. deertipal said:

    Thank you for the free gift code generator excellent article.Great information, your post free gift code generator is really awesome!Thanks for free gift codes generator posting such a nice blog.

  14. metro pcs bill pay said:

    In this article, we are going to discuss various topics about MetroPCS with some information and instructions.

    metro pcs bill pay |
    metropcs payment |
    metro pcs payment |
    metropcs pay bill

  15. mi ka said:

    The articles you share, it is very detailed and meticulous, I hope you will have more interesting articles to share with readers next time.
    candy crush soda

  16. Alisa said:

    Thank you for posting such a great article! I found your website perfect for my needs. coolmath last horizon

  17. Elisa said:

    I think this is a really good article. You make this information interesting and engaging. cool math games parking

  18. mohit said:

    Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as
    best free sports streaming sites

  19. pranav said:

    bovaj mast blog chhe aa mane bau gamyu.

  20. run 3 said:

    I have read a lot of articles, but for your article, it left me a deep impression, thank you for sharing.

  21. jennifer Winget said:

    It’s a nice article. I really like it. I like your writing style. hostsailor I’m always enjoying reading your articles and notes. From this article, we can understand the flaw in the A/b split test. Thank you for share it with us.

  22. Patricia Hoover said:

    Thanks for sharing visit this website:

  23. juice said:

    helpful tutorial. directx-12-download-for-windows-10

  24. majece majece said:

    After visiting I know more about using spy apps. It’s really important to know in modern world

  25. said:

    What about indigo kids? What kind of phenomena is that and what they can offer to the world?

  26. Spy store said:

    If you are looking for best information on how to spy on your wife cell phone then visit here

  27. Anna said:

    I have read the article on The flaw in the test challenge. This is actually a conversation in the market. This is always a part of the development of the markets. The conversation here is really funny. The author explains about the market strategies.

  28. csub canvas login said:

    Having read this I believed it was really informative. I appreciate you spending some time and effort to put this content together. ttps:// I once again find myself personally spending a significant amount of time both reading and posting comments. But so what, it was still worth it!

  29. majece majece said:

    By the way, information from will help you to learn How to Calculate Grade Point Average. I had such experience few weeks ago

  30. said:

    Excellent and very informative article, thanks

  31. said:

    Do you know that nowadays you can really save your time and hire the expert writing services that will work with your papers.

  32. Matthew said:

    The best part in the run game series. Let’s play run 3 on Android and iOS.

  33. harold26 said:

    You must have a recent purchase receipt from with Survey invitation.
    You should have a basic understanding of either English, Spanish, or French.
    You Should Have a Personal computer, Laptop, Mobile with good internet access.

  34. juliannart said:

    Activate Commonwealth Debit Card is required if you received a Commonwealth Debit Card recently. If you want to use your Commonwealth Debit Card for the first time then you need to activate Debit card to activate my new commonwealth debit card

  35. watsonmeri567 said:

    Hollister Survey asks all participants to rate and review their visit experience with Hollister service, price, environment, staff, cleanliness, and many other points. Hollister Customer Survey takes all your feedback and will try to implement it to improve its guest satisfaction.

  36. Dennis Thomas said:

    The article is a good one. thomasdennis

  37. Wil Ronald said:

    Very good post. Lots of information mentioned in this blog. Thanks for sharing here. html

  38. Wil Ronald said:

    Very good post. Lots of information mentioned in this blog. Thanks for sharing here. html

  39. All Poetry Life said:

    Wow so nice Amazing post, thanks for sharing this article.
    sad poetry in urdu

  40. Wuxiaworld said:

    Pretty good post. I just happened to see your blog and wanted to say that I really enjoy reading your blog posts. I hope you will post again soon. Thank you very much for the useful information.

  41. Wuxiaworld said:

    You have created so many great articles here, that I have read yours a few times. Much of your opinion is consistent with my opinion. This is great content for your readers. Please continue to share!
    Xem thêm v? v?n b?n ngu?n nàyNh?p v?n b?n ngu?n ?? có thông tin d?ch thu?t b? sung

  42. IndigoCard said:

    Thanks for the information. Keep sharing this type of thread.

  43. kigowosds said:

    Floyd Joy Mayweather Jr. is an American professional boxing promoter mayweather vs logan paul purse and former professional boxer. He competed between 1996 and 2015, and made a one-fight comeback in 2017.

  44. alexandrajohn said:

    TalkToFridays: How about getting the chance to receive a TGI Fridays Coupon for Free Entree Meal next time you go to TGI Fridays outlet just by taking the TGI Fridays Survey?

  45. raleveyotah01 said:

    Are you ready to share your feedback with the Nebraska Furniture Mart to win a $1000 gift card? Cause these Nebraska Furniture Mart will get your feedback by

  46. One Dasty said:

    We are dedicated to the breeding of the finest quality, sweet-tempered rough-coated pug puppies at the most reasonable prices. We take great pride in the feedback received from purchasers of our puppies that their puppies have very social and friendly adult dogs. This provides us with the necessary incentive to continue breeding these wonderful, loving and loyal companions.

  47. McOrlando said:

    All of Pitbull dog and puppyflea treated, health certificate, genetic health guarantee, and AKC registration papers. All new families are required to sign a contract that includes a basic health guarantee and a 3 year genetic health guarantee. They have been raised in our home around small children, kittens and dogs of every size to help them develop well round personalities accustomed to daily home activates. We encourage families to ask question and to stay in contact with us.
    All puppies sold as pets will go home spayed-neutered! We also sell puppies along with breeding right. Note this will include an additional fee.

  48. ligabiskepak said:

    Love Books-A-Million books, toys & tech, including ebooks, movies, music & textbooks? How about the service you received during this visit? Books-A-Million looks forward to your feedback that will help them to keep improving.

  49. Every said:

    Every morning when I wake up, I play the cookie clicker game. Bake a cake and feel the morning! Have a good day!

  50. robert said:

    Throughout the years, I often had one, two, or three exotic kittens and the occasional litter of gorgeous Bengals, Servals, Savannah and Caracal kittens.

  51. ts epass said:

    The Department of Backward Class Welfare has started disbursing Pre-Matric and Post-Matric Scholarships online through the ePass initiative, ts epass which is one of the government of Telangana’s major welfare policies aimed at improving students’ educational opportunities. The system connects numerous welfare departments, the Treasury,Colleges, and Banks to grant scholarships.

  52. Wisdom Pharma said:

    Wisdom Pharma offers you all the best selfcare products in one convenient place, such as their best anti aging cream in Pakistan. So, buy from Wisdom Pharma and get high quality Anti-Aging Cream online. Wisdom Pharma brings the best Anti-aging creams in Pakistan. Prevent premature aging today.

  53. said: offers not only the greatest collection of rugs and mats in Pakistan but also a wide variety of home accessories. Gaba has a great collection of leather dustbins online. Bring your house together with our amazing dustbins, available online.

  54. MCA Leads said:

    MCA Live Transfer Leads professional guarantees to deliver Qualified MCA Aged Leads with precise and updated knowledge prepared for conversion. Qualified MCA Leads.

  55. Wisdom Pharma said:

    Wisdom Pharma offers you all the best self-care products in one convenient place, such as their Vitamin-C serums So, buy from Wisdom Pharma and get all the Vitamin C you need online. Wisdom Pharma brings the best Vitamin C serum prices in Pakistan Prevent ageing with Vitamin C today.

  56. Saba Khan said:

    I would thoroughly recommend this water filtration system completely.Absolutely awesome Article.l I could see how each stage works and its actually very straight forward.we are best industrial water softener and
    water filter cartridge suppliers in uae All in all a great.

  57. Sandra James said:

    Yes. The flaw is very apparent and some people don’t notice it. this really helps spread information. Des Moines wedding planner

  58. thanhtongnguyen said:

    run 3 , Thank you for taking the time to talk about it; I have strong feelings about it and want to learn more about it. Would you mind adding information to your blog post? It will really benefit all of us

  59. brianwilson said:

    Hello, I read this nice article. I think You put a best effort to write this perfect wordle 2 numberle article

  60. ipc section said:

    This will make not happen only to the extent same in as far many of years that ago is i would have like my to

  61. stevedaviss said:

    We understand how difficult it might be to complete an assignment. Maybe you’re trying to push into a new grade level ,maybe you’re having trouble figuring out how to organize your work or maybe you just have to speak about a subject that you’re having trouble grasping. When you are looking for online assignment writing services UK order with us, you’re working with some of the most experienced experts in modern academia. We have a huge team of professional writers who undergo rigorous tests before they start working for us

  62. roy smithh said:

    Statistical significance is the best evidence that Version A is actually better than Version B—if the sample size is large enough. Fifty percent statistical significance is a coin toss. If you’re calling tests at 50%, you should change your profession. And no, 75% statistical confidence is not good enough either.

    Any seasoned Online Book Writer has had plenty of experiences in which a “winning” variation at 80% confidence ends up losing badly after getting pushed live to a site and exposed to more traffic.

    What about 90%? That’s pretty good, right? Nope. Not good enough. You’re performing a science experiment. Yes, you want it to be true. You want that 90% to win, but getting the truth is more important than declaring a winner.

  63. Ndure Shoes said:

    There are always amazing sale on brands on the Ndure website, bringing amazing footwear and apparel.

  64. JordanSkemp said:

    Flaw-in-the-ab-split-test-challenge is a short story about a man who has been chosen to participate in a physical test that would be the first step towards an operation to correct his severe scoliosis. The procedure is considered a medical miracle, as it would have saved the man’s life. However, as he is led into the surgery room, he begins to doubt the treatment.

  65. wix logo maker said:

    Sharing Tips : The wix logo maker tools didn’t need registration and provided a free SVG vector file for future usage with no need to pay, also high-resolution PNG is free as well.

  66. nichika said:

    Your feedback helps me a lot, A very meaningful event, I hope everything will go well fnaf security breach

  67. gil12 said:

    Best Shisha in Ireland offers you a wide range of shisha with flavourful drinks to go with it. Bring in your friends and family and enjoy the secluded atmosphere we offer you. We are offering amazing deals on drinks and shisha combos throughout the week.

  68. papa's pizzeria said:

    Copywriting for clients is enjoyable and intellectually demanding. I contend that you cannot succeed by writing just about your own goods.

  69. Love Me said:

    Great Work check this Boys and girls

  70. Shahtaj said:

    Shahtaj Qaiser offers a wide range of party wear dress for girls and women in Pakistan. Discover the luxurious party dresses with various designs at low price. Grab your favorite dresses online today.

  71. Maples Closet said:

    Shop the gorgeous and sophisticated selection of Jalabiya gowns from Maples Closet, which were modelled by traditional UAE clothing. Our collection features both contemporary and conventional styles made from premium fabrics with exquisite craftsmanship. Our Jalabiya dress uae is the ideal option whether you’re attending a major occasion or simply searching for a casual yet fashionable attire. Experience the elegance of Maple’s Closet’s assortment by shopping right away.

  72. lahanasas said:

    The New Zealand public rugby association group, regularly known Watch All Blacks vs Australia live as the All Blacks, addresses New Zealand in men’s global rugby association, which is viewed as the country’s public game.

  73. CaVPShost said:

    Remote Desktop, also known as Remote Desktop Protocol (RDP), allows you to access and control a computer or server remotely from another device. It enables you to interact with the remote server as if you were sitting in front of it, providing a convenient way to manage your server or access files and applications remotely.

  74. BSNS Consulting said:

    Bsns Consulting is a trusted digital marketing consultancy that specializes in providing comprehensive solutions to businesses looking to enhance their online presence and maximize their digital marketing efforts. With a deep understanding of the evolving digital landscape, our team of experts offers strategic guidance and practical expertise to help businesses effectively navigate the digital realm.

  75. Dropwarez said:

    Uncover hidden gems in the e-commerce landscape with DropWarez’s product research services. Our in-depth analy is and market insights empower you to make informed decisions, ensuring that your product offerings resonate with your audience.In a rapidly evolving and competitive business landscape, making informed decisions about what products to offer is crucial for achieving sustained growth and profitability.

  76. BabyBoxShop said:

    Explore our delightful collection of pink gifts, perfect for welcoming a baby girl into the world. From pretty dresses and soft blankets to charming toys, our pink-themed selection offers a variety of choices to celebrate the new addition to the family.

  77. Vallee Real Estate said:

    Trust The Property Brokers at Vallee Real Estate Group for expert guidance in buying or selling real estate. Our dedicated team specializes in delivering exceptional service and unparalleled market knowledge. Start your property journey with The Property Brokers today

  78. CAvpshost said:

    Welcome to CaVPS Host, your destination for the cheapest Windows Server hosting solutions! With our affordable plans and top-notch services, we guarantee a seamless hosting experience for your website or application. At CaVPS Host, we understand the importance of cost-effective solutions without compromising on quality. That’s why we offer the cheapest Windows Server hosting options available in the market. Our prices are designed to fit your budget, making it easy for small businesses, startups, and individuals to establish their online presence without breaking the bank.

  79. Tiptop-Einrichtung said:

    Friseureinrichtung refers to the overall furnishings and equipment within a hair salon. Creating the perfect ambiance in a hair salon is essential for both customers and staff. “Friseureinrichtung,” or hair salon furnishings, encompasses a range of elements that contribute to the overall design and functionality of a salon.

  80. BSNS Consulting said:

    At BSNS Consulting, we offer you unparalleled Business Advisory Services near me. Our team of expert consultants is dedicated to providing personalised solutions that drive growth and success for your business. Whether you’re a startup or an established company, our strategic insights and industry knowledge will empower you to make informed decisions and achieve your goals.

  81. Vallee Real Estate said:

    Investors seeking commercial opportunities will appreciate our listings of commercial buildings for sale. We provide valuable insights and assistance for profitable acquisitions.These properties are intended for use by companies, organizations, or entrepreneurs looking to operate their businesses, rather than for residential purposes. Commercial buildings come in various types and sizes to accommodate different types of businesses.

  82. Tiptop-Einrichtung said:

    A barberstuhl is a specialized chair used in barbershops. It is designed to provide comfort to male clients and facilitate optimal working posture for the barber.hese chairs often have a classic design and can be antique or retro-styled.

  83. Nyt Wordle said:

    Wordle Nyt you get six chances each day to identify the secret wordle Game. The color of your word’s tiles will change after each guess to indicate how close you came to getting it right New York Times Wordle.

    Wordle Nyt

    Nyt Wordle



  84. ARUN VIRAT said:

    Worlde Game you get six chances each day to identify the secret wordle Game. The color of your word’s tiles will change after each guess to indicate how close you came to getting it right Wordle.


  85. Vallee Real Estate said:

    The New Westminster real estate market offers a mix of housing styles, from historic heritage homes with unique architectural features to contemporary developments with modern amenities. The city’s real estate is influenced by various factors, including its proximity to Vancouver, public transportation options, local schools, parks, and the overall quality of life it offers.

  86. Vallee Real Estate said:

    Discover the perfect blend of elegance and convenience with this exceptional duplex for sale in New Westminster. This property offers a unique opportunity to own a beautifully designed home with the benefits of a duplex layout. Whether you’re a discerning buyer or an investor, this property promises both comfort and potential.

  87. SunCrop Group said:

    Suncrop Group is a dynamic and multifaceted conglomerate based in Pakistan, with a rich legacy of excellence and innovation. The group encompasses a variety of companies, each specializing in different sectors, demonstrating its versatile and expansive reach in the industry. world best pesticide company

  88. Twicsy said:

    How do you unmute Instagram Story? Instagram is a large network that is used all over the world. People are madly in love with this social media platform just because they can share their moments with others by adding stories. They display their current happenings to show the others. Not everyone likes to see all these things. Twicsy

  89. Gudmag said:

    ? “Igniting conversations through the power of words!” ? That’s me, Martina Pagani! For the past 5 years, I’ve been serving as a professional blog post article writer for GudMag ?, where I’ve had the privilege to craft compelling narratives and share stories that matter to millions. I’ve dipped my toes into a diverse pool of categories, always keen to learn and expand my knowledge. My writing journey has been nothing short of an exciting roller coaster ride ?, filled with creative challenges, and opportunities to showcase my expertise. This journey has not only honed my skills but also helped me understand the nuances of SEO and the art of conversation optimization. Beyond just writing, I’m passionate about digital marketing and the dynamic world it encapsulates. From strategizing to execution, every stage is a thrilling opportunity to connect with audiences and make an impact. ? My secret? A blend of creativity, strategy, and a dash of coffee! ? If you’re looking for someone to create content that not only ranks but also resonates, you know where to find me! Let’s ignite meaningful conversations together! ? Remember, in the world of content, ‘Good’ is not enough, you need to be ‘Gud’. ? Gudmag

  90. Leads Burners said:

    Leads Burner brings all latest and Digital news, online Digital information, tech news & updates from the world’s top leading industries and businesses.Leads Burners

  91. Himalayan Salt said:

    Beyond the kitchen, Himalayan Pink Salt has an attraction. We learn about its uses for relaxation and beauty, including calming bath salts and exfoliating scrubs. Examine how this organic component has established itself as a mainstay in self-care regimens.

  92. Lucky Cola said:

    Conquer Challenges, Collect Rewards: Your Victory, Your Rules! Lucky Cola

  93. Ginkgo Retail said:

    Ginkgo Retail is an innovative domain poised at the intersection of streamlined ecommerce and efficient order management solutions. It serves as a dynamic platform catering to businesses seeking comprehensive, user-friendly systems for their operations. With a focus on seamless integration and accessibility, GinkgoRetail offers an advanced Ecommerce Order Management System, empowering businesses to navigate, track, and optimize their processes effortlessly. Its robust features include Ecommerce Order Management System PDF
    functionalities, ensuring clear and concise documentation throughout the entire order lifecycle.

  94. TipTop Enrichtung said:

    A Barbershop Bedienplatz is the workspace within a barbershop where haircuts, shaving, and other services for male clients are carried out. This area typically includes a chair, mirror, and tools.The Barbershop Service Station is the heart of every barbershop, where skilled barbers work their magic to create fresh looks and maintain well-groomed appearances. This dedicated area is thoughtfully designed for both the barber’s convenience and the client’s comfort.

  95. We Sell Stocks said:

    Explore the pinnacle of automation technology with We Sell Stocks, your go-to destination for Programmable Logic Controller (PLCs). Our diverse inventory showcases high-quality PLCs designed for various industrial applications, ensuring top-notch performance and efficiency. Whether you’re upgrading manufacturing processes or enhancing automation solutions, find the perfect PLC to meet your unique needs at competitive prices. “We Sell Stocks” is committed to delivering excellence in quality and customer satisfaction, making us your trusted partner for acquiring advanced PLC technology. Revolutionize your operations with our exceptional selection of Programmable Logic Controllers, setting the standard for efficiency and innovation.

  96. Lodibet said:

    Customize your playstyle and create a unique gaming experience. Lodibet

  97. Go Read said:

    Grow Your Social Media Profile Instantly With GoreadSocialMedia. As you can, Followers and Likes, or can grow your profiles.

  98. Goku.Tu said:

    I am happy to find this article content very helpful and valuable for everyone, as it contains lot of topic information.

Leave a Reply