A/B testing in ecommerce: 10 things you really need to test

This is what we are going to talk about:

  • What are A/B tests?
  • Where can you apply A/B testing?
  • The process of testing
  • What is worth testing?
    • Visual elements
    • CTA
    • Navigation
    • Pop-up windows
    • Urging solutions
    • Payment solutions
    • Forms, fields to be completed
    • Registration
    • Text
    • Offers
  • And everything else


Let’s start the story at the beginning: what are A/B tests?

As its name implies, A/B tests are trials, during which you try out two different variants of a given element. For example you use a red CTA (call to action) button on one of the pages and a blue one on the other. You place the search field at the top on one of the pages and to the right on the other.


Criteria of a good A/B test:

  • Test only one element at a time – otherwise you will not know what the changes in results are due to.
  • Run the two pages parallel and simultaneously – it may easily take an effect on your results if you run the tests with a difference of a few days. The test should run at least for a week.
  • Use a significant sample: in case of public opinion surveys the minimal limit is 1000 people, this is the optimal for you, too.
  • Distribute the visitors arriving at the two pages randomly: if you select based on segments, channels or any other methods, your results will be distorted.


It will be obvious after the testing which of the two solutions your customers like better: which encourage them to remain on the page, to register, to click through or even to purchase.

However, not only your results will be better. It also has a positive effect on the work ethics of the employees of your company that they do not have to rely solely on their intuition and the results are not requested from them while all they can do is guess: you can work with accurate data and precise analytics instead.

Such things may be inspiring for those in marketing and for the designers, because you do not let go of their hands while demanding results.


Where can you apply A/B tests? Practically everywhere

There is no element on your web-site, be it text, graphics, design or function, whose effectiveness you can be sure of without testing.

tips How can you run the test efficiently?:


Usually by having it done by a professional. Even free trial version tools are available that can help you (for example Optimizely). However, if you work with a complex online store, you will need a developer, especially because this is the only way you can make sure that the criteria above will fully be realized.

In the following, you can read a list of ideas about what kind of elements you can run A/B tests on. You can run several of these simultaneously, but you have to accept that you will not be able to analyse everything at the same time.

Do not overburden yourself with data, try to achieve as good results as possible while collecting and utilizing information of a humanly processable quantity.

The number of visitors may limit the testing: if you run 3-4 tests at the same time on a page, the visitors will be split into 6 to 8 parts, which can extremely elongate the duration of the test. Provided that you want to work with a significant sample. (Yes, you do.)


Process of testing: know what you want an prioritize your goals

First you have to be clear regarding your intentions: what would you like to achieve?

It is not at all the same if you want the visitors to stay on the page for a longer time, or if it does not matter how long they stay provided that they execute a certain action.

Once you know what you want, list the main elements of the relevant pages and prioritize them according to importance.




  • The CTA and the form are of the utmost importance on a registration page.
  • In case of an article or a blog entry, multimedia solutions and certain characteristics (length, style, etc.) of the text should be tested.
  • On the main page of an online shop you might want to pay attention to how products are displayed, where the search field is, how your pop-up windows work.


You can monitor the analytics already at the definition of the priorities: if for example you notice that the majority of the visitors get to the CTA button on the page but they do not click on it, you will have to find out what may cause this.

For the launch of the test you should have an already existing page, for example a product page. Select an element that you have doubts about: a CTA, a block of text, a form to be filled in. Make a new variant of it, try to make it more effective and you will have your A and B pages right away.


What is worth testing?


1) Visual elements: make them as appealing as possible


Visual resources are basic elements of a product page. What is good about shopping in person is that you can touch the goods, look at them from every angle, you can examine them with all of your senses. In case of online shopping the salesman has to strive to deliver this experience as fully as possible. If you are selling a leather belt, use images or make videos that will make your visitor feel the scent of the real tanned leather even if only sitting in front of the monitor.

Of course the question is what kind of solutions work best. It is advisable to use as many and as high quality images as possible. This is done, and still you are not at all done with your possibilities, however, you cannot squeeze everything into one page. A fine balance should be achieved in the end.


You can use

  • rotatable models,
  • longer how-to videos,
  • infographics,
  • 10-15 second demos,
  • or even sound files.

And you can test all these solutions, even according to content: it is not at all the same if you embed a demonstration video or a how-to video with tips in the page.


2) CTA ‒ What do your customers want to click on most?


In general, you would like the users to click on something. This is not as easy to achieve as you might think at first. What characteristics does a CTA have? Position, shape, colour, text… And each and every one of these can have a great effect on conversion.

If you place the CTA in the wrong place, less people will click on it. (According to the generally accepted rule, the more complex the offer, the lower it will be placed on the page.) However, it may also be fine if there are even two or three CTAs on the page, because each member of the target audience comes to a decision at different sections of the page.

If the CTA is too small, it will not be visible enough, if it is too big, it will be deterrent.

Colour is also an important factor:

According to the colour theory, some colours have an urging effect, some strengthen your sense of security, and so on. But it also may be that the CTA simply does not differ enough from its surroundings to be visible…

And you can fine-tune the text to the extreme: every single word has significance, they all have different effects on your audience.




3) Navigation ‒ drop-down, breadcrumb, categories…


You can apply a number of various navigation elements within a page – this way you can find out how your audience reacts to each of them.




It may be a question

  • Whether you should put the categories on the right or on the left side.
  • Whether you should use a drop-down menu or perhaps a floating solution.
  • You may even test which categories you should position higher within a menu, which should be more in focus, because they can bring more clicks.
  • If you use breadcrumb navigation, there are various available of those as well (path-based, hierarchy-based or attribute-based).

In this case you have to monitor how much time the users spend on the page, how many sub-pages they view, how they behave, in other words, what they click on within the page. If they click through more to your other pages in case of one of the solutions and less of them leave the page, you have found the better navigation.


4) Pop-up windows ‒ annoying but effective?


You can use the pop-up windows for many things:

  • to request registration,
  • to encourage to social sharing,
  • to offer similar products or services,
  • to prevent the users from leaving your page by offering an immediate discount.


However, it does make a difference what kind of it you use – a window that covers the whole page or one that appears discreetly in the lower corner?

When do you display the pop-up window? After a certain time or when the user makes some sort of action, or maybe when you feel passiveness?


pop up


5) Urging solutions ‒ coping with buyer hesitation


Try how a little urging influences your customers!

In our exhaustively detailed article we wrote about how you can do that gently: by indicating that a given offer is valid only for a certain time period (even by using a counter), by displaying the number of pieces or another way. See: 5 proven strategies to overcome buyer hesitation

Make variants of your already existing product pages where you place one of these elements – but make sure that your different pages do not contain different information. It may be annoying for the users if they view a page twice and at one time they see a limited time period for an offer and at the second time the counter is not there anymore.

Therefore, if you take a product page that has not included a counter previously, make two variants that show the limited offer in different ways (one of them for example by using an active counter and the other by simply indicating a deadline) and monitor whether the conversion increases compared to the previous data and also in which case by how much.




6) Payment solutions ‒ the simpler for the shopper, the better


In certain ecmmerce stores the customers have to fight their way through four or five pages until they can pay and order the product, in others, one or two pages are enough to be able to do that.

Too many pages can increase the number of cart abandoners, which is why it is worthwhile for you trying whether you can shorten the process somewhere, and testing the solutions of more and also of fewer pages.

This may be an especially important factor if a lot of your visitors arrive from mobile devices: they are usually more impatient, so having to carry out an action through several pages significantly spoils their user experience. In their case you can try for example a solution where they can go through the whole procedure on a single page.


payment methods


7) Forms & fields to be completed


According to certain studies, your conversion rate decreases by approximately 10% with each additional field to be completed.

Users do not like when you ask for too much data – on the other hand, you can make much use of that data during the segmentation later in email marketing.

So test how much data you can request “with impunity” – can you ask for example position, company name, city or zip code beside the name and email address?

It is even more true in case of sensitive information that people are less willing to provide them – also test whether you are better off with mandatory fields (this may result in somewhat less leads but a list that can be more precisely segmented) or with optional ones (with as many email addresses as possible but with incomplete data).




8) Registration


By requesting registration for being able to make a purchase, you usually increase significantly the number of cart abandoners and bouncers. In certain cases however, this may be a loss that you will profit more from in the end.

It is possible that you will have less customers like that, but you will also be able to target them again more effectively later: you will have more data, you will be able to contact them through multiple channels, you can rely on their previous behaviour more securely.

And those who have already purchased from you, can always be contacted and converted again more cost-effectively than new customers. Therefore you should test mandatory and optional registration and calculate whether it is worth more in the long run sacrificing a few newcomers in exchange for the customers that you will be able to reactivate more easily later.




9) Text


Test what sorts of texts work well on your product pages! The long ones that work with storytelling, or the shorter and more to-the-point copy brings more conversions?

What style do you write in?

If you offer business services to large companies, perhaps the official tone will bring you more customers. On the other hand, if you are selling skateboards, you will most probably be better off with a cooler style.

You may try the negative and positive tones as well. There is a huge difference between “Buy it now!” and “Add it to cart or you will miss it!” Some customer segments may react negatively on such urging, while in other groups such scarcity can significantly increase conversions.

What kind of headlines, page titles do you use? It does matter whether you attract attention with creative or informative solutions.




10) Offers


It does make a difference what kind of offers you make to the users. We have already written about holding back information being responsible for an important part of cart abandonments, just like many times the non-indication of delivery costs.

It is possible that many people will add the product to cart if they do not see that they will also have to pay for delivery, but many of them will leave it in the cart unpaid as well, because they will feel you betrayed them if you make them realize the “real” or total price only at payment.

Consequently, you can indicate the shipping cost on the product page, moreover, we even suggest you do so – however, you might as well try offering free delivery under given conditions, for example if customers purchase products for above a certain amount, or order a given number of pieces at once.

You can increase the average value of your orders by that, given which free delivery will already be worthwhile for you, and at the same time you do not hide the costs either – but it is wort a thought whether you will lose more customers by that and it may be more beneficial to direct them towards a larger order.




+1: And everything else


There is practically no element whose operation you could not test, however, as we have already mentioned at the beginning of the article, you cannot test everything simultaneously.


Find out which points are critical based on the data:

  • on which pages the most customers bounce back?
  • for which products are there the least conversions?
  • where does activity fall or soar spectacularly following a given modification?


It is recommended that you try your own page yourself as well: examine where you look, where you click, which are the points where you yourself would drop the purchase.

A/B testing can be a real secret weapon, studies clearly show that you can improve the conversion rate of a page by a percentage of at least two digits with a single tiny modification.

To be able to realize that, you will have to carry out the testing smartly though – probably it will be best for you to start with the tips described in this article.


[bctt tweet=”With only a minor change on your product page you can considerably improve conversion.” username=”aionhill”]


If you consider implementing an A/B testing solution for Magento stores, the extension “A/B Testing” developed by Amasty seems a good option. You can run different split tests and multivariate tests in terms of design or different attributes, can set test duration and can have statistical reports about the outcome of the tests.

Thanks to its world-class features and robustness, it is probably the most popular Magento extension for A/B testing.


Magento A/B testing Amasty extension



  • Run numerous a/b tests simultaneously
  • Test particular attributes for a group of products at a time
  • Test product page design
  • Get informative stats reports
  • Manage test duration
  • Display test variants to customers according to conversion rates



We hope that the practical examples in the article put in the mood for A/B testing even those who had previously considered them to be unnecessary.

You can be sure that the changes you make based on these results will be felt in increased conversion rates.

If you have any questions, do not hesitate to ask away in the comment section or contact us, we are glad to help.



110 replies
  1. Lannie Paoletto says:

    Simply wish to say your article is as surprising. The clarity for your publish is simply spectacular and i could assume you’re knowledgeable on this subject. Fine together with your permission let me to grasp your RSS feed to keep updated with imminent post. Thanks 1,000,000 and please keep up the enjoyable work.

  2. Trend Trader says:

    We’re a bunch of volunteers and opening a brand new scheme in our community. Your web site offered us with helpful information to paintings on. You have performed an impressive job and our whole neighborhood shall be thankful to you.

  3. data stock says:

    I am often to blogging and i really appreciate your content. The article has really peaks my interest. I am going to bookmark your site and keep checking for new information.

  4. technical analysis az says:

    Its like you read my mind! You seem to know so much about this, like you wrote the book in it or something. I think that you could do with some pics to drive the message home a little bit, but instead of that, this is wonderful blog. An excellent read. I’ll certainly be back.

  5. what is hysteria says:

    obviously like your web-site but you need to check the spelling on quite a few of your posts. Many of them are rife with spelling problems and I in finding it very bothersome to tell the reality on the other hand I’ll definitely come back again.

  6. Vibrador says:

    I?¦ve recently started a website, the information you provide on this web site has helped me tremendously. Thanks for all of your time & work.

  7. Serbian clothes says:

    Good write-up, I am normal visitor of one?¦s website, maintain up the nice operate, and It is going to be a regular visitor for a lengthy time.

  8. Cheapest OpenVZ VPS says:

    Hi would you mind stating which blog platform you’re using? I’m planning to start my own blog soon but I’m having a tough time making a decision between BlogEngine/Wordpress/B2evolution and Drupal. The reason I ask is because your layout seems different then most blogs and I’m looking for something unique. P.S My apologies for getting off-topic but I had to ask!

  9. hospedagem de sites says:

    Howdy! I’m at work browsing your blog from my new apple iphone! Just wanted to say I love reading your blog and look forward to all your posts! Carry on the fantastic work!

  10. projetos arquitetônico says:

    naturally like your web-site but you have to take a look at the spelling on quite a few of your posts. Several of them are rife with spelling issues and I to find it very troublesome to tell the truth on the other hand I will surely come back again.

  11. Low Price SMM Services says:

    Hi there just wanted to give you a quick heads up and let you know a few of the pictures aren’t loading properly. I’m not sure why but I think its a linking issue. I’ve tried it in two different web browsers and both show the same results.

  12. Auto Liker for Instagram says:

    I would like to thnkx for the efforts you have put in writing this blog. I am hoping the same high-grade blog post from you in the upcoming as well. In fact your creative writing abilities has inspired me to get my own blog now. Really the blogging is spreading its wings quickly. Your write up is a good example of it.

  13. SMM Panel says:

    Hi there! This post couldn’t be written any better! Reading through this post reminds me of my previous room mate! He always kept talking about this. I will forward this article to him. Pretty sure he will have a good read. Thank you for sharing!

  14. bhim family says:

    Hello there, I found your blog via Google while looking for a related topic, your site came up, it looks great. I’ve bookmarked it in my google bookmarks.

  15. cogumelos shimeji says:

    F*ckin’ awesome issues here. I am very satisfied to look your post. Thanks so much and i am taking a look ahead to touch you. Will you please drop me a mail?

  16. bart carts says:

    Oh my goodness! a tremendous article dude. Thanks Nonetheless I am experiencing subject with ur rss . Don’t know why Unable to subscribe to it. Is there anybody getting an identical rss drawback? Anyone who is aware of kindly respond. Thnkx

  17. retrognatismo says:

    I’ve read some good stuff here. Definitely price bookmarking for revisiting. I wonder how so much effort you put to create the sort of excellent informative website.

  18. Newport says:

    Some genuinely nice and useful info on this web site, as well I think the design and style contains wonderful features.

  19. marketing 1on1 agency says:

    I am really impressed together with your writing abilities as smartly as with the structure on your blog. Is this a paid theme or did you customize it yourself? Either way keep up the nice quality writing, it is rare to peer a great weblog like this one these days..

  20. marketing 1 on 1 says:

    I’m not that much of a online reader to be honest but your blogs really nice, keep it up! I’ll go ahead and bookmark your site to come back down the road. All the best

  21. Jeff Cox Californian Estates says:

    Hello there! Quick question that’s entirely off topic. Do you know how to make your site mobile friendly? My web site looks weird when viewing from my apple iphone. I’m trying to find a template or plugin that might be able to correct this problem. If you have any suggestions, please share. Cheers!

  22. car window coating says:

    Nice post. I was checking continuously this blog and I’m impressed! Very useful info specially the last part :) I care for such info much. I was looking for this certain info for a very long time. Thank you and best of luck.

  23. sewa kereta mewah murah says:

    I’ve been absent for a while, but now I remember why I used to love this web site. Thank you, I’ll try and check back more frequently. How frequently you update your site?

  24. explainer videos says:

    I keep listening to the reports lecture about receiving free online grant applications so I have been looking around for the finest site to get one. Could you advise me please, where could i acquire some?

  25. aceite para pestañas says:

    An attention-grabbing discussion is price comment. I think that it’s best to write more on this topic, it won’t be a taboo topic but generally people are not sufficient to talk on such topics. To the next. Cheers

  26. delivery churrasco says:

    Hi, Neat post. There is a problem with your website in internet explorer, could test thisK IE still is the market leader and a good component of other people will pass over your magnificent writing due to this problem.

  27. Joker388 says:

    I simply couldn’t leave your site before suggesting that I really loved the standard information a person provide for your visitors? Is going to be back regularly in order to inspect new posts

  28. aduslot says:

    I think this web site has got some rattling excellent info for everyone :D. “Time–our youth–it never really goes, does it It is all held in our minds.” by Helen Hoover Santmyer.

  29. Slot FafaSlot says:

    Have you ever thought about creating an e-book or guest authoring on other blogs? I have a blog centered on the same ideas you discuss and would really like to have you share some stories/information. I know my audience would appreciate your work. If you are even remotely interested, feel free to send me an e mail.

  30. Joker388 Online says:

    Wow! This can be one particular of the most useful blogs We’ve ever arrive across on this subject. Actually Fantastic. I am also a specialist in this topic therefore I can understand your hard work.

  31. Download Joker388 says:

    I know this if off topic but I’m looking into starting my own weblog and was curious what all is required to get set up? I’m assuming having a blog like yours would cost a pretty penny? I’m not very internet savvy so I’m not 100 positive. Any tips or advice would be greatly appreciated. Thanks

  32. VivoSlot Online says:

    When I originally commented I clicked the -Notify me when new comments are added- checkbox and now each time a comment is added I get four emails with the identical comment. Is there any manner you’ll be able to remove me from that service? Thanks!

  33. Joker388 Gamming says:

    Thank you, I have just been searching for information approximately this subject for ages and yours is the best I have came upon till now. However, what in regards to the conclusion? Are you sure in regards to the supply?

  34. Joker123 Gamming says:

    obviously like your web-site however you need to take a look at the spelling on several of your posts. A number of them are rife with spelling problems and I in finding it very bothersome to inform the truth however I will surely come back again.

  35. hospedagem de site barata says:

    Hey there! Do you know if they make any plugins to help with Search Engine Optimization? I’m trying to get my blog to rank for some targeted keywords but I’m not seeing very good results. If you know of any please share. Thank you!

  36. Mitologia says:

    I have been exploring for a little for any high-quality articles or weblog posts on this kind of space . Exploring in Yahoo I finally stumbled upon this website. Studying this information So i’m glad to show that I’ve an incredibly just right uncanny feeling I came upon just what I needed. I so much no doubt will make sure to don’t disregard this site and provides it a glance on a constant basis.

  37. Deuses says:

    Hi my family member! I wish to say that this article is amazing, nice written and include approximately all significant infos. I would like to peer more posts like this .

  38. libid gel como usar says:

    An impressive share, I just given this onto a colleague who was doing a little analysis on this. And he in fact bought me breakfast because I found it for him.. smile. So let me reword that: Thnx for the treat! But yeah Thnkx for spending the time to discuss this, I feel strongly about it and love reading more on this topic. If possible, as you become expertise, would you mind updating your blog with more details? It is highly helpful for me. Big thumb up for this blog post!


    We stumbled over here by a different website and thought I might check things out. I like what I see so now i am following you. Look forward to looking at your web page for a second time.

  40. FafaSlot Gamming says:

    I think other web site proprietors should take this website as an model, very clean and great user friendly style and design, as well as the content. You’re an expert in this topic!

  41. Joker123 Online says:

    Hiya! Quick question that’s entirely off topic. Do you know how to make your site mobile friendly? My website looks weird when browsing from my iphone4. I’m trying to find a theme or plugin that might be able to fix this issue. If you have any recommendations, please share. Thanks!

  42. Login Joker123 says:

    I will immediately grab your rss feed as I can not to find your email subscription hyperlink or e-newsletter service. Do you’ve any? Kindly permit me recognise in order that I may subscribe. Thanks.

  43. Situs Slot Online says:

    Whats Happening i’m new to this, I stumbled upon this I’ve found It absolutely useful and it has aided me out loads. I’m hoping to contribute & help different customers like its aided me. Good job.

  44. Joker123 says:

    What’s Happening i am new to this, I stumbled upon this I’ve found It positively useful and it has aided me out loads. I hope to contribute & aid other users like its aided me. Great job.

  45. Download VivoSlot says:

    I do not even understand how I stopped up here, however I believed this publish used to be great. I don’t realize who you might be however certainly you are going to a famous blogger when you aren’t already ;) Cheers!

  46. Daftar VivoSlot says:

    Great – I should definitely pronounce, impressed with your web site. I had no trouble navigating through all the tabs as well as related information ended up being truly simple to do to access. I recently found what I hoped for before you know it in the least. Reasonably unusual. Is likely to appreciate it for those who add forums or anything, web site theme . a tones way for your customer to communicate. Excellent task.

  47. FafaSlot Gamming says:

    Have you ever thought about creating an ebook or guest authoring on other websites? I have a blog centered on the same information you discuss and would love to have you share some stories/information. I know my readers would appreciate your work. If you’re even remotely interested, feel free to shoot me an e mail.

  48. Daftar FafaSlot says:

    Hello my loved one! I wish to say that this article is amazing, nice written and include almost all significant infos. I¦d like to see extra posts like this .

  49. papel parede 3d says:

    This site is known as a stroll-by means of for the entire info you needed about this and didn’t know who to ask. Glimpse here, and you’ll definitely discover it.

  50. Ruta en España says:

    Hi there I am so happy I found your blog page, I really found you by error, while I was researching on Askjeeve for something else, Nonetheless I am here now and would just like to say kudos for a incredible post and a all round exciting blog (I also love the theme/design), I don’t have time to go through it all at the minute but I have book-marked it and also included your RSS feeds, so when I have time I will be back to read more, Please do keep up the superb job.

  51. Game podcasts says:

    There are definitely numerous details like that to take into consideration. That may be a nice level to carry up. I offer the ideas above as common inspiration however clearly there are questions just like the one you deliver up the place the most important thing can be working in sincere good faith. I don?t know if greatest practices have emerged round things like that, however I am sure that your job is clearly identified as a fair game. Both girls and boys feel the impact of only a second’s pleasure, for the remainder of their lives.

  52. criminal lawyer says:

    After study a few of the blog posts on your website now, and I truly like your way of blogging. I bookmarked it to my bookmark website list and will be checking back soon. Pls check out my web site as well and let me know what you think.

  53. criminal lawyer cost says:

    This web site is really a walk-through for all of the info you wanted about this and didn’t know who to ask. Glimpse here, and you’ll definitely discover it.

  54. criminal lawyer advice says:

    Hey, you used to write magnificent, but the last several posts have been kinda boringK I miss your tremendous writings. Past several posts are just a bit out of track! come on!

  55. Industry Watch News says:

    I like what you guys are up too. Such smart work and reporting! Keep up the superb works guys I’ve incorporated you guys to my blogroll. I think it’ll improve the value of my site :)

  56. at this says:

    I really like your blog.. very nice colors & theme. Did you create this website yourself or did you hire someone to do it for you? Plz reply as I’m looking to create my own blog and would like to know where u got this from. thanks

  57. my site says:

    This blog is definitely rather handy since I’m at the moment creating an internet floral website – although I am only starting out therefore it’s really fairly small, nothing like this site. Can link to a few of the posts here as they are quite. Thanks much. Zoey Olsen

  58. greenbar ssl says:

    What i don’t realize is in reality how you are not really a lot more neatly-liked than you may be now. You’re so intelligent. You realize therefore considerably relating to this matter, produced me in my view believe it from so many numerous angles. Its like women and men are not involved unless it is something to accomplish with Woman gaga! Your personal stuffs nice. At all times take care of it up!

  59. Lynetta Campoy says:

    I simply want to say I am just very new to weblog and certainly loved this web blog. More than likely I’m likely to bookmark your blog . You certainly come with good well written articles. Thanks a bunch for revealing your web page.

  60. SMS says:

    Very good article! We are linking to this particularly great content on our site. Keep up the great writing.

Trackbacks & Pingbacks

  1. can you buy chloroquine over the counter

    A/B testing in ecommerce: 10 things you really need to test – aionhills.com

  2. buy ciprofloxacin

    A/B testing in ecommerce: 10 things you really need to test – aionhills.com

  3. doctor7online.com

    A/B testing in ecommerce: 10 things you really need to test – aionhills.com

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published.