Fake Door Testing: The Benefits, the Risks, and How to Build Functional Tests

Fake Door Testing: The Benefits, the Risks, and How to Build Functional Tests

Wondering how to create fake door tests that work? Come on in, we’re covering how to make effective tests that benefit both your users and your business.

Ray Slater Berry
12 min read
Fake Door Testing: The Benefits, the Risks, and How to Build Functional Tests

Business: Knock, knock!
Customer: Who’s there?
Business: No one, this is a test. But...

Fake door testing is a dangerous game. Done right, you can win future customers, validate an idea for a product launch, and even find beta testers for a new feature. Done wrong, you could damage your brand reputation and risk a PR nightmare. 

So, where’s the middle ground? How can you run fake door tests that help product-led business growth while building trust and buy-in from your customers? 

Come on in; we’ll show you how.

TL;DR
  • Fake door tests present an invitation for a user to try a product or a feature that isn’t actually there. It represents what’s yet to come, and triggers curiosity, a fear of missing out, or excitement to try something new.

  • A fake door test can be shown as a CTA button, in-app notification, pop-up, slide-out, banner, or even a product video or an ad.

  • The benefits of using fake door tests include gathering relevant user feedback, minimizing the risk of failure, and evaluating even the smallest product updates before you go into development.

  • However, this research method comes with risks, too. Fake doors can cause potential distress, trigger disappointment, diminish credibility, or break trust in your brand. 

  • How to avoid risks? Be honest and transparent in your messaging. Make sure to convey a clear message that will instantly reveal what the test is all about. 

  • To make an effective fake door test, choose the right tools, pick your audience, build a test and a follow-up journey, gather data, and measure success.

  • Translate quantitative and qualitative feedback into actionable insights that will help you build better products and increase user adoption.

What is fake door testing #

Fake door testing is a lean research technique product teams use to validate new products, features, or services. Fake door tests, sometimes also called painted-door tests, garner interest by presenting people with a call to action or a low-fidelity prototype of an idea and seeing how many people will click on it or try to use it. 

Fake door testing comes with pros and cons. It’s a product development method that, done right, can help you find product-market fit and lower adoption risks of new ideas. However, if it’s done wrong, it could break customer-brand trust. 

Let’s start with the positives.

How it works (and why it works so well) #

Fake door tests present the user with an invitation to use a product that isn’t actually there. This can be a CTA button, an in-app notification, a pop-up or a slide-out, or even a product video or an ad. When the user clicks or signs up for that fake product, they receive a message or land on a page that comes out with the truth the product doesn’t exist (yet). 

However, it’s something your business is either building or considering. The data collected from a fake door test—things like clickthrough rate (CTR) or page interactions—help a product team decide what could work and what won’t. 

A fake door test, done right, triggers curiosity or fear of missing out (FOMO), and it excites users to try something new—potentially being one of the first. 

Finally, a fake door test works well because not only does it help product teams evolve, but it could also entice users to stick around by showing promise of what’s to come. Or, in case you run fake door targeted ads, it can help you find new customers, build personas, and define demographics.

Benefits of fake door testing and how to maximize their value #

Despite the general fear around fake doors, there are some great benefits for businesses. Let’s explore what some of these fake doors can do for your team.

Gather feedback on how users respond to a new offering #

First and foremost, gathering feedback with low investment is a fantastic way to minimize product build risk and save on engineering resources. 

Depending on the tools you use and the fake door test you choose to run, you have the opportunity to collect both qualitative and quantitative feedback that can radically improve the build and pitch of a new idea—and support a product launch.

Get user feedback with Chameleon

Run in-product surveys to collect contextual user feedback throughout the development process – from ideas to validation to deployment.

Evaluate even the smallest in-product new features #

You can also use fake door tests to understand how current users feel about a new feature. 

Whether your fake door test comes within an interactive product tour for new users, or you present it as a modal pop-up for existing users, you can gauge interest in the minutest of features and get people excited for what’s to come. 

How does this work? Take a look at a hypothetical test that Curtis Stanier, Product Lead at Delivery Hero, created to showcase the example he lays out in his Medium article on painted-door testing.

(Source)

Develop features your users want and disregard the ones they don’t #

Fake door tests are a fantastic tool for validating new features and winning initial buy-in from existing users. Of course, for this user research tool to work, you need great user engagement in your platform, and people need to be accustomed to providing user feedback. 

Fake doors are also great for disregarding new features you’re considering. For example, if you’re using an MVP approach to your product, you may receive customer feedback that states the importance of a feature for a particular customer. 

However, when you run a fake door test to measure the popularity of this feature idea, it could reveal that the lift is not worth the investment. 

Lastly, fake door tests are a great way to prioritize your product backlog. If you’re unsure of what to work on first, ask your customers via fake door tests and let them answer.

Segment users for beta testing and feature up-selling #

Fake door tests can be the perfect tool to help you segment your user base for beta testing. 

For example, perhaps you’re building a feature that a certain user really wants. Rather than offering the feature to all users, offer it to those that care about seeing it succeed.

Offering beta testing opportunities is a great way to build user engagement and gather qualitative user feedback. This also ups your feature adoption rates when you do bring it to market. 

Once your beta testing is over and you’re ready to release your new feature MVP, you already have an invested group of (ex-beta) users that could be willing to pay for this—just make sure you offer it at an exclusive rate. Say thanks.

Discover new customer cohorts #

Fake door tests can also be beneficial if run out of product and via targeted ads. They can help you find product-market fit early on and identify entirely new groups of customers for a product launch. 

For example, if you have an idea about your target audience, you can run social media ads or Google PPC campaigns that entice someone to a landing page and low-fi design of the product, feature, or service to come. In doing so, you can start building lookalike audiences for future advertising. 

Plus, if you’re collecting emails in your fake door tests, then you can build a list of people that want to know more and will potentially convert to a sale when your idea is ready. 

Well, we’ve covered the benefits. However, that’s not to say fake doors come without risks.

Risks of fake door testing and how to avoid them #

For all of the benefits available to a business running a fake door test, there are risks to running them as well. But these risks can be avoided, or at the least, minimized.

Fake doors can cause potential distress #

It takes time to build relationships with customers. A fake door test is essentially an upfront fib about what the customer is about to get. This causes distress and potentially diminishes a customer’s trust in a business. 

However, there are ways for you to remedy the fib immediately. On the click-through landing page, be transparent and honest with your copy. Let the visitor know that you’re considering working on this feature and thank them for their time. 

As soon as you validate a customer’s feelings, you can remedy the relationship and even get them excited about what’s to come. But, you need to act fast with your copy.

Fake door tests can trigger disappointment #

Of course, there’s a certain amount of disappointment someone will experience if the solution they thought they were getting is not available to them. 

You can avoid this by empathetically handling the user’s disappointment. Explain the reason behind your actions, and ask users if they would like to leave feedback on their expectations with a product survey or participate in a beta version of the product for free. 

Note: Integrate user feedback tools wisely, and be cautious of asking too much from someone. 

Lastly, build a sense of community around your new product and let your users know how heavily they can positively influence what’s to come. Assure them you’re essentially building something to answer their needs. Remain customer-centric.

Fake doors might diminish credibility #

If you’re a relatively new SaaS or are targeting a new audience for your fake door test, then credibility is a big thing. Ever heard the expression: starting off on the wrong foot? The same applies to your fake door. 

How do you remain credible with fake door testing? There are a few ways. 

One is with the messaging on your landing page and the things we’ve mentioned above: beta testing, feedback opportunities, sharing insights on what’s to come.

Two, if you choose to target a new audience, keep your testing cohort and spend small. You may damage a few relationships with potential customers. However, if you gain valuable insights, then you’re more likely to present a better-fitting product to the masses. 

Risks covered. Let’s move into how to make fake door tests.

How to make an effective fake door test #

Right, now that you know the benefits and risks of fake door testing, let’s get into how to make fake doors. Just saying that seems ominous, but here we are. There are plenty of product marketing tools to help you get the job done; a few fan favorites are: 

  • Mailchimp: For drag-and-drop landing pages, email collection & journeys

  • InVision: For simple and effective product mockups or wireframes

  • Semrush: For refining any PPC campaign for your fake door test

  • Chameleon: For creating in-app fake door testing pop-ups 

Now you’ve got your tools, let’s get into how to make fake doors that benefit your business and customers.

1. Pick your audience #

First and foremost, who are you creating this test for?

Will you be targeting potential customers via advertising? It’s a great option if you want to test a new business model or product. Or, will you be targeting users that are already navigating your product? This is a good option for new features and if you have an engaged user base. 

Once you decide who the test is for, you can build with greater awareness.

2. Create the fake door and what’s inside #

Next, you’re ready to build your fake door and what lurks behind it. Remember to keep things simple, and transparent once someone is inside. 

The fake door itself needs to be equal parts enticing as it is informative. Breadcrumb your users with just enough information to win a relevant click-through. 

For example, that’s exactly what Buffer did for their new Start Page feature. By placing a CTA in the notification bar on their website Home page, they encourage visitors to sign up for early access. When someone clicks on the CTA, they land on a page where they can leave their email address to join a waitlist.

(Source

As for the landing page, keep things low-fi. One of the benefits of fake door testing is to save on design resources and engineering time, so keep it that way. 

It’s also critical to be transparent and use empathetic copy here, so spend time with your copywriter to ensure the wording is just right and you remain serving a delightful experience for users.

Things to include in your test content: 

  • The purpose of the new product 

  • The key benefits of the new product

  • A potential roadmap

  • How the user can help 

  • An option to sign-up for more 

  • Visual aids: screenshots, wireframes, or a product video

Here's another real-life example. Let’s take a look at how Dropbox did it back in 2019 when they announced the new desktop app. On the announcement page, they included the opt-in option for the existing users, to get early access and take the new app for a spin. Plus, they added a dedicated button for collecting feedback.

“Ready to try out the new Dropbox experience for yourself? All Dropbox users can opt in to the new desktop app through our early access program, and we’re rolling out the other features over the next few weeks. We can’t wait to hear how you’re using the new Dropbox, what you like, and what you want more of—look for the “Send feedback” button in the lower-right corner of the new desktop app.”

All that was accompanied by an engaging new Dropbox announcement video.

3. Work on your follow-up journey #

Remain accountable for whatever you promise your page visitors. 

  • If you promise more information as you build, then deliver that. 

  • If you promise the option to be a beta tester, then invite them. 

  • If you say that the product should be ready by a certain date, then get that MVP in their hands or let them know why it’s not ready. 

Communication is critical for maintaining the relationship you’re building with the fake door test. 

Note: You may also have some stellar product-qualified leads for future marketing, so make sure you’ve got an email campaign in place for when the product is available. 

Gather data and measure success #

It’s important to set some KPIs around what would be the success of your fake door test.

Are you looking for a certain click-through rate to give your team the green light and start the build? Perhaps you need a certain number of positive feedback responses to let your PM team know an idea is valid. 

A few goals and data sets to measure are: 

  • Conversion rate

  • Click-through rate

  • Email signups 

  • Bounce rates 

  • Positive user feedback 

  • Negative user feedback 

The quantitative data and qualitative responses you receive from your fake doors will be crucial to informing your new product or feature build.

Note: Whether this is an in-product test or something you’re running externally, make a decision on whether it’s worth it as a validation method. If the negative feedback outweighs the positive, you’ll need to either reassess your fake door test or consider scrapping it from your lean development process altogether. 

You've created a fake door test. What's next? #

Fantastic, so your fake door test is up and running, and you’re ready to start taking action. It’s time to zoom out and look at the bigger picture. It can be overwhelming getting this test right. Remember why you ran this test in the first place.

Your fake door test is a crucial part of your product development process. It helps your researchers to gather data and developers to test, iterate, pivot, and evaluate new ideas. 

Document your learnings from the fake door test and share it with your team. Who needs to know what? And why? How can you translate quantitative and qualitative data into actionable insights that will help you build better products and increase user adoption?

Closing the door on fake door tests #

That’s a wrap on fake door testing. Despite the stigma around this type of test, if done right, it can be extremely beneficial to your business and your customers. 

Remember, once a user is ‘through the door,’ they’ve already provided you with valuable insight on your hypothesis. Make sure you provide value back and continue the conversation around the topic. 

Thanks for your visit—we hope to see you again soon!

Create fake door tests, no engineering needed

With Chameleon, you can launch experiments fast and iterate quickly

Boost product adoption and
reduce churn

Get started free in our sandbox or book a personalized call with our product experts