🤔 Challenge: How can a project management tool increase its speed of experimentation without relying on engineering help to improve activation and in-app engagement?
đź’ˇ Solution: Creating in-app Microsurveys, Tours, and Embeddables to gather feedback, validate hypotheses, and guide users
🏆 Outcome: An 8% increase in activation rates and 3X the speed of experimentation, increased feature discovery and engagement
Shortcut is a project management platform that combines tools for issue tracking, sprint planning, roadmaps, and goals in one interface. Shortcut helps teams plan, collaborate, and track progress efficiently and supports integrations with many popular tools.
What brought Shortcut to Chameleon?
A few years back, as the company grew, the Product team had big ideas, strong hypotheses, and a ton of energy to match. But one thing was slowing them down: being able to independently test those growth initiatives.Â
The team had a solid tech stack and used Amplitude for product analytics, Looker as their Business Intelligence (BI) tool, and Pendo for in-app messaging and further product analytics. While Shortcut’s analytics stack continued to support them as they matured and grew, one piece of the puzzle was missing. With all this product data, they had built up a backlog of experiments to run, but they could not independently test them without engineering help.Â
Shortcut was on the lookout for a dedicated Product Adoption Solution that would:Â
complement their existing stack and help them identify users’ challengesÂ
enable them to test in-app hypotheses quickly to increase adoption
unblock them in engaging new signups to drive activation
help them efficiently communicate brand and product changes to users
effectively drive feature discovery and engagement
Chameleon fit the bill with its feature offering of native A/B testing, advanced element selection, and two-way integration with Amplitude. Shortcut chose to keep Pendo for product analytics but transitioned to Chameleon for in-product communication.
Chameleon enabled us to move quickly. Being able to go in and easily create new experiences in the product was a big deal for us. We have gone from one experiment a month to three or more!
Boosting activation with Microsurveys and A/B Testing
After implementing Chameleon, the team dug into their experiment backlog right away. One of the first areas they examined was their signup flow. Something strange was happening with their signup conversion rates; half of their users were getting stuck at the first step, filling in their company name, and the team didn’t know why.
Step 1 - Finding out where users struggle to increase signup conversionÂ
Shortcut created a dropdown Microsurvey with Chameleon and targeted users who were getting stuck to help answer the question. The team used a “Timed Delay” to trigger the Microsurvey and “Element Rules” to ensure it appeared only to users who could not move past a specific point in the signup process. Those who would quickly find their way to the Shortcut app would not see this Microsurvey, so it wouldn’t intrude on the overall user experience.Â
This is why we love Chameleon – we can iterate quickly and validate an experience before building it in-house.
👇 Explore an interactive demo and learn how to create a Microsurvey with Chameleon.Â
When they analyzed the Microsurvey responses, the team discovered many users who were already part of a larger organization or even using Shortcut for a side project. Asking for their company name was adding friction to the account creation process as users were looking to join colleagues or simply didn’t have one. Knowing this, Shortcut ran an A/B test to change “Company Name” to “Workspace.” The result?
We increased the account created conversion rate by 8%, which is huge as that’s the very first step users take with Shortcut – all thanks to the data we were able to leverage with Chameleon.
đź“š Read more about this experiment on the Shortcut blog đź“š
Step 2 - Reducing dropoff rates with in-app educationÂ
Most Shortcut customers are switching from one product to another. So, the biggest hurdle for them is the fear of changing tools. But switching doesn’t have to be hard! Shortcut removed the common aches and pains of tool switching by creating an “import data” functionality.
However, many users still weren’t using it, and those users were bouncing back out of the product, never to return. So the team devised a hypothesis that they could easily test with Chameleon’s in-app patterns.
If we can reduce that mental hurdle of switching tools and show users that importing is easy, we can reduce dropoff rates.
Using Tours, the team built a Modal that they showed to new users. With a short video embedded within the Modal, the team could demonstrate how easy it is to import data from another tool into Shortcut.
Next, they launched an A/B test with Chameleon to measure the impact of users seeing this Modal. Using Chameleon’s Amplitude integration, the team can analyze how their in-app patterns affect users throughout their activation journey.Â
We do everything we can to both activate users in their early moments and guide them through the process. That's where Chameleon comes in, we use it wherever we can to leverage the in-product experience
👇 Explore an interactive demo of Chameleon’s A/B Testing.
Increasing new user engagement with Embeddable Cards
Sara Lohr, Senior Product Designer, is leading the team’s efforts to improve Shortcut’s in-app engagement and user experience. She is working to reduce the time to value for new users without distracting them from their goals and improve feature engagement.Â
To start, Sara created a series of customized Launchers and placed each on the respective feature page in the Shortcut app. Sara designed them to be “self-serve Tooltips” that explain the value of each feature and enable users to learn more. If users have questions about the feature, they can launch Shortcut’s Command Bar, which opens the respective help guide. Sara could easily set this for each specific feature using Chameleon’s “Run custom code” option.Â
However, she found that users didn’t engage with the Launchers as expected and continued iterating to improve the user experience. Sara found that Embeddable Cards were more effective in driving new users to explore Shortcut’s features. She placed a full-width Card on their “Roadmap” feature page targeted to new users. Users could watch an overview video and see more examples to get inspired.
The first sign was positive, users engaged with the Card more than with the Launchers. When looking at users’ tendency to engage with their feature, Sara noticed that users who see and interact with the Card also have a much higher tendency to then go and interact with the “Roadmap”.
Users were not getting to the point of opening the Launcher. But with the Embeddable, it’s all just there and doesn’t require users to engage to see what the options are, they can see that from the start and act when ready.
👇 Explore an interactive demo of creating Embeddable Cards with Chameleon.
Sara is considering the positive correlation of curious users who are driven to engage with in-app announcements and features. However, this is a great indication of what Shortcut users prefer for in-app education and finding the most effective patterns to create a pleasant experience for new users. Next, Sara is eager to further experiment with Cards on other feature pages to drive discovery and engagement.Â
Cards are a nice tool for engaging users in-app. I'm seeing more engagement overall with people clicking buttons and taking actions on Embeddables than I am with the Launchers, so I'd like to explore this more.
Sara is also iterating to improve their user onboarding. Shortcut built a native onboarding experience that walks users through the steps of getting started and makes accessing various features from one place easier. The team also created a Chameleon Walkthrough that users can launch from their quick start to get an overview of the features they can find in Shortcut.Â
The Walkthrough is seven Steps long and highlights Shortcut’s features one by one, giving a brief description of each. The team uses a “Progress indicator” to set expectations for the time required and the CTA buttons to allow users to browse the different features. Users can also exit the Product Tour anytime and dive into a specific feature. Although it’s a long Tour, users engage with it and most complete it to the last step. So far, it has a steady 68% completion rate and gave Sara another example of what engaging in-app experiences look like for Shortcut users.Â
👇 Explore an interactive demo of creating a Product Tour with Chameleon.Â
Now, when users engage with a step in the quick start, for example, to learn about Shortcut’s “Stories” feature, they are prompted with a modal to create a new “Story” and use the feature immediately. But, Sara wants to help users become accustomed to where each feature lives and make it easier for them to know how to leverage Shortcut’s capabilities.Â
I am building Tours as an intermediary step to help locate people inside the app. This way, they get situated more easily, they can see where the feature lives, and then they can either decide to take the action right away or dive in deeper on the feature page.
She created Product Tours that users can launch from their native quick start. The improvement? Each Tour will first point to that specific feature in the navigation menu and offer an introduction highlighting user benefits. Users can navigate to that feature page and learn more or create a new “Story”.Â
Before, it didn’t seem like users got a chance to learn much, but this way, they have the opportunity to learn more easily. I'm interested to see if this extra friction of making users learn about the feature first reduces the completion rate or if it actually increases it.
Tours enable efficient communication for major brand news and product changes
Shortcut hasn’t always been “Shortcut”. Pre-2021, they were known as Clubhouse, which caused a surge in new users looking to join a live roundtable, not manage their projects!
With a change as monumental as a new company name, the team wanted their active users to know what was happening. Email works, but people can unsubscribe or skim over it so easily. With an in-product announcement targeted to all users, the team could confidently know that anyone using the product would know about the changes and impact.
If it's something that I need to ensure that customers read and understand, having it in the product feels like a better guarantee than only having an email for that.
Using a Chameleon Modal targeted to all users, the team could announce to their users that Clubhouse was now Shortcut – reducing confusion and offering further reading on the name change by directing users to their blog announcement.
We would have been in a far worse position if we didn’t have a tool like Chameleon to communicate changes to the right people, at the right moments.
👇 Explore an interactive demo of creating a Modal with Chameleon.
While a name change is a once-in-a-company-lifetime event, the team continued to use Chameleon Tours for feature announcements, marketing events, or guiding users through Beta programs. Over time, the team experimented with many patterns, from Slideouts for “Release Notes” to Lightboxes to engage users in their app. Today, the team continues to use a combination of patterns to engage users with company news or announce product changes.Â
The team is careful to ensure a delightful user experience without interruptions, where users can always self-serve Tours or help resources if needed. The team uses Chameleon along with their native elements to engage users with different in-app patterns. Â
We’ll keep using Chameleon, wherever we can, to leverage the in-product experience.
Shortcut uses Chameleon’s Segmentation to target the right audience at the right time and enriches the user data in their account through the Segment integration, with lists sent from HubSpot or quick CSV imports. Long-term users see trigger-based product announcements, and new users see onboarding tips. To avoid overwhelming their audience with in-app messages, the team uses Rate Limiting to limit the number of Tours existing users should see daily.Â
👇 Explore an interactive demo of creating Chameleon’s Segmentation.Â
What’s next for Shortcut?
Chameleon became a key component of quick experimentation and efficient in-app engagement for Shortcut. The team is dedicated to improving in-app engagement and driving better product adoption.Â
Using Chameleon’s Amplitude integration, they continue to monitor in-app engagement and product usage to ensure users get the most value out of each engagement. Their quick start experiment aims to ease users into the app without feeling overwhelmed, creating a smooth and engaging entry point for beginners. And the team plans to continue finding better ways to engage users in their product.Â
We really try to stay out of users' way as much as possible. We try to inform users and then allow them a way back to that content and I still experiment a lot with like the best way to present things to users.