🥚 First Time User Integration Recommendations
I designed a full-screen experience in the onboarding flow to recommend relevant integrations to new users based on their business type and URL, significantly increasing integration connection rates and improving customer payoff.
📅 May 2025
🧠 Product Design / User Research / Prototyping
🛠️ Figma
Mailchimp recognized that customers who integrate their applications achieve higher payoff rates and improved retention. Building on the success of a June 2024 experiment that significantly increased e-commerce integration connections (over 40% lift), the team aimed to expand this success. The goal was to increase the integration connection rate for e-commerce customers and extend this benefit to other customer segments by surfacing relevant integrations identified through advanced data analysis.
🤔 Background
🤔 The Challenge: A Smoother Transition for Mailchimp Switchers
The existing system for recommending integrations primarily focused on interdicting self-reported e-commerce customers after account setup to connect their stores. While effective for this segment, there was a missed opportunity to provide relevant integration recommendations to non-e-commerce customers and those who skipped the e-commerce question during account setup. This limited the overall integration connection rate and potentially hindered customer payoff and retention for a broader user base.
💡 The Solution: A Personalized Onboarding Experience
The proposed solution involved implementing a new "First-Time User (FTU) Recommendations for Integrations" experience. This would leverage an A2D model, informed by web scraping and third-party data, to surface a ranked list of relevant integration options for non-e-commerce users with a provided URL. The goal was to present these recommendations as a full-screen experience post-account setup, with a fallback to display them as a task on the user's homepage if not completed immediately.
🔎 Existing Experience Analysis
The team identified the opportunity to extend successful e-commerce integration strategies to a wider audience, recognizing the correlation between integrations, payoff, and retention.
📊 Opportunity
Here is our SWAG sizing based on product input
🚧 Scope and Limitations
I had to keep in mind the constraints and technical feasibilities of creating this experience. The success of this experiment was heavily dependent on the Model Integration and its performance. The experiment's effectiveness is tied to the accuracy, coverage, latency, and update cadence of the A2D model, and any shortcomings in the model could limit the quality or relevance of recommendations.
User research from our e-commerce integration experiment revealed a key insight: customers respond more positively and are more likely to integrate their business platform when presented with direct, clear language regarding integration options. This finding underscores the importance of concise and actionable messaging in guiding users through critical onboarding steps, ultimately driving higher conversion rates for integrations.
🤓 User Research
Winning variant
🤝 Collaborations
The biggest challenge for this project was collaborating with many different teams and leaders. Due to the opportunity sizing of this project, we did not have time to user test. We spent the majority of our time leaning our ML and Data Science teams and informing the model they use to surface relevant integrations. We learned that for the previous e-commerce integration experiment that the integrations that surfaced were not personalized and that users who selected e-commerce as how they sell to customers saw a static list of integrations based on popularity. It was a top priority to make sure that we can use the A2D model to provide real-time integration recommendations during Account Setup. Key aspects include the model outputting a ranked list of the top 8 integrations, meeting performance (accuracy, coverage), latency, and update cadence requirements, and allowing for thresholding of recommendations based on offline analysis. This integration is crucial for surfacing relevant integration options to new users. Product and Design worked closely with the ML and Data Science teams to understand how many integrations we would show users based on confidence, and what we would show for low-confidence predictions.
Logic
🎯 Finalizing designs
Based on what we learned from the e-commerce experiment, we had high confidence that we could use the same insights in changing the content for the takeover screen. We chose to use direct language as well as generalizing integrations as “tools” instead of “business platforms” to be inclusive to different non-ecommerce integrations. We also didn’t want to change too many things in the design because we would not be able to pinpoint later which change pushed the needle.
Changes made
🔔 Where we are today
We ran this experiment in June 2025. We saw that users in the variant connected integrations at a higher rate, with a 677 bps lift in 14D integration connection rate, driven by both non-ecommerce (604 bps) and e-commerce (73 bps) integrations. We also saw that about one-third (31.2%) of users who viewed the recommendations clicked to connect, and about 15% completed a connection, outperforming previous e-commerce integration takeover experiences. In-app conversion and payoff rates remained flat.The hope is to see a positive lift in these metrics. This experiment is just the first step in a larger plan to personalize the entire Mailchimp onboarding experience and create a truly seamless journey for all new customers.
Because we have seen that interjection designs increase conversions in connecting tools, we have enough data to be more bold in the future. I am currently working with our Homepage team to expand the interjection screen to appear on subsequent logins to Mailchimp. Leadership also wants to expand the integrations experiment and take its learnings and apply them to other parts of our product.