# When product-led growth drives Generative AI success in healthcare _Date: 13-02-2024 , Tags:_ #AI/genAI, #digital-health , #nabla, #copilot #clinical-notes, #interview ![[adryhealtharticles.png|200]] **Interview with Chloé Simpson - Head of Business & Strategy @ Nabla** ## Introduction Physicians typically dedicate approximately 40% of their working hours to documenting patient information, leading to significant burnout. Nabla is a leading Generative AI startup that addresses this issue by automatically generating clinical notes during patient visits.  Nabla is also a rare example of Product-Led Growth in the healthcare industry.  Today, I had the opportunity to interview Chloé Simpson, the Head of Business & Strategy at Nabla. Chloé, a data scientist turned strategist, shared insights about her professional journey and best practices in the field of product data management. ![[When product-led growth drives Generative AI success in healthcare.png]] ## Hello Chloé, thanks for joining the Digital Medicine Beats Newsletter. You have quite a unique experience starting as a data scientist and moving towards strategy. Could you tell me more about your professional path? _Hello, I'm Chloé Simpson, currently the Head of Business and Strategy at Nabla. My role revolves around understanding the needs of our customers, who are primarily large health systems in the US. I work closely with them to identify the best ways to implement our AI solutions with their clinicians._ _I'm an engineer by background, I hold a Master's of data science from Stanford University, where I discovered and applied data science and AI to healthcare, by working on healthcare data from Stanford Hospital. This is really how I began my career in healthcare.  I then joined Owkin, a startup that is applying machine learning and technologies from medical research to drug discovery, and later collaborated with the French Red Cross. Finally, I joined Nabla to continue working on healthcare, this time on the strategic side._ _I transitioned from a highly technical data science role to gradually taking on less technical responsibilities. I found myself increasingly drawn to working on the strategic aspects of the product development lifecycle, as I gained insights from these diverse experiences._ _I would say having a T-shaped background is a superpower for an engineer._ _My background allows me to understand the intricacies and potential of AI technology, and I can communicate its value to our customers in a language they understand. This helps us ensure that our solutions align with their goals and requirements._ ## How can evidence and trust be generated in new AI solutions, which are crucial for innovative digital health products? _The whole point of generating evidence is to ensure the accuracy of an AI model. With Nabla Copilot, we automatically generate the transcript and note of the clinical encounter. One of our main focuses is ensuring the accuracy of the transcript since that's where most of the evidence lies. We currently use a mix of open-source models like Whisper or GPT-4 by OpenA, and our own algorithms, fine-tuned to the field of medicine, to achieve the best accuracy. We have developed our own evaluation framework for three reasons: First, to make sure there are no hallucinations in the note. Second, to test different models, and third to provide the most sophisticated product (more info on [Nabla.com](https://www.nabla.com/blog/evaluating-note-generation/))._ _Overall, accuracy is paramount to AI applications in healthcare. We are mindful that clinicians have limited attention spans, so our product needs to be reliable and deliver on our promise to save them time._  _Additionally, generating robust evidence to showcase the impact of the solution. We recently published a manuscript on the impact of Nabla on the clinician workflow (Tierney et al., 2024)._ ## Do you have any best practices for collaboration between the data and product teams? Is there something that can be scaled across other organizations? _Well, I think our team is focused on collaboration. What's even more important is that we include clinical expertise in the process of building our product._ _One crucial aspect is ensuring that data scientists have regular exposure to clinicians. To achieve this, we have implemented two best practices:_ - _External Validation: we allow clinicians to send feedback on Nabla Copilot, which we use to improve our models. Understanding the feedback from clinicians is vital in building the best product. Over the past six months, we have received over 6000 feedback submissions from clinicians, which is a significant number._ - _Internal Validation: the second point is working with clinical advisors. Our Chief Medical Officer and Clinical Director are crucial in shaping Nabla Copilot. Recently, we added coding automation, to streamline the assignment of codes and the billing process., after the visit had taken place. Our machine learning and product team have regular meetings and touchpoints with our executive clinical team to better understand how to implement different features. This way, we make sure we do not waste time going in the wrong direction. Having this clinical exposure is important and has proven to be valuable in our collaboration efforts._ ## What strategies do you employ to enhance the adoption and usage of your product in the healthcare industry? _Our product-led growth methodology is an aspect worth mentioning, especially in the healthcare industry. It allows users to immediately access our product and start getting value from utilizing our Nabla Copilot. Although this approach is not commonly seen in healthcare, it has been instrumental to Nabla in building a large clinician community._ _When it comes to selling to health organizations, such as health systems, the process is different, as it usually involves piloting the product first with a small group of physicians. During this pilot, the adoption of our product is tested. To increase the chances of clinicians embracing our product, we work on building the features that suit their needs and clinical workflow._ _To provide examples, we place great emphasis on developing features that allow clinicians to personalize copilot. They can customize instructions in their templates and adapt them based on their preferences. Our goal is to provide the most personalized experience possible._ _By focusing on boosting product-led growth, we simultaneously increase the chances of adoption during the pilot phase in health systems. This is because early users who discover the product on their own become our best ambassadors and push for quick organization-wide deployment. This alignment of strategies is crucial._ _In the process, we focus on consultations per month or week as the main metric. This allows us to track the way our newly developed features boost the adoption and usage of our technology._ ## How do you track and analyze user analytics and generate evidence? Are there any specific tools that you use? _We have been using Mixpanel to track and analyze user metrics._  _Every Monday, during our product team meetings, we review the metrics on Mixpanel to assess our progress and determine our priorities for the week._  _Additionally, we have executive meetings where we also analyze the metrics to gain insights into our overall strategy and project performance._  _On a monthly basis, we hold all-hands recap sessions that include a component of metrics analysis._  _So, while we check the data every day on Mixpanel, we have more comprehensive reviews on a weekly and monthly basis._ ## Is there any point that you think would be interesting for data scientists, product data managers, or strategic leaders to know? _I've always been fascinated by the intersection of healthcare and AI. Reading about advancements in healthcare and how AI can make a positive impact has shaped my passion. So, I would suggest diving into the world of healthcare and AI to discover the exciting possibilities it holds._ _I believe it is crucial to validate your work in clinical settings and to push for real-world user feedback._  _It's not just about publishing articles or case studies in prestigious medical journals. It's about deploying your solutions in healthcare organizations and gathering metrics on their actual usage. Seeing the impact of your work in practice is incredibly rewarding and helps align everyone within the company._  _The data science team wants to achieve the best performance, the product team wants to see positive metrics, and the strategy team wants to leverage this data to inform their decision-making._  _So, my advice would be to prioritize obtaining these metrics as soon as possible. It's a cool and valuable thing to do._ ## What counterintuitive advice would you give to yourself, looking at your journey five years ago?  _I think the counterintuitive advice I would give to myself five years ago is not to be afraid of switching careers and jobs._  _Initially, I believed I needed to start in one direction and become an expert in that field. However, over the years, I've realized the value of having an overview of different aspects and job opportunities within a company._   _By exploring different roles, I have gained valuable insights and experiences. So, my advice would be to embrace the idea of switching careers and exploring different job opportunities._ ## Before we wrap up, is there anything specific you would like to ask the community? _Yes, we are currently looking to hire talented individuals in sales, and tech roles. Additionally, we are actively seeking more clinicians and health systems to utilize Nabla._  ## Key Takeaways - **On Product Metrics (the sooner the better)**: implementing a way to measure analytics is fundamental, it ensures strategic alignment and informs the product strategy - **On the Collaboration between the Data and Product team:** data serves as the connecting thread within the organization in two ways: - everyone can access up-to-date product metrics through a dedicated dashboard; - there are regular meetings (on a weekly and monthly basis), at various levels of the organization to discuss the key metrics. - **On the Collaboration between Clinicians and the Product/Data Teams**: including clinical expertise in the process of building healthcare products is crucial. Clinicians look for custom solutions that adapt to their workflow not vice versa. Nabla implements external validation (via user-reported feedback) and internal validation (via Clinical advisors) practices to ensure feedback from clinicians is incorporated into their product development. Data scientists have regular touchpoints with the clinical team. - **On Product-led growth**: PLG in healthcare (when applicable) has the potential to foster a sense of community and trust surrounding a new solution, thereby eventually driving its adoption. Additionally, it serves as a crucial means of gathering real-world user feedback, which can be utilized to enhance the AI model and further advance commercial opportunities in the future. - **On Career advice**: data scientists should not be afraid to switch careers and explore different departments. A T-shaped profile is a superpower! ## References: - Tierney et al., (2024): “Ambient Artificial Intelligence Scribes to Alleviate the Burden of Clinical Documentation.” URL: [https://catalyst.nejm.org/doi/full/10.1056/CAT.23.0404?query=CON&cid=DM2321351](https://catalyst.nejm.org/doi/full/10.1056/CAT.23.0404?query=CON&cid=DM2321351) --- _This article is a repost of the original Digital Medicine Beats Newsletter. Subscribe to Digital Medicine Bests to receive new content._ <iframe src="https://digitalmedicinebeats.substack.com/embed" style="min-width:100%;height:300px"></iframe> --- _Copyright © 2024 - Adriano Fontanari_