- Healthcare
- Recommendation System
- Personalization
From Dream to Stream: Scaling ML Engineering at Flo Health
About Flo Health:
Flo Health is the most popular women’s health app globally. It is the #1 recommended app for period and cycle tracking based on a survey among 500 female U.S. obstetrician-gynecologists (ob/gyns). Over 300 million people have downloaded Flo Health, and 56 million use it monthly. With over 100+ medical experts working together to support Flo Health, the app assists women during their entire reproductive lives. It provides curated cycle and ovulation tracking, personalized health insights, expert tips, and a private community for women to share their questions and concerns. Flo Health prioritizes safety and focuses on being the most trusted digital source for women’s health information. The Flo Health app is available in more than 20 languages on iOS and Android.
Challenge:
Flo Health’s ML Growth Faced Operational & Governance Challenges
Flo Health’s ML team faced operational challenges, spending most of their time on project management rather than ML delivery. The lack of a centralized platform led to prolonged model updates, deployment delays, and governance issues.
Solution:
Toward a Unified ML Strategy: Flo Health’s Platformization Approach
To streamline its ML operations, Flo Health adopted Tecton as its central feature platform after thorough evaluations. This choice aimed to unify ML processes, enhance governance, and ensure data quality and traceability.
Results:
Tecton’s Feature Platform, a Crucial Part of Flo Health’s Technical Infrastructure
Flo Health’s feature store, bolstered by Tecton, now contains over 1600 features essential for their ML projects. By incorporating Tecton, Flo Health has launched significant ML-driven applications tailored for platforms like iOS, Android, and the web, including the Period and Ovulation Cycle Prediction Model and Mode Switch Prediction Model, and have been able to:
- Build and use the same pipelines for training/inference of their models
- Leverage built-in materializations for the online store
- Generate point-in-time correct joins for dataset collection from offline storage
- Share features across teams and projects easily
Ready to get started?
Flo Health is a pioneering organization that aims to provide an easy way for women to manage their menstrual health and views machine learning (ML) as an engineering discipline. Their teams, composed of engineers focused on ML, data, and ML operations, work on end-to-end models and oversee everything from research to execution and maintenance.
Currently, Flo Health uses Tecton’s feature platform to support specific predictive ML applications: the Period and Ovulation Cycle Prediction Model, Mode Switch Prediction Model, and many others, each designed for different platforms like iOS, Android, and the web.
Flo Health’s Organizational Structure & Approach
At Flo Health, there’s a strong emphasis on end-to-end responsibility within the data science (DS) and ML teams. The Head of Data Science leads these efforts, with engineers reporting to the CTO and collaborating on areas like Value Creation and Value Capturing streams.
The Value Creation stream emphasizes product engineering to deliver value to users. Key features include a period and symptoms tracker, a virtual assistant, a medical content library, and a community.
The Value Capturing stream is centered on product growth, optimizing user onboarding, pricing, and engagement experiences. The MarTech team, a specialized DS group, oversees the ML platform stack, including Tecton, and focuses on integrating value into product and marketing.
A Health Data Science team also uses the same ML platform to craft specific use cases. This organizational setup fosters cross-functional collaboration, enhancing the quality of the models produced.
Toward a Unified ML Strategy: Flo’s Platformization Approach
As a quickly growing company, Flo Health’s ML team faced significant operational challenges. Before implementing Tecton’s feature platform, around 90% of their time and resources were spent managing ML projects rather than delivering actual ML systems. Updating models could take three to six months, leading to inefficiencies and deployment delays. Consequently, they had a disjointed ML approach with systems spread out throughout the company, underscoring the urgent need for a centralized and standardized system.
Like many enterprises dealing with expanding user data and intricate ML production environments, the Flo Health team had to consider how best to apply governance, including access controls, feature reuse, and team collaboration. A central platform would allow for cohesive teamwork and efficient feature management, providing clarity on feature lineage, data quality, and the usage of certain features. Determining the return on investment (ROI) for their features became a daunting task, highlighting a need for a system to oversee, centralize, and manage their most impactful features.
To enhance and consolidate its ML operations, Flo Health adopted a platformization approach. Data for ML is often subject to the most stringent legal, regulatory, privacy, compliance, and security requirements, and operational ML systems typically power production features and business decisions representing huge amounts of value.
With this in mind, they determined the best platform for their needs through detailed comparisons using an architecture decision record. Reassured by Tecton’s thorough enterprise security and compliance standards and enterprise platform functionalities, they chose Tecton as their primary feature platform, making it a cornerstone of their feature development and management undertakings. This decision was made to unify ML processes, strengthen governance structures, and improve data quality and traceability from start to finish. With Tecton, Flo Health aims to gain deeper insights into the value and ROI of its features, addressing major hurdles in managing ML projects end-to-end.
Tecton’s Place in Flo Health’s Technical Framework
Within Flo Health’s technical structure, Tecton’s feature platform is steadily becoming an essential component. The platform integrates seamlessly with other tools, securely pulling and transforming raw data from many sources. This data is then transformed into batch, streaming, and real-time features for the models. With time, Flo Health’s feature store, powered by Tecton, has amassed over 1600 features, forming a comprehensive repository of materialized feature values and feature definitions for their machine learning initiatives. Furthermore, Flo Health is now able to:
- Build and use the same pipelines for training/inference
- Leverage built-in materialization for the online store
- Generate point-in-time correct joins for dataset collection from offline storage
- Share features across teams and projects easily
Through integrating Tecton’s platform, Flo Health has rolled out multiple distinct yet vital ML-driven applications, elevating their offerings and user interactions. This partnership with Tecton empowers Flo Health to harness ML to propel their business, and their success with these models exemplifies the advantages organizations can reap when incorporating an enterprise feature platform like Tecton into their ML strategies. Finally, platformization has allowed Flo Health to focus on business use cases while significantly reducing time spent on ML infrastructure conversations.
Landing on the ML Platform
Within Flo Health’s predictive ecosystem—comprising the Menstrual Cycle Prediction, Mode Switch Prediction, and more—all models are designed to provide real-time, on-demand predictions each time a user engages with the Flo Health app.
|
Before |
With Tecton |
|---|---|
Training vs Inference feature discrepancies. |
Tecton serves features for training & inference. |
20 SQL scripts just for training dataset collection, data leaks. |
Streamlined dataset building with a single Tecton API call with point-in-real-time correctness. |
No infrastructure to test model, shadow mode, canary release etc. |
ML Hub in place, enables easy separation between deployment modes, and monitoring. |
Feature pipeline required a restart every month: drops in prediction coverage |
Feature aggregator migrated to Databricks, results ingested into Tecton. No more restarts! |
Training on local machines, slow and insecure |
Collaborative notebook environment on Databricks, scalable, secure & reproductible. |
Model artefacts (binaries) stored in Git. |
Stored in Databricks-managed ML Flow enabling experiment tracking, containerized. |
Stake holders first to indetify model degradations & drift. |
Statistical features & output monitoring in place with the help of Evidently.ai as part of the ML Platform features. |
Tecton & Flo Health, in Practice
Through its integration with Tecton, Flo Health has crafted a platform tailored to its specific needs, especially when building intricate predictive models. This partnership has pinpointed two main use cases (among others), each catering to distinct demands and enhancing user experiences: the Period and Ovulation Cycle Prediction Model and the Mode Switch Prediction Model.
1. Period & Ovulation Cycle Prediction Model
Tecton underwent testing with Flo Health’s complex menstrual cycle prediction tool. This feature uses advanced AI and ML to accurately predict a user’s menstrual cycle, including periods and ovulation phases. The model aims to forecast the menstrual cycle and pinpoint fertile windows, giving users a clearer understanding of their own bodies. A smart notification system further enhances this feature, sending daily alerts about potential symptoms based on the menstrual cycle’s stage.
Additionally, users can log different symptoms. The system then cross-references this logged data with its predictions, delivering tailored period forecasts. Users can also set customized reminders for events like the start of a period or ovulation, supporting proactive menstrual health management.
Given the importance of cycle predictions in the app, Flo Health is meticulously integrating Tecton to ensure consistency and reliability. As a starting point, Tecton operated in a “shadow mode.” Flo Health conducts about 7 million ML predictions daily, with 2% currently powered by Tecton—a percentage set to grow soon. Currently, Tecton manages temperature data from the Apple Watch Series 8, while other data points are processed separately before being sent to the system.
In the upcoming months, Flo Health plans to:
- Boost Tecton’s role in predictions, aiming for 100% involvement
- Transition all feature processing to Tecton
As Flo Health continues to evolve, its pursuit of the best user experience remains constant. Although current symptom predictions rely on medically approved guidelines matched to a user’s cycle day and phase, past experiments hint at the potential benefits of using ML. The goal for Flo Health is to integrate ML-based symptom predictions, pending the alignment of client and backend functionalities across the app.
2. Mode Switch Prediction Model
Tecton’s feature platform also enhances Flo Health’s Mode Switch Prediction Model. This ML-based tool identifies users who might benefit from the app’s pregnancy mode. The ultimate goal is to ensure users make the most of app’s features, like Health Insights and Health Assistant, especially during their fertile window.
Once the Pregnancy Mode is on, users get a tailored experience. The app showcases the gestational age, offers daily updates on the baby’s growth, and adjusts the calendar to highlight key pregnancy events starting from the first day of the user’s last period. Beyond tracking, this mode gives a unique view of pregnancy. A delivery logging option appears after 20 weeks.
Pregnancy Mode doesn’t stop at logging; it offers custom content for each week of pregnancy. Users receive daily personalized stories full of insights. They can dive into a handpicked collection of articles, videos, and courses in the Insights section for a comprehensive look at their pregnancy journey.
The development of this model introduced approximately 1600 new features to the Tecton feature store. By centralizing and advancing these features, Flo Health has enhanced its ability to create additional predictive models, like those for predicting user churn and helping improve user retention strategies.
About Flo Health:
Flo Health is the most popular women’s health app globally. It is the #1 recommended app for period and cycle tracking based on a survey among 500 female U.S. obstetrician-gynecologists (ob/gyns). Over 300 million people have downloaded Flo Health, and 56 million use it monthly. With over 100+ medical experts working together to support Flo Health, the app assists women during their entire reproductive lives. It provides curated cycle and ovulation tracking, personalized health insights, expert tips, and a private community for women to share their questions and concerns. Flo Health prioritizes safety and focuses on being the most trusted digital source for women’s health information. The Flo Health app is available in more than 20 languages on iOS and Android.
Challenge:
Flo Health’s ML Growth Faced Operational & Governance Challenges
Flo Health’s ML team faced operational challenges, spending most of their time on project management rather than ML delivery. The lack of a centralized platform led to prolonged model updates, deployment delays, and governance issues.
Solution:
Toward a Unified ML Strategy: Flo Health’s Platformization Approach
To streamline its ML operations, Flo Health adopted Tecton as its central feature platform after thorough evaluations. This choice aimed to unify ML processes, enhance governance, and ensure data quality and traceability.
Results:
Tecton’s Feature Platform, a Crucial Part of Flo Health’s Technical Infrastructure
Flo Health’s feature store, bolstered by Tecton, now contains over 1600 features essential for their ML projects. By incorporating Tecton, Flo Health has launched significant ML-driven applications tailored for platforms like iOS, Android, and the web, including the Period and Ovulation Cycle Prediction Model and Mode Switch Prediction Model, and have been able to:
- Build and use the same pipelines for training/inference of their models
- Leverage built-in materializations for the online store
- Generate point-in-time correct joins for dataset collection from offline storage
- Share features across teams and projects easily
Ready to get started?
