Steering Gemini for health coaching
“Do I get better sleep after exercising?” sounds like a simple question, but answering it like a proactive, personalized and adaptive coach required several technical innovations.
First, we need the coach to understand and do numerical reasoning on physiological time series data such as sleep and activity, using capabilities similar to those showcased by PH-LLM. For questions like this, the coach verifies recent data availability, chooses the right metrics, contrasts relevant days, contextualizes results against personal baselines and population-level statistics, incorporates prior interactions with the coach, and finally uses the analysis to provide tailored answers and insights.
Second, we utilize a multi-agent framework that coordinates expert sub-agents to provide clear, consistent and holistic support, such as (1) a conversational agent for multi-turn conversations, intent understanding, agent orchestration, context gathering and response generation; (2) a data science agent that iteratively uses tools to fetch, analyze, and summarize relevant data (e.g., sleep and workout data), leveraging code-generation capabilities as needed; and (3) a domain expert, such as a fitness expert that analyzes user data to generate personalized fitness plans and adapt them as progress and context change.