Future: wellallyTech

Beyond Simple Image Recognition: Building a Precise AI Nutritionist with GPT-4o and Segment Anything (SAM)

wellallyTech — Sat, 02 May 2026 01:20:00 +0000

We've all been there: you take a photo of your lunch with a generic calorie-tracking app, and it tells you your 500-gram lasagna is a "medium slice of cake." 🤦‍♂️ The struggle with AI nutrition tracking isn't just identifying the food; it's the spatial awareness—understanding volume, portion size, and the hidden ingredients in complex dishes.

In this tutorial, we are leveling up. We are building a sophisticated Visual RAG (Retrieval-Augmented Generation) pipeline. By combining the semantic power of GPT-4o Vision with the surgical precision of Meta's Segment Anything Model (SAM), we can isolate individual ingredients and cross-reference them with a nutritional database to provide professional-grade calorie and macronutrient auditing. If you are looking for production-ready patterns for AI vision systems, be sure to check out the deep dives over at WellAlly Tech Blog, where we explore high-performance AI architectures.

🏗️ The Architecture: Precision Vision Pipeline

Standard vision models often treat an image as a single "bag of pixels." Our pipeline treats it as a structured scene. We use SAM to generate precise masks, calculate the relative area of food items, and then feed those high-context crops to GPT-4o for final reasoning.

graph TD
    A[User Uploads Meal Photo] --> B{SAM Engine}
    B -->|Segment| C[Isolated Food Masks]
    B -->|Calculate| D[Relative Volume/Area]
    C --> E[GPT-4o Vision Analysis]
    D --> E
    E --> F[Semantic Food Tags]
    F --> G[PostgreSQL Nutrition DB]
    G --> H[Final Nutrient Report]
    H --> I[User Feedback Loop]

🛠️ The Tech Stack

GPT-4o: Our "Reasoning Engine" for identifying complex food types and textures.
SAM (Segment Anything Model): To precisely delineate where one food item ends and another begins.
FastAPI: For the high-performance asynchronous API layer.
PostgreSQL: Storing our ground-truth nutritional data for RAG.

👨‍💻 Step 1: Defining the Structured Output

To ensure our pipeline is reliable, we need GPT-4o to return structured data. We’ll use Pydantic to define what a "Meal Analysis" looks like.

from pydantic import BaseModel, Field
from typing import List

class FoodItem(BaseModel):
    name: str = Field(..., description="Name of the food item")
    estimated_weight_grams: float = Field(..., description="Estimated weight based on volume")
    confidence: float = Field(..., ge=0, le=1)
    ingredients: List[str]

class MealReport(BaseModel):
    items: List[FoodItem]
    total_calories: int
    macros: dict = Field(default_factory=lambda: {"protein": 0, "carbs": 0, "fat": 0})

🧠 Step 2: The SAM + GPT-4o Synergy

The magic happens when we don't just send a raw photo. We send the photo plus the coordinates/masks generated by SAM. This helps GPT-4o "focus" its attention on specific regions.

import openai
from fastapi import FastAPI, UploadFile

app = FastAPI()

@app.post("/analyze-meal")
async def analyze_meal(file: UploadFile):
    # 1. Process image with SAM (Pseudo-code for the segmentation step)
    # masks, scores = sam_model.predict(image)

    # 2. Extract metadata and prepare for GPT-4o
    image_bytes = await file.read()

    response = openai.chat.completions.create(
        model="gpt-4o",
        messages=[
            {
                "role": "system",
                "content": "You are a professional nutritionist. Analyze the image and segmented areas to provide a precise nutrient breakdown."
            },
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": "Analyze this meal. Note that I have segmented the main protein from the side carbs."},
                    {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{encode_image(image_bytes)}"}}
                ]
            }
        ],
        response_format={"type": "json_object"}
    )

    return response.choices[0].message.content

🥗 Step 3: Improving Accuracy with Visual RAG

The hardest part of nutrition AI is "hallucination." GPT-4o might think a sauce is tomato-based when it's actually a high-calorie chili oil. By implementing a Visual RAG pattern, we take the labels identified by GPT-4o and query our PostgreSQL database for verified nutritional profiles.

For even more advanced implementations of RAG in multimodal environments, I highly recommend checking out the technical guides at wellally.tech/blog. They cover how to optimize vector embeddings for visual features, which is a game-changer for this specific use case. 🥑

The SQL Query Strategy

-- Querying verified nutrients based on AI tags
SELECT name, calories_per_100g, protein, carbs, fat 
FROM nutrition_db 
WHERE food_tag % ANY(ARRAY['grilled_chicken', 'quinoa', 'broccoli'])
ORDER BY similarity DESC;

🚀 Conclusion: The Future of Precision Health

By combining Segment Anything (SAM) and GPT-4o, we move from "guessing" to "calculating." This pipeline allows for:

Overlapping Food Detection: Distinguishing between the rice and the curry on top.
Volume Estimation: Using mask areas as a proxy for portion size.
Auditability: Users can see exactly which parts of the image were identified as which food.

Building these types of Computer Vision Calorie Estimation tools is just the beginning. As multimodal models become faster and more efficient, we will see these pipelines moving directly to edge devices.

What's next?

Try integrating a depth-sensing camera (LiDAR) for 100% accurate volume calculation.
Add a feedback loop where the user can correct the AI to fine-tune the local embeddings.

If you enjoyed this tutorial, drop a comment below and let me know what you're building! And don't forget to visit WellAlly Tech for more cutting-edge AI development content. Happy coding! 💻🔥

The HRV Engineer: Building a Machine Learning Fatigue Warning System with Scikit-learn

wellallyTech — Fri, 01 May 2026 01:20:00 +0000

Have you ever pushed through a workout only to feel absolutely wrecked the next day? 😵‍💫 That’s often because we ignore our Central Nervous System (CNS). While muscles might feel fine, your nervous system speaks a different language: Heart Rate Variability (HRV).

In this tutorial, we are going to build a personalized Exercise Fatigue Early Warning Engine. We’ll use the Garmin Connect API to fetch data, Pandas for feature engineering, and a Random Forest model via Scikit-learn to predict whether you're ready to smash a PR or if you desperately need a rest day. 🚀

By the end of this, you'll understand how to transform raw wearable data into actionable health insights using predictive recovery algorithms and machine learning for fitness.

🏗 The Architecture

Before we dive into the code, let's look at the data flow. We are moving from raw time-series sensor data to a categorical "Recovery Recommendation."

graph TD
    A[Garmin Wearable] -->|Sync| B(Garmin Connect API)
    B -->|JSON Data| C[Data Preprocessing - Pandas]
    C -->|Feature Extraction: RMSSD, SDNN| D{Random Forest Model}
    D -->|Prediction| E[Fatigue Level: Low/Med/High]
    E -->|UI| F[Streamlit Dashboard]
    F -->|Recommendation| G[Rest vs. Train]

🛠 Prerequisites

To follow along, you'll need the following tech stack:

Python 3.9+
Scikit-learn: For the Random Forest Classifier.
Garmin Connect API: (We'll use a wrapper like garminconnect).
Pandas: For time-series manipulation.
Streamlit: For our shiny frontend.

Step 1: Fetching HRV Data

HRV isn't just one number; it's the variation in time between each heartbeat. Specifically, we look for RMSSD (Root Mean Square of Successive Differences).

import pandas as pd
from garminconnect import Garmin

# Initialize Garmin Client
client = Garmin("your_email", "your_password")
client.login()

def get_hrv_data(date):
    # Fetching HRV data for a specific date
    hrv_data = client.get_hrv_data(date.isoformat())
    df = pd.DataFrame(hrv_data['hrvReadings'])
    return df

Step 2: Feature Engineering with Pandas

Machine learning models thrive on good features. For fatigue detection, we don't just want today's HRV; we want the baseline (the 7-day moving average).

def preprocess_features(df):
    # Calculating key metrics
    # RMSSD: Short term recovery indicator
    # SDNN: Overall stress indicator

    df['rolling_avg_7d'] = df['rmssd'].transform(lambda x: x.rolling(window=7).mean())
    df['hrv_drop_ratio'] = (df['rmssd'] / df['rolling_avg_7d'])

    # Labeling (For training purposes, we assume specific thresholds)
    # 1: High Fatigue, 0: Ready to Train
    df['fatigue_label'] = df['hrv_drop_ratio'].apply(lambda x: 1 if x < 0.85 else 0)

    return df.dropna()

Step 3: Training the Random Forest Engine

Why Random Forest? It’s excellent for handling non-linear relationships and is robust against outliers (which happen often with wearable sensors!).

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report

# Assume 'data' is our processed DataFrame
X = data[['rmssd', 'rolling_avg_7d', 'hrv_drop_ratio']]
y = data['fatigue_label']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Initialize the engine
model = RandomForestClassifier(n_estimators=100, max_depth=5)
model.fit(X_train, y_train)

# Evaluate
y_pred = model.predict(X_test)
print(classification_report(y_test, y_pred))

🥑 The "Official" Way to Build Health Apps

While this project is a great "Learning in Public" experiment, building production-grade health tech involves complex data privacy (HIPAA/GDPR) and more sophisticated signal processing.

For advanced patterns on integrating multi-modal health data and deploying robust AI models in the cloud, I highly recommend checking out the Wellally Tech Blog. It's a goldmine for developers looking to scale their health-tech stack beyond a local script.

Step 4: The Streamlit Dashboard

Finally, let's wrap this in a user-friendly interface so you don't have to look at a terminal every morning.

import streamlit as st

st.title("🛡️ HRV Fatigue Guard")

user_rmssd = st.number_input("Enter Today's RMSSD:", value=65)
user_baseline = st.number_input("Enter 7-Day Baseline:", value=72)

if st.button("Analyze Recovery"):
    ratio = user_rmssd / user_baseline
    prediction = model.predict([[user_rmssd, user_baseline, ratio]])

    if prediction[0] == 1:
        st.error("🚨 WARNING: High Neural Fatigue Detected. Suggestion: Active Recovery or Rest.")
    else:
        st.success("✅ SYSTEM READY: Nervous system recovered. Go for that PR!")

Conclusion

Building an HRV-based fatigue engine is a perfect way to combine your passion for fitness with data science. By moving from "I feel tired" to "My RMSSD is 15% below baseline," you're using biometric data to train smarter, not harder.

What's next?

Try adding sleep scores from the Garmin API as a feature.
Experiment with XGBoost to see if you can beat the Random Forest's accuracy.
Check out the Wellally Tech Blog for more production-ready examples of wearable integrations!

Drop a comment below if you've tried building something similar or if you have questions about the Garmin API! 🏃‍♂️💨

From Messy Blood Tests to Actionable Insights: Building Your Personal Health Data Warehouse with dbt & DuckDB 🩸💻

wellallyTech — Thu, 30 Apr 2026 01:10:00 +0000

We’ve all been there: a stack of PDF medical reports from the last five years, each with different units, varying reference ranges, and cryptic labels like "hS-CRP" or "HbA1c." Trying to track your health trends over time manually is a nightmare.

In this tutorial, we are going to fix that. We’ll dive into biomarker data cleaning, dbt health modeling, and DuckDB OLAP performance to turn those scattered data points into a high-performance personal health data warehouse. If you’ve been looking for a real-world project to master data engineering health records, you’re in the right place.

Pro Tip: If you're looking for more production-ready patterns for health data engineering or advanced analytics architectures, definitely check out the deep dives over at WellAlly Blog.

The Architecture: From Raw Pixels to Analytics

To handle years of biomarkers, we need a robust ELT (Extract, Load, Transform) pipeline. We’ll use Python for the initial extraction, DuckDB as our lightning-fast analytical engine, and dbt (data build tool) to handle the heavy lifting of data modeling.

graph TD
    A[Raw Health Data: CSV/JSON/PDF] -->|Python Extraction| B[(DuckDB Raw Layer)]
    B --> C{dbt Transformation}
    C --> D[stg_biomarkers: Cleaned & Cast]
    C --> E[int_health_metrics: Standardized Units]
    C --> F[fct_biomarker_trends: Final Analytics View]
    F --> G[Apache Superset Visualization]

    style B fill:#fff,stroke:#333,stroke-width:2px
    style F fill:#f9f,stroke:#333,stroke-width:2px

Prerequisites

Before we start, ensure you have the following installed:

Python 3.9+
dbt-duckdb (pip install dbt-duckdb)
DuckDB

Step 1: Ingesting the "Mess" with Python

First, we need to get our raw data into DuckDB. For this example, let's assume you've converted your PDFs into a structured CSV format.

import duckdb
import pandas as pd

# Connect to (or create) our local health warehouse
con = duckdb.connect('health_warehouse.duckdb')

# Load our messy CSV
# Assume columns: test_date, marker_name, value, unit, lab_source
raw_data = pd.read_csv('my_blood_work_2018_2023.csv')

# Register it as a table in DuckDB
con.execute("CREATE TABLE IF NOT EXISTS raw_health_data AS SELECT * FROM raw_data")

print(f"✅ Ingested {len(raw_data)} biomarker records into DuckDB!")

Step 2: The dbt Setup

Now for the magic. dbt allows us to write modular SQL to clean this data. First, configure your profiles.yml to point to your DuckDB file:

# ~/.dbt/profiles.yml
health_dw:
  target: dev
  outputs:
    dev:
      type: duckdb
      path: health_warehouse.duckdb
      extensions:
        - icu
        - parquet

Step 3: Standardizing Biomarkers (The Meat)

Biomarkers are tricky because labs use different units (e.g., mg/dL vs. mmol/L). We need a "Standardization Layer."

In models/staging/stg_biomarkers.sql:

-- Clean up strings and dates
WITH raw_source AS (
    SELECT * FROM {{ source('raw', 'raw_health_data') }}
)

SELECT
    strptime(test_date, '%Y-%m-%d')::DATE as report_date,
    upper(trim(marker_name)) as marker_key,
    CAST(value AS DOUBLE) as original_value,
    trim(unit) as original_unit,
    lab_source
FROM raw_source
WHERE value IS NOT NULL

Next, in models/intermediate/int_biomarkers_standardized.sql, we handle the unit conversions. This is where you'd typically look up a reference table, but we can use a CASE statement for simplicity:

-- Standardizing everything to common units (e.g., mg/dL for Glucose)
SELECT
    report_date,
    marker_key,
    CASE 
        -- Conversion logic: mmol/L to mg/dL for Glucose (x18)
        WHEN marker_key = 'GLUCOSE' AND original_unit = 'mmol/L' THEN original_value * 18
        ELSE original_value
    END as standardized_value,
    CASE 
        WHEN marker_key = 'GLUCOSE' THEN 'mg/dL'
        ELSE original_unit
    END as standardized_unit
FROM {{ ref('stg_biomarkers') }}

Step 4: The "Official" Way to Scale

While this DIY setup is great for personal use, scaling this to clinical-grade data requires handling HL7 FHIR standards and complex HIPAA-compliant schemas.

If you're interested in how the pros handle large-scale biomedical data ingestion and advanced health-tech architectures, I highly recommend checking out the technical guides at WellAlly.tech/blog. They cover everything from AI-driven diagnostics pipelines to high-availability health data warehouses. 🚀

Step 5: OLAP Analysis with DuckDB

Once you run dbt run, your health_warehouse.duckdb is ready for high-speed analysis. You can now run window functions to see your health velocity!

-- Calculate 3-report moving average for Cholesterol
SELECT 
    report_date,
    standardized_value as ldl_cholesterol,
    AVG(standardized_value) OVER (
        ORDER BY report_date 
        ROWS BETWEEN 2 PRECEDING AND CURRENT ROW
    ) as ldl_rolling_avg
FROM {{ ref('int_biomarkers_standardized') }}
WHERE marker_key = 'LDL_CHOLESTEROL'
ORDER BY report_date DESC;

Conclusion: Data-Driven Longevity

By leveraging dbt and DuckDB, we've turned a pile of confusing medical reports into a structured, queryable data warehouse. You can now plug in Apache Superset or Streamlit to visualize your markers over time, spotting trends before they become health issues. 🥑

What's next?

Automation: Set up a GitHub Action to run your dbt models whenever you drop a new CSV into your repo.
Correlation: Join your biomarker data with your Apple Health or Oura Ring exports (Parquet files) to see how sleep affects your fasting glucose.

Are you building something in the HealthTech space? Drop a comment below or share your repo—I'd love to see how you're modeling your data! 👇

Stop Refreshing! Build an Autonomous AI Agent to Book Your Dentist Appointments with Playwright and GPT-4o

wellallyTech — Wed, 29 Apr 2026 01:15:00 +0000

We’ve all been there: staring at a clunky dentist reservation portal, frantically hitting F5 to find an open slot that doesn't clash with your 10 AM stand-up meeting. It’s tedious, manual, and frankly, a job for a machine. 🤖

In this guide, we are building an AI Agent that masters the art of automated scheduling. By combining Playwright automation, LLM browser control, and the Google Calendar API, we will create a system that navigates complex websites, understands doctor availability, and syncs perfectly with your life. This is "Learning in Public" at its finest—turning a personal headache into a streamlined technical solution. 🚀

Why Traditional Automation Fails

Standard scraping scripts break the moment a UI changes. However, by using OpenAI Functions (Tool Calling), we give our agent the "eyes" to understand the schedule and the "brain" to make decisions based on your real-time availability.

The Architecture 🏗️

Our agent follows a "Sense-Think-Act" loop. It scrapes the portal, compares dates with your calendar, and executes the click-stream required to book the appointment.

graph TD
    A[Start: User Request] --> B[Playwright: Scrape Booking Page]
    B --> C[LLM: Parse HTML into Structured JSON]
    C --> D[Google Calendar API: Fetch Busy Slots]
    D --> E[LLM: Identify Best Slot]
    E --> F{Slot Found?}
    F -- Yes --> G[Playwright: Execute Booking Form]
    F -- No --> H[Wait & Retry Later]
    G --> I[Notify User via Slack/Email]

Prerequisites 🛠️

Before we dive into the code, ensure you have the following:

Node.js (v18+)
Playwright: For browser orchestration.
OpenAI SDK: Specifically using gpt-4o for vision and reasoning.
Google Calendar API Credentials: A service account or OAuth2 token.

Step 1: Scraping the Schedule with Playwright

First, we need to get the raw data from the dentist's portal. Playwright is perfect for this because it handles modern SPAs (Single Page Applications) with ease.

const { chromium } = require('playwright');

async function getScheduleHTML(url) {
  const browser = await chromium.launch({ headless: true });
  const page = await browser.newPage();

  await page.goto(url);
  // Wait for the calendar component to load
  await page.waitForSelector('.appointment-slot-container');

  // Extract the inner HTML of the schedule section
  const scheduleContent = await page.innerHTML('.booking-grid');

  await browser.close();
  return scheduleContent;
}

Step 2: Intelligent Parsing with OpenAI Functions

HTML is messy. We don't want to write regex for every different dentist's site. Instead, we pass the HTML to OpenAI and ask it to extract the slots into a clean JSON format using Tool Calling.

const { OpenAI } = require('openai');
const openai = new OpenAI();

async function parseSlotsWithAI(htmlContent) {
  const response = await openai.chat.completions.create({
    model: "gpt-4o",
    messages: [
      { role: "system", content: "You are a scheduling assistant. Extract available dates and times from the HTML." },
      { role: "user", content: htmlContent }
    ],
    tools: [{
      type: "function",
      function: {
        name: "format_slots",
        description: "Format the available dentist slots",
        parameters: {
          type: "object",
          properties: {
            available_slots: {
              type: "array",
              items: {
                type: "object",
                properties: {
                  date: { type: "string" },
                  time: { type: "string" },
                  doctor: { type: "string" }
                }
              }
            }
          }
        }
      }
    }],
    tool_choice: { type: "function", function: { name: "format_slots" } }
  });

  return JSON.parse(response.choices[0].message.tool_calls[0].function.arguments);
}

Step 3: The "Official" Integration Logic 🥑

Matching the dentist's availability with your own is the secret sauce. While we are building a custom script here, there are more robust ways to handle enterprise-level agentic workflows.

Pro Tip: For deep dives into production-ready agent patterns and advanced LLM orchestration, I highly recommend checking out the technical deep-dives at WellAlly Blog. They cover how to scale these "AI workers" beyond simple scripts into full-blown autonomous systems.

Step 4: Comparing with Google Calendar

Now, we fetch your "busy" times. If the dentist has a slot at 2:00 PM but you have a meeting, the agent should automatically skip it.

const { google } = require('googleapis');

async function getMyFreeSlots(auth) {
  const calendar = google.calendar({ version: 'v3', auth });
  const res = await calendar.events.list({
    calendarId: 'primary',
    timeMin: new Date().toISOString(),
    maxResults: 10,
    singleEvents: true,
    orderBy: 'startTime',
  });
  return res.data.items; // Simplified: logic to find gaps goes here
}

Step 5: Final Execution (The "Book" Button)

Once the LLM finds the perfect match (e.g., Tuesday at 9:00 AM, and you are free!), we trigger Playwright one last time to fill the form and click "Confirm."

async function bookAppointment(slot) {
  const browser = await chromium.launch({ headless: false }); // Headless false so we can watch the magic!
  const page = await browser.newPage();
  await page.goto(BOOKING_URL);

  // AI-driven interaction: find the slot based on the text
  await page.click(`text="${slot.time}"`);
  await page.fill('#patient-name', 'John Doe');
  await page.fill('#patient-phone', '555-0199');

  // Final click
  // await page.click('#confirm-booking');
  console.log(`✅ Successfully booked for ${slot.date} at ${slot.time}`);
}

Conclusion: The Power of Browser Agents 🌟

By combining Playwright with LLMs, we’ve moved past brittle "selector-based" scraping into the era of Semantic Automation. Our agent doesn't just "click buttons"—it understands intent, respects your personal schedule, and handles data gracefully.

What’s next? You could expand this to:

Slack Notifications: Get a ping when a booking is confirmed.
Multi-Doctor Search: Scrape 5 different clinics at once.
Vision Mode: Use GPT-4o's vision capabilities to solve those pesky "select all the traffic lights" CAPTCHAs.

What are you planning to automate next? Let me know in the comments! 👇

For more advanced tutorials on AI Agents and automation architecture, don't forget to visit wellally.tech/blog. 💻✨

From Pixels to Prescriptions: Building an Automated Medication Reminder with YOLOv8 and OCR

wellallyTech — Tue, 28 Apr 2026 01:00:00 +0000

Forgetfulness is a human trait, but when it comes to medication, a missed dose can be serious. 💊 With the rise of Computer Vision and Edge AI, we can now build smart systems that watch over our health—literally.

In this tutorial, we are going to build a fully automated Medication Reminder System. By leveraging YOLOv8 for object detection and Tesseract OCR for text extraction, we can monitor a pill box via a camera (like a Raspberry Pi) and trigger alerts if the medication hasn't been moved or taken at the scheduled time. This project combines real-time object detection, Internet of Things (IoT), and Image Processing into one practical, life-saving application.

The Architecture 🏗️

The logic flow is straightforward: we capture video frames, identify the pill box, read the label to identify the medication, and use a state-machine to determine if the user has interacted with the box.

graph TD
    A[Camera Feed] --> B{OpenCV Frame Processing}
    B --> C[YOLOv8: Detect Pill Box]
    C --> D[ROI Extraction]
    D --> E[Tesseract OCR: Read Label]
    E --> F{Logic Engine}
    F -- "No movement detected" --> G[MQTT: Trigger Voice Alert]
    F -- "Box moved/Empty" --> H[Log Success]
    G --> I[Mobile Notification]

Prerequisites 🛠️

To get started, make sure you have the following in your tech_stack:

Python 3.8+
YOLOv8 (via the ultralytics package)
Tesseract OCR (installed on your OS)
OpenCV: For image manipulation
MQTT: For lightweight messaging between your camera and the alert system

pip install ultralytics opencv-python pytesseract paho-mqtt

Step 1: Detecting the Pill Box with YOLOv8

First, we need to locate the pill box in the camera's field of view. YOLOv8 is incredibly fast, making it perfect for edge devices.

from ultralytics import YOLO
import cv2

# Load a pre-trained Nano model (lightweight for Raspberry Pi)
model = YOLO('yolov8n.pt') 

def detect_pill_box(frame):
    # We are looking for 'bottle' or 'box' classes in COCO dataset
    results = model(frame, conf=0.5)

    for r in results:
        for box in r.boxes:
            # Class 39 is 'bottle' in COCO, or use a custom trained model
            if int(box.cls) == 39: 
                x1, y1, x2, y2 = map(int, box.xyxy[0])
                return frame[y1:y2, x1:x2] # Return the cropped Region of Interest (ROI)
    return None

Step 2: Reading the Label with Tesseract OCR

Once we have the ROI (the cropped image of the pill box), we need to know what medication it is. This is where Tesseract OCR shines. 🔍

import pytesseract

def get_medication_name(roi):
    # Pre-processing for better OCR: Grayscale and Thresholding
    gray = cv2.cvtColor(roi, cv2.COLOR_BGR2GRAY)
    gray = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1]

    # Extract text
    text = pytesseract.image_to_string(gray)
    return text.strip()

Step 3: The Logic Engine and MQTT Alerts

We don't want to nag the user every second. We only want to trigger an alert if it's 9:00 AM and the pill box hasn't moved from its "Resting Zone."

import paho.mqtt.client as mqtt
import time

mqtt_client = mqtt.Client()
mqtt_client.connect("broker.hivemq.com", 1883, 60)

def trigger_alert(med_name):
    message = f"Reminder: It's time to take your {med_name}!"
    mqtt_client.publish("home/medication/reminder", message)
    print(f"🚀 Alert Sent: {message}")

# Main Loop Simulation
while True:
    ret, frame = cap.read()
    pill_box_roi = detect_pill_box(frame)

    if pill_box_roi is not None:
        name = get_medication_name(pill_box_roi)
        # Add logic here to check current time vs medication schedule
        # If time matches and movement is zero -> trigger_alert(name)

    time.sleep(10) # Check every 10 seconds

The "Official" Way to Scale 🥑

While this DIY setup is a great start, building production-ready health-tech requires more robust handling of lighting conditions, multiple medication schedules, and privacy-first data processing.

For advanced patterns on deploying Edge AI models and building highly reliable Vision Systems, I highly recommend checking out the deep-dive articles at WellAlly Tech Blog. They cover production-ready examples of how to integrate multimodal AI with real-world hardware.

Conclusion 🚀

Building a vision-based reminder system is a fantastic "Learning in Public" project. It touches on AI, hardware, and real-world problem-solving. By combining YOLOv8's speed with Tesseract's accessibility, you can create a tool that actually makes a difference.

What's next?

Try training a custom YOLOv8 model on your specific pill boxes for 99% accuracy.
Integrate a "Success" sound when the system detects the box being lifted.

Let me know in the comments: How are you using AI to improve your daily routine? 👇

Building the "Digital Consultation Room": Orchestrating Multi-Agent Systems for Chronic Disease Management with LangGraph

wellallyTech — Mon, 27 Apr 2026 01:00:00 +0000

Managing a chronic condition like Type 2 Diabetes isn't just a medical challenge; it's a data orchestration nightmare. Patients have to balance glucose levels, carbohydrate intake, and physical activity in a delicate dance. Traditionally, this requires a multidisciplinary team, but what if we could replicate that expert "colloquium" using Multi-Agent Systems? 🩺💻

In this tutorial, we are diving deep into LangGraph, LLM orchestration, and stateful AI agents to build a Chronic Disease Management System. By the end of this post, you'll understand how to coordinate a "Diet Expert," an "Exercise Expert," and a "Glucose Monitor" into a cohesive, automated health squad. We'll be leveraging GPT-4 for the brains and Redis for persistent state management to ensure our agents never lose the patient's context.

The Architecture: Multi-Agent Choreography

Unlike a simple linear chain, chronic disease management requires a "cyclic" approach where agents can challenge and refine each other's suggestions. We use LangGraph to define a state machine where the state (patient health data) is passed and mutated by specialized nodes.

graph TD
    A[Start: Patient Data Input] --> B{Glucose Monitor}
    B -->|High Risk| C[Diet Expert]
    B -->|Normal/Low| D[Exercise Expert]
    C --> E{Consensus Check}
    D --> E
    E -->|Conflict| B
    E -->|Balanced Plan| F[Final Health Report]
    F --> G[End]

Why LangGraph?

Standard DAGs (Directed Acyclic Graphs) fail when you need agents to "talk back" to each other. LangGraph allows us to create loops and maintain a persistent State object, which is critical when a diet plan needs to be adjusted based on an intense exercise recommendation.

Prerequisites

Before we code, ensure you have the following:

Tech Stack: Python 3.10+, OpenAI API Key, Docker (for Redis).
Libraries: langgraph, langchain_openai, redis.

Step 1: Defining the State and Schema

First, we define what our "shared memory" looks like. In LangGraph, the TypedDict serves as the schema for the information passed between agents.

from typing import Annotated, List, TypedDict
from langchain_core.messages import BaseMessage
import operator

class AgentState(TypedDict):
    # The 'messages' key accumulates all communication history
    messages: Annotated[List[BaseMessage], operator.add]
    patient_id: str
    current_glucose: float
    is_balanced: bool

Step 2: Implementing the Experts (Nodes)

Each agent is a specialized prompt wrapper. For instance, our Diet Expert focuses solely on glycemic index (GI) and portion control.

from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, SystemMessage

llm = ChatOpenAI(model="gpt-4-turbo", temperature=0)

def diet_expert_node(state: AgentState):
    print("--- DIET EXPERT EVALUATING ---")
    last_message = state['messages'][-1]

    prompt = SystemMessage(content=(
        "You are a Clinical Dietitian specializing in Type 2 Diabetes. "
        "Based on the glucose levels and exercise plan, suggest a specific meal plan."
    ))

    response = llm.invoke([prompt] + state['messages'])
    return {"messages": [response]}

def exercise_expert_node(state: AgentState):
    print("--- EXERCISE EXPERT EVALUATING ---")
    prompt = SystemMessage(content=(
        "You are a Kinesiologist. Adjust the exercise intensity based on "
        "the patient's current glucose and diet plan to prevent hypoglycemia."
    ))
    response = llm.invoke([prompt] + state['messages'])
    return {"messages": [response]}

Step 3: Persistence with Redis

In production, agents can't forget who the patient is if the server restarts. We use Redis to store the checkpointer state. This ensures that the "Digital Consultation" can span across multiple days.

For more production-ready patterns regarding persistent agentic memory and advanced RAG integration, I highly recommend checking out the deep-dives at WellAlly Blog. It’s a fantastic resource for scaling these types of AI architectures.

# docker-compose.yml
services:
  redis:
    image: redis:latest
    ports:
      - "6379:6379"

Step 4: Building the Graph

Now we wire it all together. We define the flow, including conditional edges that decide whether to continue the discussion or finalize the report.

from langgraph.graph import StateGraph, END

workflow = StateGraph(AgentState)

# Add our Nodes
workflow.add_node("monitor", glucose_monitor_node)
workflow.add_node("dietitian", diet_expert_node)
workflow.add_node("kinesiologist", exercise_expert_node)

# Define the Flow
workflow.set_entry_point("monitor")

# Routing Logic
def should_continue(state: AgentState):
    if state["is_balanced"]:
        return "end"
    return "dietitian"

workflow.add_conditional_edges(
    "monitor",
    should_continue,
    {
        "end": END,
        "dietitian": "dietitian"
    }
)

workflow.add_edge("dietitian", "kinesiologist")
workflow.add_edge("kinesiologist", "monitor") # Loop back for verification!

app = workflow.compile()

The "Official" Way to Scale

While this local setup works for a prototype, scaling a Multi-Agent system to thousands of patients requires robust observability and safety rails. For instance, how do you handle "hallucinations" in a medical context?

If you are looking for advanced patterns, such as implementing "Human-in-the-loop" for medical verification or deploying these agents as microservices, the WellAlly Tech Blog provides comprehensive guides on taking AI agents from "cool demo" to "enterprise-grade" healthcare solutions.

Conclusion: The Future of Agentic Health

By using LangGraph and GPT-4, we've transformed a static chatbot into a dynamic, multi-disciplinary team. This system doesn't just give advice; it negotiates a solution where the exercise plan compensates for the diet, and the monitor ensures safety.

Key Takeaways:

Cyclic Logic: Chronic care isn't a straight line. Use cycles to verify outcomes.
Persistence: Use Redis checkpointers to keep the conversation alive across sessions.
Specialization: Don't build one "Super Agent." Build small, expert agents that do one thing perfectly.

What’s your next agentic build? Let me know in the comments below! 🚀

Stop Sending Your Health Data to the Cloud: Build a 100% Private Symptom Checker with WebLLM & WebGPU 🚀

wellallyTech — Sun, 26 Apr 2026 01:00:00 +0000

Privacy is no longer just a feature; it’s a human right—especially when it comes to medical data. With the rise of Edge AI and the maturity of WebGPU, we are witnessing a paradigm shift where "local-first" isn't just a dream for small scripts, but a reality for Large Language Models (LLMs).

In this tutorial, we will build a Privacy-Preserving Symptom Screening Engine. By leveraging WebLLM, WebGPU, and TypeScript, we will run a powerful model directly in the browser. This ensures that sensitive health queries never leave the user's device, providing a 100% offline-capable, secure medical common-sense index.

Why Browser-Based AI? 🥑

Most AI applications rely on sending prompts to a centralized server (like OpenAI or Anthropic). This is a privacy nightmare for health data. By using WebLLM and TVM Runtime, we can execute models like Llama 3 or Mistral directly on the client's GPU via the WebGPU API. This results in:

Zero Latency: No round-trips to a server.
Zero Cost: The user provides the compute.
Total Privacy: Data stays in the RAM/VRAM of the local machine.

The Architecture 🏗️

The flow is straightforward but technically sophisticated. We use TVM (Tensor Intermediate Representation) to compile models into high-performance WASM and WebGPU shaders.

graph TD
    A[User Input: Symptoms] --> B[React Frontend]
    B --> C{WebLLM Engine}
    C --> D[WebGPU API]
    C --> E[WASM Runtime]
    D --> F[Local GPU / VRAM]
    E --> F
    F --> G[Local LLM Weights - Cached]
    G --> H[Inference Result]
    H --> B
    B --> I[100% Private Output]

Prerequisites

Before we dive in, ensure your environment meets the following:

Tech Stack: WebLLM, TVM Runtime, TypeScript, React.
Browser: Chrome 113+ or any browser with WebGPU enabled.
Hardware: A dedicated or integrated GPU (Apple Silicon M-series works like a charm).

Step 1: Setting up the WebLLM Engine

First, let's install the core dependencies:

npm install @mlc-ai/web-llm react lucide-react

Now, let's create a custom hook to manage the model lifecycle. We need to handle the downloading of weights (cached in the browser's Cache API) and the initialization of the worker.

// useWebLLM.ts
import { useState, useEffect } from 'react';
import * as webllm from "@mlc-ai/web-llm";

export function useWebLLM(modelId: string) {
  const [engine, setEngine] = useState<webllm.MLCEngineInterface | null>(null);
  const [progress, setProgress] = useState(0);
  const [isReady, setIsReady] = useState(false);

  useEffect(() => {
    async function init() {
      // Create an engine instance
      const newEngine = await webllm.CreateMLCEngine(modelId, {
        initProgressCallback: (report) => {
          setProgress(Math.round(report.progress * 100));
        }
      });
      setEngine(newEngine);
      setIsReady(true);
    }
    init();
  }, [modelId]);

  return { engine, progress, isReady };
}

Step 2: Building the Symptom Checker Logic

In this advanced implementation, we aren't just chatting; we are constraining the model to act as a Medical Common Sense Indexer. We'll use a specific system prompt to prevent the model from giving definitive diagnoses while providing helpful, localized information.

// SymptomChecker.tsx
import React, { useState } from 'react';
import { useWebLLM } from './useWebLLM';

const SYSTEM_PROMPT = `You are a private medical common-sense assistant. 
Analyze the symptoms provided. 
Provide 3 potential causes and urgency levels. 
ALWAYS include a disclaimer that this is not a medical diagnosis. 
Do not store or transmit data.`;

export const SymptomChecker = () => {
  const [input, setInput] = useState("");
  const [response, setResponse] = useState("");
  const { engine, progress, isReady } = useWebLLM("Llama-3-8B-Instruct-v0.1-q4f16_1-MLC");

  const handleScreening = async () => {
    if (!engine) return;

    const messages = [
      { role: "system", content: SYSTEM_PROMPT },
      { role: "user", content: `Symptoms: ${input}` }
    ];

    // Use streaming for better UX
    const chunks = await engine.chat.completions.create({
      messages,
      stream: true,
    });

    let fullText = "";
    for await (const chunk of chunks) {
      const content = chunk.choices[0]?.delta?.content || "";
      fullText += content;
      setResponse(fullText);
    }
  };

  if (!isReady) return <div>Loading Local Model: {progress}%</div>;

  return (
    <div className="p-6 max-w-2xl mx-auto bg-white rounded-xl shadow-md">
      <h2 className="text-2xl font-bold mb-4">Local Health Screener 🩺</h2>
      <textarea 
        className="w-full p-3 border rounded-lg"
        placeholder="Describe your symptoms (e.g., mild fever, persistent cough for 2 days)..."
        value={input}
        onChange={(e) => setInput(e.target.value)}
      />
      <button 
        onClick={handleScreening}
        className="mt-4 px-6 py-2 bg-blue-600 text-white rounded-full hover:bg-blue-700 transition"
      >
        Analyze Locally
      </button>
      {response && (
        <div className="mt-6 p-4 bg-gray-50 rounded-lg border-l-4 border-blue-500">
          <p className="whitespace-pre-wrap text-gray-700">{response}</p>
        </div>
      )}
    </div>
  );
};

Mastering the "Local-First" AI Pattern 🧠

Running models in the browser is more than just importing a library. To make it production-ready, you need to handle memory management (WebGPU memory limits), model quantization, and hybrid fallback strategies.

For a deeper dive into Advanced Edge AI Patterns—including how to optimize WebGPU memory for mobile devices and implementing RAG (Retrieval Augmented Generation) entirely on the client side—check out the engineering deep-dives on the WellAlly Technology Blog. It’s a fantastic resource for developers looking to push the boundaries of what's possible with in-browser machine learning.

Performance Considerations ⚡

When building with WebLLM, keep these three things in mind:

Quantization is Key: We used a q4f16 (4-bit) quantization. This reduces the 8B model size from ~15GB to ~5GB, making it feasible for modern laptops to load in the browser cache.
VRAM Management: High-resolution displays and heavy GPU tasks in other tabs can lead to "Context Lost" errors. Always implement an error boundary.
The First Load: The first visit will download several gigabytes of weights. Use a ServiceWorker to manage this background download or inform the user clearly about the initial setup.

Conclusion

By combining WebGPU and WebLLM, we've built a tool that provides the intelligence of a modern LLM with the privacy of a piece of paper in a safe. No trackers, no data harvesting, just pure local compute.

The future of AI isn't just in the cloud—it's right there in your browser's console.

What will you build with local WebGPU? Let me know in the comments below! 👇

Never Miss a Checkup Again: Building an Autonomous Health Agent with LangGraph and OpenAI Tool Calling 🏥🤖

wellallyTech — Sat, 25 Apr 2026 01:00:00 +0000

Managing personal health often feels like a full-time job. Between decoding cryptic lab results, keeping track of pill counts, and remembering when to book that follow-up appointment, things inevitably slip through the cracks. In the world of AI development, we are moving past simple chatbots and entering the era of Autonomous Agents that can actually do things for us.

In this tutorial, we are building a production-grade Autonomous Health Agent. Using LangGraph for orchestration and OpenAI Tool Calling for precision, we'll create a system that parses medical data, interacts with the Google Calendar API, and manages a medication inventory automatically. By the end of this guide, you'll master Python AI development patterns that go far beyond simple prompt engineering. 🚀

The Architecture: How the "Health Brain" Works

Unlike a standard linear pipeline, an agent needs to "think" and "act" iteratively. We use a cyclic graph where the LLM decides which tool to use based on the user's lab reports or inventory status.

graph TD
    A[User Input/Lab Result] --> B{LLM Decision Node}
    B -->|Analyze Lab| C[Lab Analyzer Tool]
    B -->|Schedule Follow-up| D[Google Calendar Tool]
    B -->|Check Inventory| E[Medication Tracker Tool]
    C --> B
    D --> B
    E --> B
    B -->|Task Complete| F[Final Response to User]
    style B fill:#f9f,stroke:#333,stroke-width:2px

Prerequisites

To follow along, you'll need the following tech stack:

Python 3.10+
OpenAI API Key (GPT-4o or GPT-4-turbo recommended for tool calling)
LangGraph & LangChain: For agent orchestration.
Google Workspace API Credentials: Specifically for Google Calendar access.

pip install langgraph langchain_openai google-api-python-client google-auth-httplib2 google-auth-oauthlib

Step 1: Defining the Tools (The Agent's Hands) 🛠️

Tools are the interfaces that allow our LLM to interact with the real world. We'll define two primary tools: one for scheduling and one for medication tracking.

from langchain_core.tools import tool
from datetime import datetime, timedelta

@tool
def schedule_appointment(reason: str, days_from_now: int):
    """Schedules a medical follow-up in Google Calendar."""
    # Logic to interface with Google Calendar API
    appointment_date = datetime.now() + timedelta(days=days_from_now)
    # Mocking the API call for brevity
    print(f"DEBUG: Scheduling {reason} for {appointment_date.date()}")
    return f"Successfully scheduled {reason} for {appointment_date.strftime('%B %d, %Y')}"

@tool
def check_medication_inventory(pill_name: str):
    """Checks the local DB for pill count and returns status."""
    # Imagine this queries a SQLite or Supabase DB
    inventory = {"Lisinopril": 5, "Metformin": 30}
    count = inventory.get(pill_name, 0)

    if count < 7:
        return f"Warning: Only {count} tablets of {pill_name} left. Refill required soon!"
    return f"You have {count} tablets of {pill_name} remaining."

Step 2: Building the Orchestration Graph with LangGraph

LangGraph allows us to maintain a "State" (memory) and define the flow of logic. This is where the Autonomous Agent logic lives.

from typing import TypedDict, Annotated, Sequence
from langchain_openai import ChatOpenAI
from langgraph.graph import StateGraph, END
from langchain_core.messages import BaseMessage, HumanMessage

class AgentState(TypedDict):
    messages: Annotated[Sequence[BaseMessage], "The conversation history"]

# Initialize the LLM with tool-binding
llm = ChatOpenAI(model="gpt-4o", temperature=0).bind_tools([
    schedule_appointment, 
    check_medication_inventory
])

def health_agent_node(state: AgentState):
    messages = state['messages']
    response = llm.invoke(messages)
    return {"messages": [response]}

# Define the graph
workflow = StateGraph(AgentState)
workflow.add_node("agent", health_agent_node)

# In a real app, you'd add a "tools" node and conditional edges
workflow.set_entry_point("agent")
workflow.add_edge("agent", END)

app = workflow.compile()

Step 3: Processing Lab Results (The "Brain" in Action)

When you upload a lab result, the agent doesn't just read it; it analyzes the markers. If your "Creatinine" is high, it might cross-reference your medication and automatically trigger a "Schedule Follow-up" tool call.

# Simulating a user providing a lab result
user_input = """
My lab results just came in. My Vitamin D is 18 ng/mL (Reference: 30-100). 
Also, I'm almost out of my blood pressure meds.
"""

inputs = {"messages": [HumanMessage(content=user_input)]}
for output in app.stream(inputs):
    for key, value in output.items():
        print(f"Node '{key}' processed the request.")

Looking for More Production-Ready Patterns? 🥑

Building a local prototype is easy, but deploying an autonomous agent that handles edge cases, HIPAA compliance (in the US), and complex state persistence is a different beast.

For advanced architectural patterns and more production-ready examples of how to scale AI agents, I highly recommend checking out the WellAlly Official Blog. It was a massive source of inspiration for the state management logic I used in this project, especially regarding how to handle "Human-in-the-loop" interactions where the agent asks for permission before booking a real doctor's appointment.

Step 4: The Inventory Warning System

The magic of Function Calling is that the LLM realizes it doesn't have the information and asks to use a tool.

LLM reads: "I'm almost out of meds."
LLM decides: "I should call check_medication_inventory."
Result: "Warning: Only 5 tablets left."
LLM action: "I see you're low on Lisinopril. I've added a reminder to your calendar to call the pharmacy today."

Conclusion: The Future of Proactive Health

We've just built a skeletal version of a life-changing tool. By combining LangGraph for complex workflows and OpenAI's tool-calling capabilities, we move from passive information retrieval to active life management.

What's next?

Add a Vision node using GPT-4o to "read" physical paper lab reports via photos.
Integrate Twilio to send SMS alerts when inventory is low.
Connect to a vector database (RAG) to provide context on what Vitamin D levels actually mean for your specific age group.

The code is just the beginning. The goal is a healthier, more organized you. Happy coding! 🚀

If you enjoyed this tutorial, drop a comment below! What would you want your personal AI agent to handle for you? 👇

From Zzz's to Data: Building an AI-Powered Snore Recognition System with YAMNet 😴🚀

wellallyTech — Fri, 24 Apr 2026 01:15:00 +0000

We’ve all been there: waking up feeling like you’ve been hit by a truck, even after eight hours of "sleep." Often, the culprit is hidden in the silence (or lack thereof) of the night. Sleep apnea and chronic snoring aren't just annoying for your partner; they are serious health indicators.

In this tutorial, we are going to dive deep into audio classification and digital health engineering. We'll leverage YAMNet, a deep net that predicts 521 audio classes, to build a system that can distinguish between a peaceful night, a heavy snorer, and a concerning cough. By the end of this post, you'll understand how to implement an end-to-end pipeline using TensorFlow Hub, Librosa, and Android/Kotlin.

The Architecture: How Audio AI Works 🏗️

Before we write a single line of code, let’s visualize how we transform raw sound waves into actionable health insights. Our system follows a classic Digital Signal Processing (DSP) to Inference pipeline.

graph TD
    A[Raw Audio Input .wav] --> B[Resampling & Normalization]
    B --> C[Feature Extraction: Mel Spectrograms]
    C --> D[YAMNet Pre-trained Model]
    D --> E{Transfer Learning Layer}
    E --> F[Class: Snore]
    E --> G[Class: Cough]
    E --> H[Class: Ambient Noise]
    F --> I[Android Dashboard / Risk Assessment]

Prerequisites 🛠️

To follow along, you'll need:

TensorFlow Hub: To access the pre-trained YAMNet weights.
Librosa: The Swiss Army knife for audio preprocessing in Python.
Android Studio: If you want to deploy this as a mobile health tracker using Kotlin.

Step 1: Preprocessing with Librosa 🎼

YAMNet expects audio sampled at exactly 16,000 Hz. Most phone microphones record at 44.1kHz or 48kHz, so resampling is our first hurdle.

import librosa
import numpy as np

def preprocess_audio(file_path):
    # Load audio file and resample to 16kHz
    audio, sr = librosa.load(file_path, sr=16000)

    # Normalize the audio to a range of [-1.0, 1.0]
    audio = librosa.util.normalize(audio)

    # Ensure it's mono channel (YAMNet requirement)
    if len(audio.shape) > 1:
        audio = np.mean(audio, axis=1)

    return audio

# Example usage
waveform = preprocess_audio("night_recording_001.wav")
print(f"Waveform shape: {waveform.shape}")

Step 2: Fine-Tuning YAMNet with TensorFlow Hub 🧠

YAMNet is great, but it’s trained on the YouTube-8M dataset. To make it a specialized medical tool, we use Transfer Learning. We freeze the early layers and train a new "head" to specifically recognize "Snoring" vs "Coughing."

import tensorflow as tf
import tensorflow_hub as hub

# Load the YAMNet model from TF Hub
model = hub.load('https://tfhub.dev/google/yamnet/1')

# Define a custom classifier head
def build_health_classifier(yamnet_model):
    inputs = tf.keras.layers.Input(shape=(16000,), dtype=tf.float32)
    # Get the embedding from YAMNet
    scores, embeddings, spectrogram = yamnet_model(inputs)

    # Add our custom layers
    x = tf.keras.layers.Dense(256, activation='relu')(embeddings)
    x = tf.keras.layers.Dropout(0.3)(x)
    outputs = tf.keras.layers.Dense(3, activation='softmax')(x) # Snore, Cough, Noise

    return tf.keras.Model(inputs, outputs)

health_model = build_health_classifier(model)
health_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

Step 3: Deploying to Android (Kotlin) 📱

Once we export our model to TFLite, we can run inference on-device. This is crucial for privacy—no one wants their bedroom recordings sent to a cloud server!

// Android/Kotlin snippet for TFLite Inference
class SnoreDetector(context: Context) {
    private var tflite: Interpreter? = null

    init {
        val model = FileUtil.loadMappedFile(context, "snore_model.tflite")
        tflite = Interpreter(model)
    }

    fun classifyAudio(audioData: FloatArray): String {
        val output = Array(1) { FloatArray(3) }
        tflite?.run(audioData, output)

        // Find index with max probability
        val labels = listOf("Snore", "Cough", "Ambient")
        val maxIndex = output[0].indices.maxByOrNull { output[0][it] } ?: -1
        return labels[maxIndex]
    }
}

Scaling for Production: The "Official" Way 🥑

Building a prototype is easy, but making it robust enough for a clinical setting requires advanced signal processing and data validation patterns.

If you are looking for advanced architectural patterns or want to see how this integrates into a full-scale healthcare backend, I highly recommend checking out the technical deep-dives at WellAlly Blog. They cover production-ready AI deployments and mobile health (mHealth) security standards that are essential if you plan to move beyond a local script.

Conclusion 🌙

Identifying sleep risks doesn't require a full sleep lab anymore. With YAMNet and TensorFlow, we can turn a standard smartphone into a powerful diagnostic tool. By focusing on local processing (Edge AI), we ensure user privacy while providing meaningful health data.

What's next for your project?

[ ] Add a "Sleep Cycle" graph based on audio intensity.
[ ] Integrate with Apple HealthKit or Google Fit.
[ ] Implement a low-pass filter to remove fan noise.

Have you tried building audio classifiers before? Let’s chat in the comments! 👇

Stop Losing Your Health Data! Build a Lifelong Electronic Health Record (EHR) System with Neo4j and GraphRAG 🏥💻

wellallyTech — Thu, 23 Apr 2026 01:10:00 +0000

Let’s be honest: our medical history is usually a chaotic mess of scattered PDFs, blurry smartphone photos of prescriptions, and "I think I had a fever in 2019" memories. When you're dealing with long-term health tracking, traditional search fails. You don't just need to find a keyword; you need to understand the relationship between a medication you took three years ago and a lab result from last week.

In this tutorial, we are going to solve this by building a Personal Lifelong EHR Analysis System. We will transform "dirty" unstructured medical reports into a structured Knowledge Graph using Neo4j and leverage GraphRAG (Graph Retrieval-Augmented Generation) to answer complex health queries with 100% traceability. 🚀

The Architecture: From Chaos to Context

To build a robust medical knowledge system, we need more than just a vector database. We need to preserve the relational nature of medical data. Here is how the data flows from a messy PDF to a structured response:

graph TD
    A[Unstructured PDF/Images] -->|Unstructured.io| B(Clean Text & Tables)
    B -->|LangChain + LLM| C{Entity & Relation Extraction}
    C -->|Cypher Queries| D[(Neo4j Graph Database)]
    D -->|LlamaIndex GraphStore| E[GraphRAG Engine]
    F[User Query: 'How has my fasting blood sugar trended?'] --> E
    E -->|Contextual Retrieval| G[LLM Final Answer + Source Citation]

Prerequisites

Before we dive in, make sure you have the following tools in your kit:

Neo4j: Our graph database (AuraDB is great for a quick start).
LangChain: For orchestrating the extraction chain.
Unstructured.io: For parsing those pesky medical PDFs.
LlamaIndex: To implement the GraphRAG retrieval logic.

Step 1: Parsing the "Dirty" Data

Medical reports are notorious for having complex layouts—tables, multi-column text, and signatures. We'll use unstructured to handle the heavy lifting.

from unstructured.partition.pdf import partition_pdf

# Extract elements from a medical report PDF
elements = partition_pdf(
    filename="report_2023_checkup.pdf",
    infer_table_structure=True,
    strategy="hi_res",
)

# Filter for tables and text
raw_text = "\n".join([str(el) for el in elements])
print(f"Successfully extracted {len(raw_text)} characters from the report.")

Step 2: Defining the Medical Schema

A Knowledge Graph is only as good as its schema. For EHR, we want to capture entities like Patient, Condition, Medication, and LabResult.

Using LangChain and an LLM (like GPT-4o), we can extract these nodes and their relationships (e.g., PATIENT -> DIAGNOSED_WITH -> CONDITION).

from langchain_community.graphs import Neo4jGraph
from langchain_experimental.graph_transformers import LLMGraphTransformer
from langchain_openai import ChatOpenAI

# Initialize the Graph Transformer
llm = ChatOpenAI(temperature=0, model="gpt-4o")
transformer = LLMGraphTransformer(
    llm=llm,
    allowed_nodes=["Patient", "Condition", "Medication", "LabResult", "Date"],
    allowed_relationships=["HAS_CONDITION", "PRESCRIBED", "RESULTS_IN", "OCCURRED_ON"]
)

# Convert text to graph documents
graph_documents = transformer.convert_to_graph_documents(documents)

# Push to Neo4j
graph = Neo4jGraph()
graph.add_graph_documents(graph_documents)

Step 3: Implementing GraphRAG for Deep Insights

Traditional RAG might find a "Lab Result" chunk, but GraphRAG allows us to traverse the graph. If you ask, "How did my medication change after my 2022 blood work?", the system follows the path: LabResult -> Date -> Condition -> Medication.

We use LlamaIndex to create a query engine over our Neo4j instance.

from llama_index.core import PropertyGraphIndex
from llama_index.graph_stores.neo4j import Neo4jPropertyGraphStore

# Link LlamaIndex to our existing Neo4j DB
graph_store = Neo4jPropertyGraphStore(
    username="neo4j",
    password="your_password",
    url="bolt://localhost:7687"
)

index = PropertyGraphIndex.from_existing(
    property_graph_store=graph_store,
    llm=llm
)

query_engine = index.as_query_engine(include_text=True)
response = query_engine.query("Analyze the correlation between my Vitamin D levels and energy complaints.")
print(response)

🥑 Pro-Tip: The "Official" Way to Production

Building a prototype is easy, but handling medical data in production requires strict adherence to data privacy and more complex entity resolution (ensuring "Vitamin D" and "Vit D3" are mapped to the same node).

For more advanced patterns in healthcare AI, complex entity linking strategies, and production-ready RAG architectures, I highly recommend checking out the technical deep dives at WellAlly Blog. It's a goldmine for developers looking to move beyond "Hello World" in the medical AI space.

Why this matters?

By moving from a Vector-only approach to GraphRAG, you gain:

Explainability: You can literally see the nodes and edges that led to an answer.
Long-term Memory: The graph naturally links a record from 10 years ago to today if they share the same Condition node.
Data Integrity: No more hallucinating lab values—the LLM reads directly from the structured graph properties.

Conclusion

We’ve just turned a pile of messy PDFs into a high-functioning, structured medical brain. Using Neo4j for storage and LlamaIndex for GraphRAG, you can now query your health history like a pro.

Are you building something in the Medical AI space? Let's chat in the comments! And don't forget to star the repo if this helped you. 🌟

Your Health Data is Yours: Build a Fully Local AI Health Assistant with Llama 3 and MLX 🍏💻

wellallyTech — Wed, 22 Apr 2026 01:09:00 +0000

Privacy isn't just a feature anymore; it’s a human right. As we integrate AI deeper into our lives, the thought of sending heart rate variability, sleep cycles, and activity levels to a cloud server feels... invasive. But what if you could have the reasoning power of a world-class LLM living entirely on your MacBook?

In this tutorial, we are building a Privacy-First Health Predictor. We’ll extract data from Apple HealthKit, process it locally, and run inference using Llama 3 via the MLX framework—Apple’s powerhouse library for machine learning on Silicon. This is Edge AI at its finest: zero latency, zero internet required, and zero data leaks.

The Architecture: From Pulse to Prediction

To achieve a 0-leak architecture, we need a seamless bridge between the iOS/macOS sandbox and the MLX environment. Here is how the data flows:

graph TD
    A[Apple HealthKit] -->|Swift / HKQuery| B(Local CSV/JSON Export)
    B --> C{Data Preprocessing}
    C -->|Python/Pandas| D[Llama 3 MLX Model]
    D --> E[LoRA Fine-Tuning / RAG]
    E --> F[Local Health Insights]
    subgraph MacBook Pro - Apple Silicon
    D
    E
    F
    end
    style D fill:#f96,stroke:#333,stroke-width:2px

Prerequisites

Before we dive into the code, ensure you have:

A MacBook with Apple Silicon (M1/M2/M3).
Xcode installed (for HealthKit data extraction).
Python 3.10+ and the MLX library (pip install mlx-lm).

Step 1: Extracting the "Gold" from HealthKit

Apple Health data is strictly guarded. To use it, we must first request permission and query the HKHealthStore. Here is a Swift snippet to get you started on extracting step counts or heart rates.

import HealthKit

let healthStore = HKHealthStore()

func fetchStepCount(completion: @escaping (Double) -> Void) {
    let stepsQuantityType = HKQuantityType.quantityType(forIdentifier: .stepCount)!
    let now = Date()
    let startOfDay = Calendar.current.startOfDay(for: now)
    let predicate = HKQuery.predicateForSamples(withStart: startOfDay, end: now, options: .strictStartDate)

    let query = HKStatisticsQuery(quantityType: stepsQuantityType, quantitySamplePredicate: predicate, options: .cumulativeSum) { _, result, _ in
        guard let result = result, let sum = result.sumQuantity() else {
            completion(0.0)
            return
        }
        completion(sum.doubleValue(for: HKUnit.count()))
    }
    healthStore.execute(query)
}

Pro-tip: For a real-world app, you'd export this to a local JSON file that our Python script can consume.

Step 2: Preparing Llama 3 with MLX

The MLX framework allows us to leverage the Unified Memory Architecture of Apple Silicon. This means the GPU and CPU share the same memory pool, making it incredibly efficient to run 8B or even 70B parameter models like Llama 3.

First, let's install the model from Hugging Face:

python -m mlx_lm.generate --model mlx-community/Meta-Llama-3-8B-Instruct-4bit --prompt "hello"

Step 3: Local Inference & Health Reasoning

Now, let's pipe our HealthKit data into the model. We aren't just doing "keyword searches." We are asking the model to look for patterns—like how a drop in sleep hours correlates with an increased resting heart rate.

from mlx_lm import load, generate

# Load the quantized Llama 3 model
model, tokenizer = load("mlx-community/Meta-Llama-3-8B-Instruct-4bit")

# Simulated HealthKit data extracted from Swift
health_data = {
    "avg_heart_rate": 72,
    "sleep_hours": 5.5,
    "activity_level": "Low",
    "recent_stress_score": 8/10
}

prompt = f"""
[INST] You are a private health assistant. Analyze this data: {health_data}. 
Provide a concise health insight. Stay objective. [/INST]
"""

response = generate(model, tokenizer, prompt=prompt, verbose=True)
print(f"Personal Health Insight: {response}")

Why this works

Because MLX uses the AMX (Apple Matrix) co-processor, the inference above happens in milliseconds without heating up your laptop. 🥑

The "Official" Way: Elevating Your AI Strategy 🚀

While building local experiments is fun, scaling private AI for production requires a deeper understanding of data sanitization and model quantization.

If you're looking for more production-ready examples and advanced architectural patterns for Edge AI, I highly recommend checking out the technical deep-dives at WellAlly Blog (https://www.wellally.tech/blog). They have fantastic resources on how to handle sensitive biometric data and optimize model weights for mobile environments that go far beyond this introductory guide.

Step 4: Fine-Tuning for Personalized Health (LoRA)

If you want the model to sound more like a doctor or understand specific biometric trends (like glucose monitoring), you can use LoRA (Low-Rank Adaptation). MLX makes this trivial:

python -m mlx_lm.lora \
  --model mlx-community/Meta-Llama-3-8B-Instruct-4bit \
  --data ./my_health_data_jsonl/ \
  --train \
  --iters 500

This creates a small "adapter" file (~50MB) that sits on top of the base Llama 3 model, giving it specialized knowledge of your health history without ever touching the cloud.

Conclusion: The Future is Local

By combining Apple HealthKit with the MLX framework, we’ve built a system that respects the most sensitive data a human has. No more wondering if your insurance company is scraping your AI chats.

Summary of benefits:

Latency: Sub-second responses.
Privacy: Data never leaves the disk.
Cost: $0 in API tokens.

Have you tried running Llama 3 on your Mac yet? What’s your experience with MLX? Let's discuss in the comments below! 👇

Building a Closed-Loop Health Agent with LangGraph: Automatically Manage Your Blood Sugar via Notion & CGM 🚀🥗

wellallyTech — Tue, 21 Apr 2026 01:15:00 +0000

We’ve all been there: you treat yourself to a massive pasta bowl, and an hour later, your Continuous Glucose Monitor (CGM) starts screaming. Usually, you’d just feel guilty and sluggish. But what if your calendar actually reacted to your biology?

In this tutorial, we are building a Stateful AI Health Agent using LangGraph, the Dexcom API, and the Notion API. This agent doesn't just watch your data; it takes action. When a blood sugar spike is detected, it triggers a "Closed-Loop" workflow: analyzing the cause, fetching low-glycemic index (Low-GI) alternatives, and dynamically updating your Notion meal plan for the next 24 hours.

If you're interested in LangGraph orchestration, stateful AI agents, and biometric data automation, you're in the right place. Let's turn those glucose spikes into actionable insights! 🩸💻

The Architecture: The Closed-Loop Logic 🏗️

Standard LLM chains are linear. But health is iterative. We need a system that can maintain state, check conditions, and persist data using Redis for long-term memory.

Here is how our Agentic workflow looks:

graph TD
    A[Start: Dexcom Sync] --> B{Check Glucose Spike}
    B -- No Spike --> C[Log & Sleep]
    B -- Spike Detected! --> D[Analyze Context]
    D --> E[Query Low-GI Alternatives]
    E --> F[Update Notion Schedule]
    F --> G[Notify User via Redis State]
    G --> A

    style B fill:#f96,stroke:#333,stroke-width:2px
    style F fill:#00ff00,stroke:#333,stroke-width:2px

Prerequisites 🛠️

To follow this advanced guide, you'll need:

LangGraph & LangChain: For agent orchestration.
Dexcom Developer Account: To fetch real-time CGM data.
Notion API: To manage your meal database and schedule.
Redis: For state checkpointing (keeping the agent's memory alive across sessions).
OpenAI GPT-4o: Our reasoning engine for dietary suggestions.

Step 1: Defining the Agent State 📝

In LangGraph, everything revolves around the State. We need to track the current glucose levels, the spike history, and the pending suggestions.

from typing import Annotated, TypedDict, List
from langgraph.graph import StateGraph, END

class HealthAgentState(TypedDict):
    current_glucose: float
    is_spike: bool
    glucose_history: List[float]
    dietary_suggestions: str
    notion_page_id: str
    status_message: str

Step 2: The Logic Nodes 🧠

Now, let's build the functional components. First, we need a node to check the Dexcom API for current trends.

Node: Monitoring the Spike

import requests

def monitor_cgm(state: HealthAgentState):
    # Simulated Dexcom API Call
    # In production: requests.get(DEXCOM_URL, headers=headers)
    latest_reading = 185.0  # mg/dL (A bit high!)

    spike_detected = latest_reading > 160.0

    return {
        "current_glucose": latest_reading,
        "is_spike": spike_detected,
        "glucose_history": state.get("glucose_history", []) + [latest_reading]
    }

Node: Generating Low-GI Alternatives

If a spike is detected, our LLM (GPT-4o) acts as a nutritionist.

from langchain_openai import ChatOpenAI

def get_dietary_alternatives(state: HealthAgentState):
    llm = ChatOpenAI(model="gpt-4o")
    prompt = f"""
    The user's blood sugar is currently {state['current_glucose']} mg/dL. 
    They had a high-carb meal. Suggest 3 low-GI snacks or dinner alternatives 
    to help stabilize their levels. Format as a Notion-ready markdown list.
    """
    response = llm.invoke(prompt)

    return {"dietary_suggestions": response.content}

Step 3: Integrating with Notion 📓

The "Closed-Loop" is completed when the agent modifies the environment. We use the Notion API to append these suggestions directly into the user's "Daily Planner."

def update_notion_plan(state: HealthAgentState):
    notion_token = "your_secret_here"
    page_id = state["notion_page_id"]

    # Logic to append a block to a Notion page
    # Using the Notion SDK or raw requests
    headers = {"Authorization": f"Bearer {notion_token}", "Content-Type": "application/json"}
    payload = {
        "children": [
            {
                "object": "block",
                "type": "heading_2",
                "heading_2": {"rich_text": [{"type": "text", "text": {"content": "🚨 Glucose Alert Action Plan"}}]}
            },
            {
                "object": "block",
                "type": "paragraph",
                "paragraph": {"rich_text": [{"type": "text", "text": {"content": state['dietary_suggestions']}}]}
            }
        ]
    }
    # requests.patch(f"https://api.notion.com/v1/blocks/{page_id}/children", json=payload, headers=headers)

    return {"status_message": "Notion updated with low-GI alternatives."}

Step 4: Compiling the Graph with Redis Persistence 💾

To ensure our agent doesn't "forget" where it is if the server restarts, we use Redis Checkpointing.

Pro Tip: For more production-ready patterns regarding agent persistence and multi-user health state management, definitely check out the deep-dive articles at WellAlly Tech Blog. They cover scaling LangGraph apps in much more detail! 🥑

from langgraph.checkpoint.redis import RedisCheckpointSaver

# Setup Redis connection
with RedisCheckpointSaver.from_conn_info(host="localhost", port=6379, db=0) as checkpointer:

    workflow = StateGraph(HealthAgentState)

    # Add Nodes
    workflow.add_node("monitor", monitor_cgm)
    workflow.add_node("suggest", get_dietary_alternatives)
    workflow.add_node("update_notion", update_notion_plan)

    # Define Edges
    workflow.set_entry_point("monitor")

    # Conditional Routing
    workflow.add_conditional_edges(
        "monitor",
        lambda state: "suggest" if state["is_spike"] else END
    )

    workflow.add_edge("suggest", "update_notion")
    workflow.add_edge("update_notion", END)

    # Compile with persistence
    app = workflow.compile(checkpointer=checkpointer)

Why This Matters: The Power of Stateful Agents 🔋

Traditional "If-This-Then-That" (IFTTT) automations are too brittle for health. A spike while you're sleeping is different from a spike while you're at the gym. By using LangGraph, we can add a "Context Node" that checks your Oura Ring or Apple Watch data to see if you're exercising before suggesting a diet change.

By persisting the state in Redis, the agent remembers that it already warned you 30 minutes ago, preventing notification fatigue.

Conclusion: Taking Control of Your Bio-Data 🚀

We’ve just built a system that:

Listens to your body (Dexcom).
Thinks about the solution (GPT-4o).
Acts on your schedule (Notion).
Remembers everything (Redis).

This is the future of personalized medicine—where AI agents act as the connective tissue between our biometric sensors and our daily lives.

What's next?

Add a Twilio node to text you if the spike lasts more than 2 hours.
Integrate Google Fit to correlate spikes with sedentary behavior.

For more advanced AI Agent patterns and production-grade implementation guides, visit WellAlly Tech.

Happy hacking, and stay healthy! 🥑💪

Found this useful? Drop a comment below or follow for more "Learning in Public" AI tutorials! ✌️