WhatsApp Expense Bot

This guide covers building a WhatsApp-first expense tracker and financial coach using Ozzie and Evolution API. Users simply send text messages or photos of receipts to a WhatsApp number — your backend parses the input using ozzie-openapi, stores the results, and replies with a personalized response in the user's language.

How it works

User sends WhatsApp message
        |
        v
Evolution API instance
        |
        | POST /api/webhooks/evolution
        v
Your Backend (webhook handler)
        |
        |── Is this user registered?
        |     No → Send onboarding prompts, collect info, call POST /v1/users
        |     Yes → Forward to ozzie-openapi chat/transaction processing
        v
Your Backend calls ozzie-openapi and generates reply
        |
        v
Your Backend sends reply back via Evolution API
        |
        v
User receives reply on WhatsApp

Prerequisites

Evolution API instance running and connected to a WhatsApp number (Evolution API docs)
Ozzie API credentials (client_id and client_secret)
A backend server reachable from the internet with HTTPS (use ngrok for local development)
EVOLUTION_WEBHOOK_SECRET set to the same value in both your environment and your Evolution API instance config

Setup steps

1. Deploy your webhook server

Your server needs a public HTTPS endpoint at /api/webhooks/evolution. Here's a minimal Express setup:

// server.js
import express from 'express';
import { handleEvolutionWebhook } from './webhooks/evolution.js';

const app = express();
app.use(express.json({ limit: '10mb' })); // Allow larger payloads for image messages

app.post('/api/webhooks/evolution', handleEvolutionWebhook);

app.listen(3000, () => console.log('Webhook server running on port 3000'));

2. Configure Evolution API

In your Evolution API dashboard, set the webhook URL for your instance:

https://your-server.com/api/webhooks/evolution

Set the apikey (webhook secret) to match your EVOLUTION_WEBHOOK_SECRET.

Subscribe to the messages.upsert event.

3. Set environment variables

# .env
OZZIE_CLIENT_ID=ozz_client_a1b2c3d4
OZZIE_CLIENT_SECRET=sk_live_xK9mP2qR7tL...
EVOLUTION_WEBHOOK_SECRET=a7f3c8e1b2d4e5f60718293a4b5c6d7e8f901234
EVOLUTION_API_URL=https://your-evolution-instance.com
EVOLUTION_INSTANCE=your_instance_name

4. Implement the Ozzie client

// lib/ozzie.js
const BASE_URL = 'https://api.ozzieapp.com/v1';

const token = Buffer.from(
  `${process.env.OZZIE_CLIENT_ID}:${process.env.OZZIE_CLIENT_SECRET}`
).toString('base64');

const headers = {
  'Authorization': `Bearer ${token}`,
  'Content-Type': 'application/json',
};

export async function ozzieRequest(method, path, body = null) {
  const response = await fetch(`${BASE_URL}${path}`, {
    method,
    headers,
    body: body ? JSON.stringify(body) : undefined,
  });
  const json = await response.json();
  if (!response.ok) throw new Error(`[${json.error?.code}] ${json.error?.message}`);
  return json.data;
}

5. Implement the webhook handler

// webhooks/evolution.js

import { findUserByPhone, createUser } from '../db/users.js';
import { ozzieRequest } from '../lib/ozzie.js';
import { sendWhatsAppMessage } from '../lib/evolution.js';

export async function handleEvolutionWebhook(req, res) {
  // Verify the request is from Evolution API
  const apiKey = req.headers['apikey'];
  if (apiKey !== process.env.EVOLUTION_WEBHOOK_SECRET) {
    return res.status(401).json({ error: 'Unauthorized' });
  }

  const { event, data } = req.body;

  // Only process incoming user messages
  if (event !== 'messages.upsert' || data?.key?.fromMe) {
    return res.sendStatus(200);
  }

  const { remoteJid, id: messageId } = data.key;
  const phoneRaw = remoteJid.replace('@s.whatsapp.net', '');
  const phone = `+${phoneRaw}`;
  const messageType = data.messageType;

  // Find the user by phone number in your database
  let user = await findUserByPhone(phone);

  if (!user) {
    // User not registered — start onboarding flow
    await handleNewUser(phone, data);
    return res.sendStatus(200);
  }

  // Process the message based on type
  if (messageType === 'conversation') {
    const text = data.message?.conversation;
    await handleTextMessage(user, phone, text);
  } else if (messageType === 'imageMessage') {
    const mimeType = data.message?.imageMessage?.mimetype;
    await handleImageMessage(user, phone, data.key, mimeType);
  } else {
    await sendWhatsAppMessage(phone, "Sorry, I can only process text messages and images right now.");
  }

  res.sendStatus(200);
}

6. Implement the Evolution API sender

// lib/evolution.js

export async function sendWhatsAppMessage(phone, text) {
  const phoneDigits = phone.replace('+', '');

  const response = await fetch(
    `${process.env.EVOLUTION_API_URL}/message/sendText/${process.env.EVOLUTION_INSTANCE}`,
    {
      method: 'POST',
      headers: {
        'apikey': process.env.EVOLUTION_WEBHOOK_SECRET,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({
        number: phoneDigits,
        text,
      }),
    }
  );

  if (!response.ok) {
    throw new Error(`Failed to send WhatsApp message: ${response.status}`);
  }
}

Supported input types

Text messages

Users can send natural language descriptions of expenses, questions, or commands:

What user types	How your backend handles it
`spent $45 on groceries`	Calls `POST /v1/transactions/parse`, stores result
`paid $120 electricity bill`	Calls `POST /v1/transactions/parse`, stores result
`how am I doing this month?`	Calls `POST /v1/users/:id/chat/completion`, sends reply
`what's my savings goal?`	Calls `POST /v1/users/:id/chat/completion`, sends reply
`I got a $500 bonus at work`	Calls `POST /v1/users/:id/chat/completion`, sends reply

Receipt photos

Users can photograph a receipt and send it directly to WhatsApp. Your backend fetches the base64 image from Evolution API and submits it to POST /v1/transactions/parse with type: "image".

Sample conversation flow

User: Hi, I want to track my expenses

Bot: Hey! I'm Ozzie, your personal money coach. I can help you track spending and reach your savings goals — right here on WhatsApp. What's your name?

User: Sofia

Bot: Nice to meet you, Sofia! What's your monthly take-home income (after taxes)?

User: About $3,500

Bot: Got it. And roughly how much do you spend on fixed expenses each month?

User: Around $1,600

Bot: Perfect. Last question: what's your main financial goal — saving money or paying off debt?

User: Saving. I want to build an emergency fund.

Bot: Great goal, Sofia! I've set up your account and built your first financial plan. Based on your income and expenses, I recommend saving $262 per month. At that pace, you'd have a $3,000 starter emergency fund in about 11 months.

A few days later:

User: [sends a photo of a grocery receipt]

Bot: Got it! I parsed your receipt from Walmart: $67.82 logged under Groceries for May 7th.

Onboarding via WhatsApp

When an unknown number sends a message, guide them through registration:

// webhooks/evolution.js (onboarding portion)

const ONBOARDING_SESSIONS = new Map(); // In production, use Redis or your DB

async function handleNewUser(phone, data) {
  const text = data.message?.conversation?.trim();
  const session = ONBOARDING_SESSIONS.get(phone) ?? { step: 'start' };

  if (session.step === 'start') {
    await sendWhatsAppMessage(
      phone,
      "Hey! I'm Ozzie, your personal money coach. What's your name?"
    );
    ONBOARDING_SESSIONS.set(phone, { step: 'name' });
    return;
  }

  if (session.step === 'name') {
    session.name = text;
    await sendWhatsAppMessage(
      phone,
      `Nice to meet you, ${text}! What's your monthly take-home income? Just send the number, like: 3500`
    );
    ONBOARDING_SESSIONS.set(phone, { ...session, step: 'income' });
    return;
  }

  if (session.step === 'income') {
    const income = parseFloat(text?.replace(/[^0-9.]/g, ''));
    if (isNaN(income)) {
      await sendWhatsAppMessage(phone, "Please send just the number — like: 3500");
      return;
    }
    session.income = income;
    await sendWhatsAppMessage(
      phone,
      `Got it — $${income.toFixed(0)}/month. What's your main financial goal?\n\n1 Save money\n2 Pay off debt`
    );
    ONBOARDING_SESSIONS.set(phone, { ...session, step: 'goal' });
    return;
  }

  if (session.step === 'goal') {
    const isDebt = text?.includes('2') || text?.toLowerCase().includes('debt');
    session.goal = isDebt ? 'debt' : 'savings';

    try {
      // Create the Ozzie user
      const ozzieUser = await ozzieRequest('POST', '/users', {
        external_user_id: `whatsapp_${phone.replace('+', '')}`,
        name: session.name,
        language: 'en',
      });

      // Submit financial intake
      await ozzieRequest('POST', `/users/${ozzieUser.id}/financial-intake`, {
        monthly_income: session.income,
        monthly_expenses: session.income * 0.45,
        financial_goal: session.goal,
      });

      // Score a default personality (or ask the quiz — see Personality guide)
      const score = await ozzieRequest('POST', '/personality/score', {
        answers: [{ question_id: 'q_001', option_id: 'q_001_b' }],
      });
      const personalityType = score.personality_type;

      // Generate plan
      const plan = await ozzieRequest('POST', `/users/${ozzieUser.id}/plan/generate`, {
        personality_type: personalityType,
      });

      // Store user + personality + plan_tier in your database
      await createUser({
        phone,
        name: session.name,
        ozzie_user_id: ozzieUser.id,
        personality_type: personalityType,
        plan_tier: plan.plan_tier,
      });

      ONBOARDING_SESSIONS.delete(phone);

      await sendWhatsAppMessage(
        phone,
        `You're all set, ${session.name}! I've built your first financial plan. ` +
        `Send me expenses anytime — like "spent $45 on groceries" — or ask how you're doing.`
      );
    } catch (err) {
      console.error('Onboarding error:', err);
      await sendWhatsAppMessage(phone, "Sorry, something went wrong. Please try again in a moment.");
      ONBOARDING_SESSIONS.delete(phone);
    }
  }
}

Handling text and image messages

async function handleTextMessage(user, phone, text) {
  if (!text) return;

  try {
    // Load recent conversation history from your database
    const history = await db.messages.findRecent(user.id, 10);

    const reply = await ozzieRequest(
      'POST',
      `/users/${user.ozzie_user_id}/chat/completion`,
      {
        message: text,
        personality_type: user.personality_type,
        history,
      }
    );

    // Store both the user message and reply in your database
    await db.messages.create({ user_id: user.id, role: 'user', content: text });
    await db.messages.create({ user_id: user.id, role: 'assistant', content: reply.content });

    await sendWhatsAppMessage(phone, reply.content);
  } catch (err) {
    await sendWhatsAppMessage(phone, "I had trouble processing that. Can you try again?");
  }
}

async function handleImageMessage(user, phone, messageKey, mimeType) {
  try {
    // Download the image via Evolution API
    const mediaRes = await fetch(
      `${process.env.EVOLUTION_API_URL}/chat/getBase64FromMediaMessage/${process.env.EVOLUTION_INSTANCE}`,
      {
        method: 'POST',
        headers: { 'apikey': process.env.EVOLUTION_WEBHOOK_SECRET, 'Content-Type': 'application/json' },
        body: JSON.stringify({ message: { key: messageKey } }),
      }
    );
    const { base64, mimetype } = await mediaRes.json();

    const result = await ozzieRequest('POST', '/transactions/parse', {
      type: 'image',
      content: base64,
      mime_type: mimetype ?? mimeType ?? 'image/jpeg',
    });

    const txs   = result.transactions ?? [];
    const total = txs.reduce((s, t) => s + t.amount_cents, 0);

    // Store parsed transactions in your database
    for (const tx of txs) {
      await db.transactions.create({ user_id: user.id, ...tx });
    }

    await sendWhatsAppMessage(
      phone,
      `Got it! Logged ${txs.length} transaction${txs.length !== 1 ? 's' : ''} — $${(total / 100).toFixed(2)} under ${txs[0]?.envelope ?? txs[0]?.category}.`
    );
  } catch (err) {
    await sendWhatsAppMessage(
      phone,
      "I couldn't read that receipt clearly. Try taking the photo in better lighting, or type the expense manually."
    );
  }
}

Error handling

Scenario	What to send to the user
Unknown phone number	Trigger onboarding flow
Image unreadable	Ask for better photo or manual entry
`RATE_LIMIT_EXCEEDED`	"I'm a bit busy right now — try again in a minute"
`INTERNAL_ERROR`	"Something went wrong on my end. Give me a moment and try again."

warning

Never send raw error messages or stack traces to users via WhatsApp. Always translate API errors into friendly, actionable messages in the user's language.

How it works​

Prerequisites​

Setup steps​

1. Deploy your webhook server​

2. Configure Evolution API​

3. Set environment variables​

4. Implement the Ozzie client​

5. Implement the webhook handler​

6. Implement the Evolution API sender​

Supported input types​

Text messages​

Receipt photos​

Sample conversation flow​

Onboarding via WhatsApp​

Handling text and image messages​

Error handling​