Lead Management — Implementation Spec

Overview

This document defines the implementation plan to introduce first-class Leads into HoneyGrid with lifecycle states, automation, conversations, matchback, and conversion reporting to ad platforms. It maps code changes across API, Durable Objects, types, and the web-app, and breaks delivery into milestones.

Goals and KPIs

• Acceptance: Leads created in HoneyGrid; Lead interactions recorded
• Correlated: Average time to contact
• Performance: Qualified rate, Converted rate
• Insight: Loss reasons, Unqualified reasons
• Core: CPL, CAC

Solution Summary

• Add a workspace-scoped Durable Object LeadManager to store leads, appointments, interactions, timers, and compute metrics
• Expose Hono routes under /leads for CRUD, state transitions, and metrics
• Wire integrations to advance state automatically (CallRail, email notifications, matchback)
• Report lifecycle conversions to Google and Meta. Google conversions should be reported via the GoogleAdsService. We need to create a new MetaAdsService
• Add Leads list, details, actions, and metrics UI in the web-app

---

Detailed Implementation

1) Data model and storage (Durable Object)

• New DO: apps/api/src/durable-objects/LeadManager.ts
  • Storage via this.ctx.storage.sql with migrations (SQLSchemaMigrations)
  • Database interactions, schema, and migrations should be managed via drizzle
  • Tables (all implicitly scoped to the DO instance for a workspace_id):
    • leads: id (uuid), workspace_id, created_at, created_by, source (web|phone|email|fb_msg|import|other), status (new|qualified|scheduled|completed|lost), loss_reason (nullable), unqualified_reason (nullable), contacted_at, qualified_at, scheduled_at, completed_at, auto_close_alarm_at, display_name, phone, email, notes, platform_ref (json), attributes (json)
    • appointments: id (uuid), lead_id, external_ref (json), scheduled_for (datetime), status (scheduled|attended|no_show|canceled), matched_via (matchback|email|manual), created_at
    • lead_interactions: id (uuid), lead_id, type (call_inbound|call_outbound|sms|email|fb_msg|note|status_change|system_auto), payload (json), at (datetime)
    • conversations: id (uuid), lead_id, channel (sms|email|fb_msg), direction (inbound|outbound), body, meta (json), at (datetime)
  • Index strategy: by status, by auto_close_alarm_at; implicit partitioning by DO key (workspace)
  • Alarm logic:
    • New leads with no contact after 30 days → status lost; loss_reason=No contact; unqualified_reason=No contact
    • Qualified leads with no activity 30 days → status lost; loss_reason=Lost contact
  • Methods:
    • CRUD: createLead, getLead, listLeads(filters), updateLead, deleteLead
    • Transitions: markContacted, markQualified, scheduleAppointment, completeFirstVisit, closeLost({ loss_reason, unqualified_reason })
    • Appointments: addAppointment, linkAppointmentByExternalRef, autoMatchFromSignals
    • Interactions: logInteraction
    • Metrics: timeToContact, timeToQualify, timeToSchedule, funnel counts, loss reason breakdown
    • alarm() handler: sweep pending auto transitions

2) Types and validation

• New: packages/honeygrid-types/validators-and-types/LeadTypes.ts
  • Zod schemas and TS types: Lead, LeadCreate, Appointment, Interaction, enums for status and reasons
  • Export from packages/honeygrid-types/index.ts
  • Optional generators: extend packages/honeygrid-types/generators.ts with generateLead/generateInteraction

3) API routes (Hono)

• New router: apps/api/src/routes/leads.ts
  • GET / — list (filters: workspaceId, status, q, from, to)
  • GET /:id — details (lead + appointments + interactions)
  • POST / — create lead
  • PUT /:id — update lead fields
  • POST /:id/contacted — mark contacted + log interaction
  • POST /:id/qualify — move to qualified
  • POST /:id/schedule — attach appointment (required) and set scheduled
  • POST /:id/complete — mark completed (first visit)
  • POST /:id/lost — close lost with reasons
  • GET /metrics — KPIs and aggregates for workspace
• Register route in apps/api/src/index.ts: app.route('/leads', leadsRouter)
• Permissions: extend apps/api/src/middleware/workspacePermissions.ts to authorize lead actions

4) Services and automation

• New service: apps/api/src/rpc-services/LeadService.ts
  • WorkerEntrypoint<Env> wrapper around DO stub; centralizes validation, analytics writes, and error handling
• CallRail linkage: edit apps/api/src/routes/callRail.ts
  • On answered/recorded calls: upsert by phone, set contacted, optionally set qualified/scheduled from transcript/tags; log interaction payloads
• Email scheduling: new apps/api/src/routes/emailIngest.ts
  • Parse provider webhooks; schedule via LeadService with matched_via='email'
• Matchback: update existing workflow(s) to call LeadService.linkAppointmentByExternalRef to set scheduled/completed
  • Either edit apps/api/src/workflows/create-data-source-version.ts or add apps/api/src/workflows/lead-matchback.ts

5) Conversion reporting to ad platforms

• New service: apps/api/src/rpc-services/ConversionReportingService.ts
  • Google Ads: upload offline conversions on status transitions (qualified, scheduled, completed) using connection credentials
  • Meta: Conversions API for equivalent events
• Extend apps/api/src/rpc-services/GoogleAdsService.ts with helpers to upload lifecycle conversions
• Optional admin re-sync endpoint in apps/api/src/routes/googleAds.ts for backfills

6) Frontend (web-app)

• Navigation: ensure apps/web-app/src/routes/routes.ts has Routes.Leads = '/leads'; add nav item in apps/web-app/src/components/ConnectionsLeftNav/ConnectionsLeftNav.tsx
• Settings consolidation: single page at /settings using existing user menu subnav styles (indented subnav with scrollspy)
  • Workspace (top): Users, Locations, Data Sources (moved), Workspace Billing
  • Account (below): Profile, Auth, Connections (moved), Billing Profiles
  • Legacy routes redirect into anchors on /settings
• Leads list page
  • apps/web-app/src/routes/Leads/LeadsListPage.tsx
  • apps/web-app/src/components/Leads/LeadsTable.tsx
  • apps/web-app/src/components/Leads/LeadFilters.tsx
• Lead details page
  • apps/web-app/src/routes/Leads/LeadDetailsPage.tsx
  • apps/web-app/src/components/Leads/LeadHeader.tsx
  • apps/web-app/src/components/Leads/LeadTimeline.tsx
  • apps/web-app/src/components/Leads/LeadActions.tsx
  • apps/web-app/src/components/Leads/AppointmentCard.tsx
• Metrics dashboard
  • apps/web-app/src/routes/Leads/LeadsMetricsPage.tsx
  • apps/web-app/src/components/Leads/MetricsSummary.tsx
  • apps/web-app/src/components/Leads/LossReasonsChart.tsx
• React Query hooks under apps/web-app/src/queries/leads/
  • useLeads.ts, useLead.ts, useCreateLead.ts, useUpdateLead.ts, useLeadActions.ts, useLeadMetrics.ts
• Conversation UI
  • apps/web-app/src/components/Leads/Conversation.tsx — display interaction history; outbound send can be added later

7) Config and bindings

• Edit apps/api/wrangler.jsonc
  • Durable Objects: add binding and migration for LeadManager
  • Analytics Engine: add LEAD_EVENTS dataset binding (or reuse an existing dataset)
  • Optional scheduled triggers (backstop sweep); primary timers via DO alarm()
  • Compatibility: compatibility_date = "2025-02-11", compatibility_flags = ["nodejs_compat"]
  • Observability: observability.enabled = true, head_sampling_rate = 1
• Edit apps/api/env.d.ts
  • Add types for DO binding, analytics dataset, provider secrets/IDs for Google and Meta, email webhook secret

8) Security and validation

• Validate requests using Zod schemas in LeadTypes.ts
• Sanitize free text (notes, message body)
• Enforce workspace authorization for all lead endpoints
• Consistent CORS handling for new routes

9) Observability and KPIs

• Emit analytics data points on lifecycle events: lead_created, lead_contacted, lead_qualified, lead_scheduled, lead_completed, lead_lost
• Include workspace_id, lead_id, timestamps, durations, and reasons
• /leads/metrics computes KPIs from DO storage for UI

10) Tests

• API: apps/api/tests/leadsRoutes.test.ts, apps/api/tests/leadService.test.ts
• CallRail integration: extend apps/api/tests/callRailService.test.ts
• Web-app: basic UI and hooks tests in apps/web-app/src/test-utils/
• Fixtures: apps/api/tests/fixtures/leads.json

11) Backward compatibility

• Leads live in a new DO; no D1 schema changes required
• Existing routes and services remain unchanged

---

Milestones

Milestone 1 — Foundation (DO, Types, Basic API, UI List/Details, Settings Shell)

• Implement LeadManager DO with schema and CRUD
• Add LeadTypes.ts and exports; add basic generators
• Add /leads routes: list, get, create, update
• Add web-app pages for Leads list and Lead details; basic React Query hooks
• Create /settings route and subnav shell (reusing current user menu subnav styles) to prepare for settings consolidation
• Add metrics endpoint returning counts only (placeholder for durations)

Deliverable: View/create/update leads; see basic list/details in UI

Milestone 2 — Lifecycle Transitions, Alarms, Metrics

• Implement transitions: contacted, qualify, schedule (requires appointment), complete, close lost
• Implement DO alarm() for 30-day auto-close rules
• Implement /leads/metrics for time-to-contact, time-to-qualify, time-to-schedule, funnel rates, loss reason breakdown
• Add UI actions and reason pickers; enforce appointment requirement for scheduled

Deliverable: Full manual lifecycle with auto-close; meaningful metrics in UI

Milestone 3 — Signal Ingestion and Matchback

• CallRail linkage in callRail.ts: upsert by phone, set contacted/qualified/scheduled as applicable; log interactions
• Email ingestion route to detect online scheduling notifications and auto-schedule
• Matchback integration in workflow(s) to set scheduled/completed based on external Orders
• Conversation history UI populated from interactions

Deliverable: Phone/email/matchback signals move leads automatically; visible interaction history

Milestone 4 — Conversion Reporting to Ad Platforms

• Implement ConversionReportingService and extend GoogleAdsService for offline conversions
• Trigger uploads on transitions (qualified, scheduled, completed)
• Add optional re-sync/backfill endpoint
• Basic success/failure observability for uploads

Deliverable: Google/Meta receive conversion events improving optimization

Milestone 5 — Hardening and DX

• Security reviews: input sanitation, auth checks, rate limits (as needed)
• Observability: structured logs, analytics events for all mutations
• E2E tests for critical flows; sample fixtures
• Docs and examples in dev-docs for API usage

Deliverable: Stable, observable, documented system ready for wider rollout

---

Risks and Mitigations

• Matching by phone/email can be ambiguous → deterministic rules and manual override
• Provider webhooks vary → isolate parsers, start with one provider and add adapters
• Offline conversion attribution requires ids (gclid/fbp/fbc) → capture at lead creation when available; add tracking template guidance

Out of Scope (initial)

• Outbound messaging providers (SMS/email/FB) for sending new messages (UI displays history only initially)
• Multi-location routing and complex permissions beyond existing workspace model