Personal AI Notetaker

# Personal AI Notetaker

AI Summary

Purpose:

• Working plan for Hyunwook's personal Tiro-like AI notetaker across macOS, iOS, and Android.

Key points:

• Build for personal daily use first, not a public SaaS clone.
• Use Flutter + Dart as the main app stack because it covers iOS, Android, and macOS from one codebase and has usable audio, SQLite, and whisper.cpp ecosystem support.
• Use local-first storage: audio files remain on device first; transcript/note metadata lives in SQLite/Drift; Markdown export is a first-class escape hatch.
• Start with offline/batch transcription before realtime streaming; macOS system audio is the main platform-specific spike.
• Initial MVP excludes billing, team workspaces, Slack sharing, Apple Watch, and public API/MCP.

Relevant when:

• Starting the implementation repository.
• Choosing stack, architecture, model, or feature scope for the notetaker.
• Comparing future requirements against the original personal-MVP intent.

Do not read full document unless:

• You are implementing or revising the notetaker architecture.

Linked documents:

• ai/sources/tiro-ai-notetaker-research-2026-07-01.md
• ONBOARDING.md

Open Questions

• Target repository name/path: not created yet.
• Final product name: not decided.
• Cloud STT/LLM fallback: optional, not decided.
• Sync backend: not decided; defer until local-first MVP is useful.
• Apple Watch support: out of first MVP unless explicitly requested.

Details

Goal

Create a personal AI notetaker inspired by Tiro. The target is not to copy the whole service. The target is a tool Hyunwook can actually use often: record meetings/calls/lectures, transcribe them, create Korean notes/action items, and search/export the result across Mac and phone.

The product should remain privacy-conscious and simple enough to release for free later if it proves useful.

Product scope

#### First MVP must include

• macOS, iOS, and Android app shell.
• Microphone recording on all three platforms.
• Offline-first recording: save audio locally before any processing or upload.
• Pause/resume/stop and crash-safe recording recovery.
• File upload/import from local audio/video files.
• Batch transcription from a finished audio file.
• Korean-first summary generation:

- short summary; - decisions; - action items; - keywords/proper nouns; - raw transcript.

• Local note list and note detail screen.
• Search over note title, summary, and transcript.
• Markdown export with audio attachment reference.
• Context field before recording: meeting type, participants, proper nouns, desired tone.

#### First MVP should not include

• Team workspace, billing, credits, roles, invitations.
• Slack/Google Calendar automation unless a later daily-use need appears.
• Apple Watch.
• Public web app.
• MCP/API/CLI server.
• Full speaker diarization.
• Perfect realtime transcription.

Chosen language and framework

Recommended default:

• Flutter + Dart for the app.
• Drift + SQLite for local-first storage.
• Native Swift helper only where needed for macOS system audio capture.

Why Flutter:

• One primary codebase for iOS, Android, and macOS.
• Faster personal MVP iteration than maintaining SwiftUI + Kotlin + macOS app separately.
• Existing packages cover cross-platform microphone recording, local storage, and whisper.cpp bindings.
• The UI needs to be practical and consistent, not deeply platform-native at first.

Why not React Native first:

• React Native is strong on mobile but macOS support and native audio/system-audio behavior add complexity.
• A desktop requirement is first-class here, not an afterthought.

Why not Tauri/Electron first:

• Desktop is strong, but iOS/Android would become a second stack.
• Browser/mobile audio and background recording constraints would dominate the implementation.

Why not fully native first:

• Swift/SwiftUI would be excellent for Apple platforms and macOS system audio, but Android would require a separate Kotlin implementation.
• Native can come later for platform-specific helpers, not as the whole app stack.

Architecture

Flutter UI
  ├─ RecordingController
  │   ├─ Mobile/Desktop microphone capture via record
  │   └─ macOS system-audio spike via desktop_audio_capture or Swift ScreenCaptureKit helper
  ├─ AudioFileStore
  │   └─ local app documents directory, crash-safe session folders
  ├─ TranscriptionService
  │   ├─ whisper.cpp / whisper_ggml_plus batch transcription
  │   └─ optional cloud fallback later ([REDACTED] credentials only outside repo)
  ├─ NoteGenerationService
  │   ├─ local or configured LLM summarization
  │   └─ template prompts for Korean summary/action items
  ├─ LocalDatabase
  │   └─ Drift/SQLite notes, transcripts, speakers, templates, word memories
  └─ ExportService
      └─ Markdown + audio reference bundle

Local data model draft:

notes(
  id, title, created_at, updated_at, source_type,
  status, language, context_text,
  summary_markdown, transcript_text,
  audio_path, duration_ms
)

segments(
  id, note_id, start_ms, end_ms,
  speaker_label, text, confidence
)

templates(
  id, name, instruction_markdown, created_at, updated_at
)

word_memories(
  id, phrase, pronunciation_hint, notes, created_at, updated_at
)

sync_queue(phase-2 only)

Recording strategy

Mobile:

• Use offline recording as the default mode.
• Allow background recording with native permissions and platform UI indicators.
• Handle phone-call interruption by auto-pausing and requiring manual resume.

macOS:

• Microphone recording should work through a Flutter package first.
• System audio is a separate spike:

1. try desktop_audio_capture on macOS 13+; 2. if it is unstable, write a small Swift helper using ScreenCaptureKit; 3. bridge helper output as a local audio stream/file back into Flutter.

Transcription strategy

Start with finished-file batch transcription:

• Convert to 16kHz mono WAV if needed.
• Run local whisper.cpp via a Flutter binding such as whisper_ggml_plus.
• Use tiny or base model for first device tests; allow small or large-v3-turbo later if hardware is good.
• Store the raw audio path and transcript result separately so failed summaries can be retried without re-recording.

Realtime mode is phase 2:

• Chunk audio into short windows.
• Show partial text only as a convenience.
• Still reprocess the final audio for higher-quality transcript after stop.

Summarization strategy

For personal use, summary quality matters more than full offline purity. Start with a pluggable summarizer:

• Input: transcript, context, participant names, word memory.
• Output: Markdown note.
• Default template:

- ## Summary - ## Decisions - ## Action Items - ## Important Details - ## Proper Nouns / Terms - ## Transcript

Secrets and API keys must never be committed. Example configs must use [REDACTED].

Implementation phases

#### Phase 0 — Spike

Acceptance criteria:

• Flutter app runs on Android, iOS, and macOS.
• Can record 30 seconds from microphone on each platform and save a file.
• Can transcribe one saved file locally on at least one mobile platform and macOS.
• macOS system-audio capture has a clear pass/fail result.
• Markdown export works from one transcript.

#### Phase 1 — Personal MVP

Acceptance criteria:

• End-to-end note flow: create → record/import → transcribe → summarize → search → export.
• Works without network until the summarization step if a cloud LLM is used.
• Audio is not deleted automatically until the user chooses cleanup.
• Failed transcription/summarization jobs can be retried.

#### Phase 2 — Daily-use polish

• Model management screen.
• Word memory and reusable context.
• Template management.
• macOS menu bar quick-record.
• Optional local-folder sync or Supabase-backed sync.
• Better long-recording progress, cancellation, and resume.

#### Phase 3 — Free release candidate

• Privacy policy and data-handling page.
• Signed builds and auto-update strategy.
• Crash reporting decision.
• Export/import compatibility promise.
• Public API/MCP/CLI only after the local note schema stabilizes.

Verification checklist for the future implementation repo

• [ ] Real Android device microphone recording.
• [ ] Real iPhone microphone recording and background behavior.
• [ ] macOS microphone recording.
• [ ] macOS system audio from Zoom/Meet/browser.
• [ ] 30-second, 10-minute, and 60-minute recording tests.
• [ ] App kill during recording and recovery.
• [ ] Local Whisper model download and deletion.
• [ ] Korean transcription quality sample.
• [ ] Markdown export round-trip.
• [ ] No credentials in repo.

Source grounding

This plan is grounded in ai/sources/tiro-ai-notetaker-research-2026-07-01.md, which records the Tiro public-doc feature scan and implementation research notes.