Overhear: Local Meeting Transcriber & Summarizer

Overhear is a privacy-first, offline-capable desktop application for transcribing meetings in real-time and generating intelligent summaries using LLMs. It captures both your microphone and system audio (e.g., Zoom, Google Meet, Teams) to provide a complete, timestamped transcript and an actionable summary of your conversations.

Features

• 100% Local Processing (Transcription): Transcriptions are performed on your machine using faster-whisper. No audio data ever leaves your computer.
• Transcribe Pre-Recorded Video/Audio Files: Drop in one or more .mp4 (or .mov, .mkv, .m4a, .mp3, .wav, …) files and get transcripts back — either one transcript per file or a single merged transcript stitched together in the order you set. Files transcribe a few at a time, each as its own task with its own progress/✓, you can keep adding files mid-run, and an optional Fast mode (VAD batching) runs ~3–4× faster. See Video Files below.
• Language Selection: Pick the meeting's spoken language before recording, or leave it on Auto-detect (the default) to let Whisper detect it automatically.
• Agnostic Summarization (LLM): Generate summaries using any provider supported by litellm (Gemini, OpenAI, Anthropic, etc.).
• Customizable Meeting Types: Pick a meeting type (1:1, Prospect Call, Team Meeting, Therapy Session, or your own) and its tailored prompt template loads automatically — see Meeting Types below.
• Meeting Context for Summaries: Add optional free-text details (purpose, background) and a list of participants (name + affiliation/role) that get folded into the summarization prompt so the LLM can attribute voices instead of guessing.
• Auto-Summarize: Automatically trigger a summary generation immediately after a recording stops.
• Meeting-Relative Timestamps: Generates transcripts with [HH:MM:SS - HH:MM:SS] timestamps relative to the start of the meeting.
• Automatic Organization: Transcripts and summaries are saved as Markdown files (.md) and organized by date.
• Real-time Volume Indicator: Visual feedback while recording to ensure your audio levels are correct.
• Switch Audio Source Mid-Meeting: Pause a recording to change the input device (e.g. swap from your Aggregate device to a plain microphone) without losing your transcript so far.

---

Prerequisites

• Python 3.9+
• Audio Loopback Software:
  • macOS: BlackHole (2ch) is required. Download here.
  • Windows: VB-Cable is recommended. Download here.

---

Installation

1. Clone the Repository

git clone https://github.com/jpbohorquez/overhear.git
cd overhear

2. Setup the Environment

macOS / Linux

Run the provided setup script:

chmod +x setup.sh
./setup.sh

Windows

Run these commands in PowerShell:

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt

---

System Audio Setup

macOS (Using BlackHole)

To capture both your microphone and system audio (e.g., Zoom/Google Meet), configure Audio MIDI Setup. You can create as many common configurations as you like (e.g. speakers, headphones, airpods, etc.)

Step A: Create a Multi-Output Device (To Listen)

1. Open Audio MIDI Setup.
2. Click + > Create Multi-Output Device.
3. Name it Overhear - Listen.
4. Check BlackHole 2ch AND your actual output (e.g., "External Headphones").
5. Set your output device as the Master Device.

Step B: Create an Aggregate Device (To Record)

1. Click + > Create Aggregate Device.
2. Name it Overhear - Aggregate.
3. Check BlackHole 2ch AND your Microphone.

Windows (Using VB-Cable)

Step A: Configure System Output

1. Right-click the Volume icon in the Taskbar and select Sound settings.
2. Set Output to CABLE Input (VB-Audio Virtual Cable).
3. (Optional but recommended): To hear the audio yourself, go to the Recording tab in the old Sound Control Panel, right-click CABLE Output, select Properties > Listen, check "Listen to this device", and select your actual headphones/speakers.

Step B: Select the Device in Overhear

In the app's dropdown, select CABLE Output (VB-Audio Virtual Cable) to capture the system audio. If you want to capture your microphone simultaneously, you may need to use a tool like VoiceMeeter to mix them into the virtual cable.

---

Usage

1. Set System Output

Before starting your meeting, click the Volume icon in your macOS menu bar (or Windows Sound Settings) and set your output to your virtual loopback device (e.g., Overhear - Listen on Mac or VB-Cable on Windows)

2. Launch the Application

• macOS: Double-click launch.command.
• Windows: Double-click launch.bat.

3. Summarization Setup

1. Go to the Settings tab.
2. Select your API Provider (e.g., GEMINI).
3. Enter your API Key.
4. The LLM Model dropdown will dynamically populate with available models for that provider.
5. Click Save Settings.

4. Transcribe & Summarize

1. Recording Tab: Enter meeting name, select device, choose a Language (or leave it on Auto-detect), and click Record. The Audio Levels meter becomes active once recording starts.
2. Switch Source or Language Mid-Meeting: Click Pause to enable the device and language dropdowns, change either, then Resume — the transcript continues in the same file, with a note marking where the language changed.
3. Auto-Summarize: Toggle "Auto-Summarize after stop" to get an instant summary when you finish.
4. Video Files Tab: Have recordings already? Add .mp4/.mov/.m4a/… files, order them, pick one-transcript-per-file or a single merged transcript, optionally tick Fast mode, and click Transcribe. Files transcribe a few at a time, each completes independently with its own ✓, and you can keep adding files while a batch runs. See Video Files.
5. Summarization Tab: Manually process any existing transcript by selecting it, choosing a Meeting Type (loads its tailored prompt), optionally filling in Meeting Details and adding Participants (name + affiliation/role), and clicking Generate Summary. You can still hand-edit the loaded prompt before generating.

---

Video Files

The Video Files tab transcribes media you already have on disk — recorded calls, webinars, lecture captures — without going through the live recording flow.

1. Open the Video Files tab and click Add Files…. Pick one or more video/audio files (.mp4, .mov, .mkv, .avi, .webm, .m4v, .m4a, .mp3, .wav, .flac, .aac, .ogg, .opus). Audio is pulled straight from the container — no separate conversion step, no ffmpeg install.
2. Reorder the list with the ▲ / ▼ buttons (order matters for the merged output) and drop files with ✕.
3. Choose the Output mode:
   • One transcript per file — writes a separate .md per file, each timestamped from 00:00:00.
   • Single merged transcript — writes one .md for the whole batch. Timestamps are continuous across the files (file 2 picks up where file 1's audio ended), and each file gets a ## section header marking the boundary. Give the merged file a name in the field that appears. Individual per-file transcripts are not kept in this mode.
4. Optionally tick Fast mode (VAD batching) — see below.
5. Pick a Language (or leave Auto-detect), then click Transcribe.
6. Each row shows its own state: ⏳ pending → ◐ n% → ✓ done (or ✗ with the error). Files transcribe a few at a time (2 by default, auto-scaled from your CPU — override with file_workers in config.toml), so completion order can interleave. Cancel stops submitting new files and lets the in-flight ones finish.
7. Add Files… stays live during a run — new files join the current batch and are processed after the in-flight ones. Both output modes support this (in merged mode they're appended as new sections). When the queue empties the batch waits ~2 seconds for any last additions before finalizing.
8. Clear finished removes all ✓ / ✗ rows so you can start a fresh batch without clearing them one by one.

Output lands in the same place as recorded transcripts — transcriptions/YYYY-MM-DD/ — so the results are immediately available in the Summarization tab.

Fast mode (VAD batching)

Fast mode runs voice-activity detection, drops silence, and transcribes speech chunks in parallel batches — roughly 3–4× faster per file. It's off by default; the app remembers your choice (file_fast_mode in config.toml).

Turn it ON for: lectures, webinars, conference talks, podcasts, screen recordings — long, mostly single-speaker audio, or any bulk job where turnaround matters more than transcript polish; and when you're using a larger Whisper model (small / medium / large-v3).

Leave it OFF for: therapy sessions, sensitive 1:1s, legal/HR conversations — anything where silences and verbatim fidelity carry meaning; heavy-crosstalk multi-speaker meetings on the tiny / base model; or when the transcript must match the live-recording output exactly.

Why: chunks are transcribed independently, so word accuracy holds up (often with fewer hallucinations) but rare proper-noun spelling / punctuation can vary across chunk boundaries, and non-speech gaps are removed (timestamps still jump, so pauses stay visible).

---

Meeting Types

The Summarization tab's Meeting Type dropdown is entirely defined in config.toml — nothing is hardcoded. Out of the box it ships with 1:1, Prospect / Sales Call, Team Meeting, and Therapy Session, each with its own prompt template tuned for that kind of conversation (different sections, tone, and focus). "General / Default" (the base system_prompt) is always offered first.

To add your own type, add a [meeting_types.<key>] table:

[meeting_types.customer_support]
label = "Customer Support Call"
prompt = """# Role
Act as a support analyst summarizing a customer call.
...
Known Participants:
$participants$

Additional Meeting Details: $meeting_details$

# Input Data
$transcript$
..."""

Each type's prompt is a full, independent template — not a diff against the default — and supports the same three placeholders as system_prompt: $participants$, $meeting_details$, and $transcript$. Selecting a type in the app loads its prompt into the editable Summary Prompt box, where you can still tweak it before generating.

---

Configuration

Settings are stored in config.toml. Secrets are stored in .secrets.toml (which is git-ignored).

[transcription]
model_size = "base"
output_dir = "transcriptions"
file_workers = "auto"   # Video Files tab: files transcribed at once ("auto" = 2, 3 on ≥12-core CPUs, or an int 1–4)
file_fast_mode = false  # Video Files tab: default state of the "Fast mode (VAD batching)" checkbox

[summarization]
model_name = "gemini/gemini-1.5-flash"
summaries_dir = "summaries"
system_prompt = "..."

[meeting_types.one_on_one]
label = "1:1"
prompt = "..."

---

Troubleshooting

• No Audio: Verify MIDI/Sound settings and ensure the correct Aggregate/CABLE device is selected in the app.
• LLM Errors: Ensure your API key is correct and you have internet access for the summarization phase.
• Permissions: Grant Microphone permissions to your terminal/Python in System Settings.