Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 12 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,15 @@ geometry, raise, and engagement lifecycle — the mechanism behind an
infinite-canvas "window stream" tile, extracted as a federated plugin so a
second platform (Windows) can be added without touching consumers.

> **Status.** macOS backend is complete and in production use (extracted from
> the Campus `work_canvas` Runner): per-window capture, geometry/raise/engage,
> and the engagement-event push. A runnable `example/` app exercises the API
> (list windows → capture → engage). The **Windows** backend
> (`flutter_window_portal_windows`) implements window enumeration + control +
> the engagement events today; per-window **capture** (Windows.Graphics.Capture
> → Flutter texture) is the remaining slice — see the roadmap below.
> **Status.** Both backends implement the full v1 contract. **macOS** is in
> production use (extracted from the Campus `work_canvas` Runner):
> ScreenCaptureKit capture, geometry/raise/engage, and the engagement-event
> push. **Windows** implements the same surface over Windows.Graphics.Capture
> (frames reach the Flutter texture as DXGI shared handles, so they never leave
> the GPU), Win32 window control, and `SetWinEventHook` engagement events — see
> [its README](packages/flutter_window_portal_windows/README.md) for the
> platform-specific conventions. A runnable `example/` app exercises the API on
> both (list windows → capture → engage).

## Why a plugin (and why federated)

Expand All @@ -23,8 +25,8 @@ consumer's tile code doesn't change a line.

Unlike a CEF-style plugin, there is **no bundled runtime binary**: every
backend compiles from source against OS frameworks (macOS: ScreenCaptureKit
+ Accessibility + CoreGraphics; Windows: Windows.Graphics.Capture + Win32 +
UI Automation). So there is nothing to fetch, content-hash-match, or drift —
+ Accessibility + CoreGraphics; Windows: Windows.Graphics.Capture + Direct3D
+ Win32). So there is nothing to fetch, content-hash-match, or drift —
the failure mode that a fetched prebuilt (like `cef_host`) introduces simply
does not exist here.

Expand All @@ -35,7 +37,7 @@ flutter_window_portal/ app-facing API — WindowPortal
packages/
flutter_window_portal_platform_interface/ the contract + method-channel default
flutter_window_portal_macos/ endorsed macOS impl (ScreenCaptureKit + AX)
flutter_window_portal_windows/ endorsed Windows impl (planned)
flutter_window_portal_windows/ endorsed Windows impl (WGC + Win32)
```

## The contract
Expand Down
13 changes: 11 additions & 2 deletions packages/flutter_window_portal_windows/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,12 @@
## 0.0.1
## 0.1.0

* TODO: Describe initial release.
* Per-window capture via Windows.Graphics.Capture, delivered into a Flutter
texture as a DXGI shared handle: `startStream`, `stopStream`,
`reconfigureStream`, `streamStatus`.
* One-shot PNG grabs for a window (`captureSingleFrame`) and a display
(`captureSingleDisplayFrame`), encoded with WIC off the platform thread.
* Engagement lifecycle events pushed from `SetWinEventHook`: `alignmentBroken`
on external move/resize, `disengaged(windowClosed)` on destroy, and
`disengaged(thirdApp | hostActivated)` on focus change.
* `detectSeparateWindows` reports same-process windows overlapping the source.
* Window enumeration and geometry/raise/engage control over Win32.
87 changes: 78 additions & 9 deletions packages/flutter_window_portal_windows/README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,84 @@
# flutter_window_portal_windows

A new Flutter plugin project.
The Windows implementation of [`flutter_window_portal`][portal] — capture a
native OS window into a Flutter `Texture` and control its geometry, raise, and
engagement lifecycle.

## Getting Started
This package is **endorsed**: depend on `flutter_window_portal` and this
implementation is used automatically on Windows. You never import it directly.

This project is a starting point for a Flutter
[plug-in package](https://flutter.dev/to/develop-plugins),
a specialized package that includes platform-specific implementation code for
Android and/or iOS.
## How it works

For help getting started with Flutter development, view the
[online documentation](https://docs.flutter.dev), which offers tutorials,
samples, guidance on mobile development, and a full API reference.
| Concern | Windows API |
| --- | --- |
| Window enumeration | `EnumWindows` + `QueryFullProcessImageName` |
| Capture | `Windows.Graphics.Capture` (`Direct3D11CaptureFramePool`) |
| Frame delivery | D3D11 → DXGI shared handle → Flutter `GpuSurfaceTexture` |
| One-shot PNG | WGC → staging texture → WIC encoder |
| Geometry / raise | `SetWindowPos` · `SetForegroundWindow` · `SetCursorPos` |
| Engagement events | `SetWinEventHook` |

Captured frames never leave the GPU: each frame is copied into one of two
BGRA textures created with `D3D11_RESOURCE_MISC_SHARED`, and the engine opens
that shared handle directly. The two textures rotate so the engine is never
reading the surface being overwritten.

Everything compiles from source against the OS — there is no bundled runtime
binary to fetch or version-match.

## Requirements

- Windows 10 1803 (build 17134) or newer for capture. `startStream` and the
one-shot grabs return a typed `capture_unsupported` error on older systems;
window control and enumeration still work.
- Cursor omission needs Windows 10 2004; suppressing the yellow capture border
needs Windows 11. Both are applied when the OS exposes them and skipped
cleanly when it does not.
- No permission prompt. Unlike macOS Screen Recording, Windows gates none of
this, so `checkAccessibilityPermission` always reports `granted`.

## Platform notes

These are the places where the Windows backend must make a choice the shared
contract does not dictate:

- **Window ids** are `HWND` values, stringified. They are not stable across
restarts of the source app.
- **Display ids** are `HMONITOR` values. Windows has no `CGDirectDisplayID`
equivalent, so `captureSingleDisplayFrame` takes the monitor handle — and
accepts `'0'` to mean the primary display, which is what a consumer with no
display list can ask for. Display thumbnails are capped at 480px wide, the
same cap the macOS backend applies.
- **Sizes** cross the channel as logical units (physical pixels ÷ the window's
DPI scale), matching what the macOS backend reports in points. Capture itself
runs at the source's native pixel size.
- **`getContentMeasurement`** always reports zero insets. The transparent
drop-shadow margin it exists to crop is an Electron-on-macOS artifact; DWM
composites shadows outside the window, so there is nothing to trim.
- **`reconfigureStream`** reports rather than applies. The capture pool resizes
itself when a frame's content size changes, so this call tells the consumer
whether the source has outgrown the frames published so far.

## Engagement events

The backend pushes the same three events the macOS backend does, driven by
`SetWinEventHook` rather than an `AXObserver`:

- `alignmentBroken` — `EVENT_OBJECT_LOCATIONCHANGE` moved the engaged window
away from where engage left it. Reported once per engagement; engage and
conform mask their own moves for 1.5s so only genuinely external motion
counts.
- `disengaged(windowClosed)` — `EVENT_OBJECT_DESTROY`.
- `disengaged(thirdApp | hostActivated)` — `EVENT_SYSTEM_FOREGROUND` moved
focus to another app, or back to the Flutter host. Focus landing on another
window of the *source's own* process (a modal dialog, a save sheet) keeps the
engagement alive.

Location and destroy hooks are scoped to the source's process — a system-wide
`EVENT_OBJECT_LOCATIONCHANGE` hook fires constantly.

Minimize is deliberately *not* an event: like macOS, it is surfaced through the
`isWindowMinimized` poll so the consumer can show an overlay over the frozen
stream rather than tear the tile down.

[portal]: https://github.com/FlutterFlow/flutter_window_portal
4 changes: 2 additions & 2 deletions packages/flutter_window_portal_windows/pubspec.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: flutter_window_portal_windows
description: "Windows implementation of flutter_window_portal: per-window capture via Windows.Graphics.Capture into a Flutter texture, plus Win32/UIA geometry, raise, and engagement control."
description: "Windows implementation of flutter_window_portal: per-window capture via Windows.Graphics.Capture into a Flutter texture, plus Win32 geometry, raise, and engagement control."
version: 0.1.0
homepage: https://github.com/FlutterFlow/flutter_window_portal
repository: https://github.com/FlutterFlow/flutter_window_portal
Expand Down Expand Up @@ -32,7 +32,7 @@ flutter:
platforms:
windows:
# C++ host plugin — capture (Windows.Graphics.Capture) + window control
# (Win32 / UI Automation).
# and engagement events (Win32).
pluginClass: FlutterWindowPortalWindowsPluginCApi
# Dart entrypoint: endorses the default method-channel platform instance.
dartPluginClass: FlutterWindowPortalWindows
59 changes: 57 additions & 2 deletions packages/flutter_window_portal_windows/windows/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,20 @@ set(PLUGIN_NAME "flutter_window_portal_windows_plugin")
list(APPEND PLUGIN_SOURCES
"flutter_window_portal_windows_plugin.cpp"
"flutter_window_portal_windows_plugin.h"
"engagement_watcher.cpp"
"engagement_watcher.h"
"platform_task_runner.cpp"
"platform_task_runner.h"
"capture/capture_controller.cpp"
"capture/capture_controller.h"
"capture/d3d_context.cpp"
"capture/d3d_context.h"
"capture/png_encoder.cpp"
"capture/png_encoder.h"
"capture/texture_sink.cpp"
"capture/texture_sink.h"
"capture/wgc_session.cpp"
"capture/wgc_session.h"
)

# Define the plugin library target. Its name must not be changed (see comment
Expand All @@ -35,18 +49,59 @@ add_library(${PLUGIN_NAME} SHARED
# full control over build settings.
apply_standard_settings(${PLUGIN_NAME})

# C++/WinRT routes its coroutine support through <experimental/coroutine> when
# building as C++17, and MSVC 14.51 hard-errors on that header (STL1011) ahead
# of removing it. Under C++20 it selects the standard <coroutine> instead.
# apply_standard_settings above asks for C++17; this raises the floor for this
# target only, leaving the rest of the build alone.
target_compile_features(${PLUGIN_NAME} PRIVATE cxx_std_20)

# Symbols are hidden by default to reduce the chance of accidental conflicts
# between plugins. This should not be removed; any symbols that should be
# exported should be explicitly exported with the FLUTTER_PLUGIN_EXPORT macro.
set_target_properties(${PLUGIN_NAME} PROPERTIES
CXX_VISIBILITY_PRESET hidden)
target_compile_definitions(${PLUGIN_NAME} PRIVATE FLUTTER_PLUGIN_IMPL)
# NOMINMAX: <windows.h> defines min/max as macros, which breaks std::min and
# std::max in the capture sources.
target_compile_definitions(${PLUGIN_NAME} PRIVATE FLUTTER_PLUGIN_IMPL NOMINMAX)

# Source include directories and library dependencies. Add any plugin-specific
# dependencies here.
target_include_directories(${PLUGIN_NAME} INTERFACE
"${CMAKE_CURRENT_SOURCE_DIR}/include")
target_link_libraries(${PLUGIN_NAME} PRIVATE flutter flutter_wrapper_plugin dwmapi shcore)

# The C++/WinRT projection headers (winrt/Windows.Graphics.Capture.h and
# friends) ship in the Windows SDK under Include/<version>/cppwinrt. The MSVC
# toolchain normally puts that on the include path alongside um/ and shared/,
# but which SDK components are installed varies, so locate it and add it when
# it is not already reachable. Nothing is added when the compiler can already
# find the headers.
find_path(FWP_CPPWINRT_INCLUDE_DIR
NAMES "winrt/Windows.Graphics.Capture.h"
PATHS
"$ENV{WindowsSdkDir}Include/$ENV{WindowsSDKVersion}cppwinrt"
"$ENV{WindowsSdkDir}Include/${CMAKE_VS_WINDOWS_TARGET_PLATFORM_VERSION}/cppwinrt"
"C:/Program Files (x86)/Windows Kits/10/Include/${CMAKE_VS_WINDOWS_TARGET_PLATFORM_VERSION}/cppwinrt"
NO_DEFAULT_PATH
)
if(FWP_CPPWINRT_INCLUDE_DIR)
target_include_directories(${PLUGIN_NAME} PRIVATE "${FWP_CPPWINRT_INCLUDE_DIR}")
endif()

# d3d11/dxgi: the capture device and the shared textures Flutter opens.
# windowscodecs + shlwapi: PNG encoding for the one-shot grabs.
# runtimeobject: the WinRT activation/string entry points C++/WinRT calls.
target_link_libraries(${PLUGIN_NAME} PRIVATE
flutter
flutter_wrapper_plugin
dwmapi
shcore
d3d11
dxgi
windowscodecs
shlwapi
runtimeobject
)

# List of absolute paths to libraries that should be bundled with the plugin.
# This list could contain prebuilt libraries, or libraries created by an
Expand Down
Loading
Loading