Add macOS platform scaffolding - #68
Conversation
|
Nice! Did you have any questions in mind about KeyboardLayout? Did you understand my comment in the other PR (#64 (comment)), especially the part about extending the A good start would be to look into what the equivalent of Windows's keyboard and mouse hook would be on macOS: mousemaster/src/main/java/mousemaster/platform/windows/WindowsPlatform.java Lines 212 to 224 in e425397 How can we receive keyboard and mouse events, what does that look like, what data structures do we get, what identifiers do we get to identify keys? |
I do have some questions regarding this part, it's actually the one that's a bit confusing to me. So I'll first outline the overall key concepts to make sure I fully understand them:
If the above is correct, I see a couple of options to introduce a new set of key codes for a new OS platform:
My opinions on each option: Option 1 Separates key codes across enums, which makes for a good separation of concerns in that regard, which I like to make things clearer when it comes to changing codes in the future. However, it adds multiple OS responsibilities into the Option 2 This is something I already somewhat proposed in my previous PR, I like the abstraction because each type will have a single OS responsibility to translate to Key, which makes things easier to read and therefore maintain. Option 3 This option to me is the worst, as it overloads multiple types with multiple OS responsibilities. That's how I see it all based on my current understanding of the project, please let me know what you think, if I misunderstood something about the architecture in general, and also please let me know about your opinions on the potential approaches to address this, or if you have a different one that I didn't mention. Thanks! |
|
Hey @LikeTheSalad, I agree with option #1: Creating a new platform-specific enum. I don't think it will be named MacosVirtualKey though, because virtual key is a Windows concept. To be able to know what this platform-specific enum is going to be, I had to start doing some research on the macOS APIs for handling keyboard events. This should help us understand where we're going and it will drive the overall architecture. There's a lot of information below, don't be afraid if you're not familiar with this stuff. It starts making sense once you look into it and tries it yourself 😊 But yeah, we're really getting into the hard part of the macOS port here. I initially digged into the CGEventTap API for macOS, which is the obvious equivalent to the Windows keyboard hook API. But a few hours later, I realized that it has several limitations that would prevent us from implementing some of the mousemaster features:
See the results of my research on CGEventTap below ("Approach 2") for more details on how we would integrate it in mousemaster. The solution to the CGEventTap limitations is to use something lower level. I've looked into how kanata does it for macOS (there's a lot of overlap between kanata and mousemaster, so it makes sense to take inspiration from it). Kanata does not use CGEventTap for the keyboard at all; on macOS it grabs the keyboard at the HID/DriverKit layer via the Karabiner-DriverKit-VirtualHIDDevice driver. With this custom driver, we can get the model that mousemaster expects: everything is just a key press event or a key release event. The cost is: it requires running as root, Input Monitoring + Accessibility permissions, and the Karabiner VirtualHIDDevice driver + daemon installed. Now I'll share my findings about each of these 2 approaches (Karabiner-DriverKit vs CGEventTap). 1. Approach 1: Karabiner-DriverKit (ideal, but initial implementation a tiny bit complex 😊)1.1 Wiring Karabiner-DriverKit into mousemaster with JNAJNA can only call a flat C ABI, so we can't call the Karabiner C++ client directly. We need a native .dylib (macOS's equivalent of a Windows .dll) that exposes a few kanata's grab/inject code is exactly that native layer. The 1. Build the dylib. git clone --recursive https://github.com/Psych3r/driverkit
cd driverkit
clang++ -std=c++23 -dynamiclib -O2 -o libdriverkit.dylib \
c_src/driverkit.cpp \
-I c_src/Karabiner-DriverKit-VirtualHIDDevice/include/pqrs/karabiner/driverkit \
-I c_src/Karabiner-DriverKit-VirtualHIDDevice/vendor/vendor/include \
-I c_src/Karabiner-DriverKit-VirtualHIDDevice/src/Daemon/vendor/include \
-framework IOKit -framework CoreFoundation2. The C ABI (from struct DKEvent { uint64_t value; uint32_t page; uint32_t code; uint64_t device_hash; };
bool driver_activated(); // driver installed + approved?
bool register_device(const char* product); // must register >=1 device BEFORE grab()
int grab(); // 0 = success; nonzero = failure (1 = no device registered)
int wait_key(struct DKEvent* e); // BLOCKING: 1 = event written to *e, 0 = input released (EOF)
int send_key(struct DKEvent* e); // 0 = success; 1 = bad usage page, 2 = sink not ready
bool is_sink_ready(); // virtual output device connected?
void release(); // ungrab / cleanup
3. Bind it in Java. public interface Driverkit extends Library {
Driverkit INSTANCE = Native.load("driverkit", Driverkit.class); // libdriverkit.dylib
@Structure.FieldOrder({ "value", "page", "code", "deviceHash" })
class DKEvent extends Structure {
public long value; // uint64_t (1 = press, 0 = release)
public int page; // uint32_t HID usage page (0x07 = keyboard_or_keypad)
public int code; // uint32_t HID usage (the individual key)
public long deviceHash; // uint64_t
}
boolean driver_activated();
boolean register_device(String product);
int grab(); // 0 = success
int wait_key(DKEvent e); // JNA passes the Structure by pointer, auto-reads after return
int send_key(DKEvent e);
boolean is_sink_ready();
void release();
}Thread t = new Thread(() -> {
if (!Driverkit.INSTANCE.driver_activated()) return; // driver not installed / approved
Driverkit.INSTANCE.register_device(...); // register the keyboard(s) first
if (Driverkit.INSTANCE.grab() != 0) return; // grab failed (root? permissions? no device?)
Driverkit.DKEvent ev = new Driverkit.DKEvent();
while (running) {
if (Driverkit.INSTANCE.wait_key(ev) == 0) break; // input released (EOF)
ev.read();
Key key = keyFromHidUsage(ev.page, ev.code);
KeyEvent e = ev.value == 1 ? new PressKeyEvent(now, key) : new ReleaseKeyEvent(now, key);
// feed KeyboardManager exactly like keyboardHookCallback does today
}
Driverkit.INSTANCE.release();
});
t.setName("mac-hid-loop"); t.start();1.2 Mapping HID usages to mousemaster
|
| WindowsPlatform | MacosPlatform |
|---|---|
SetWindowsHookEx(WH_KEYBOARD_LL, ...) |
CGEventTapCreate(...) |
Callback function WinDef.LRESULT keyboardHookCallback(int nCode, WinDef.WPARAM wParam, WinUser.KBDLLHOOKSTRUCT info) |
CGEventRef cb(proxy, CGEventType type, CGEventRef event, void* ctx) |
wParam = WM_KEYDOWN / WM_KEYUP |
type = kCGEventKeyDown / kCGEventKeyUp / kCGEventFlagsChanged. modifiers come as kCGEventFlagsChanged |
return LRESULT(1) to eat / CallNextHookEx(...) to pass |
return NULL to eat / return event to pass |
KBDLLHOOKSTRUCT.scanCode |
CGEventGetIntegerValueField(event, kCGKeyboardEventKeycode) → CGKeyCode (position-based, like a scan code) |
KBDLLHOOKSTRUCT.vkCode |
no equivalent, not needed |
KBDLLHOOKSTRUCT.flags (modifier state) |
CGEventGetFlags(event) → CGEventFlags bitmask |
LLKHF_INJECTED / dwExtraInfo signature |
CGEventGetIntegerValueField(event, kCGEventSourceUserData) |
KBDLLHOOKSTRUCT.time |
CGEventGetTimestamp(event) |
GetAsyncKeyState (stuck-key sanity check) |
CGEventSourceKeyState() |
SendInput (regurgitation / injection) |
CGEventPost(kCGHIDEventTap, CGEventCreateKeyboardEvent(...)) |
activeKeyboardLayout() via foreground window |
TISCopyCurrentKeyboardLayoutInputSource() + kTISPropertyUnicodeKeyLayoutData |
2.1 Mapping macOS keyCode to mousemaster Key
macOS CGKeyCode (from CGEventGetIntegerValueField(event, kCGKeyboardEventKeycode)) is position-based, like a Windows scan code: kVK_ANSI_A is always the physical "A" position regardless of the active layout. So the mapping splits into two buckets:
- Layout-independent keys (Key.keyboardLayoutIndependentKeys): the static
kVK_* -> Keytable below. - Character-producing keys (letters, digits, punctuation): the input keycode needs to be translated
UCKeyTranslate(current layout) to the base character, thenKey.ofCharacter(char).
2.2 Layout-independent keys in macOS (table is shortened, this is just to give a general idea)
| Keycode | hex | macOS kVK_* |
mousemaster Key |
|---|---|---|---|
| 48 | 0x30 | kVK_Tab | tab |
| 36 | 0x24 | kVK_Return | enter |
| 76 | 0x4C | kVK_ANSI_KeypadEnter | enter |
| 53 | 0x35 | kVK_Escape | esc |
Modifiers (arrive as kCGEventFlagsChanged, not KeyDown/KeyUp — see note) |
|||
| 56 | 0x38 | kVK_Shift | leftshift |
| 60 | 0x3C | kVK_RightShift | rightshift |
| 59 | 0x3B | kVK_Control | leftctrl |
| 63 | 0x3F | kVK_Function (Fn) | no Windows equivalent. To do (?) |
| Function keys | |||
| 122 | 0x7A | kVK_F1 | f1 |
| 120 | 0x78 | kVK_F2 | f2 |
| Numpad | |||
| 82 | 0x52 | kVK_ANSI_Keypad0 | numpad0 |
| 83 | 0x53 | kVK_ANSI_Keypad1 | numpad1 |
2.3 Modifiers: macOS vs Windows
mousemaster treats a modifier as just another key (press -> decide to eat -> release).
2.3.1 Reading modifier keys
On Windows, leftshift arrives as WM_KEYDOWN/WM_KEYUP, same shape as a. On macOS, modifier keys never come as KeyDown/KeyUp — only as kCGEventFlagsChanged, which carries no press/release bit. So MacosPlatform must diff CGEventGetFlags(event) against the previous flags to decide down-vs-up, then emit the same PressKeyEvent/ReleaseKeyEvent the rest of mousemaster expects.
2.3.2 Eating modifier keys
Eating a normal key is clean on both (LRESULT(1) on Windows == return NULL on macOS). Eating a modifier key is the one hard case: eating a leftshift key press event is not enough. If you press leftshift and mousemaster eats it, then press a, macOS will still emit a a with shift flag true event. The right approach seems to be to eat the leftshift press event and generate a "leftshift is up" event.
2.3.3 Hard limitation: capslock
Capslock is keycode 57 (kVK_CapsLock), flag kCGEventFlagMaskAlphaShift (0x10000), and like the other modifiers it arrives as kCGEventFlagsChanged. The difference: the flag bit encodes the lock latch (caps on/off), not whether the key is physically held. The bit flips once per toggle and then stays.
Consequence: we can't translate that cleanly to mousemaster's "key press event" / "key release event" model. Capslock can't be used as any other key.
2 participants
This is a continuation from #64 that:
The complicated part, which I'm still not sure I fully understand, is about handling platform-agnostic key names within
KeyboardLayout. So I would appreciate some guidance on that topic.These changes are AI-assisted.