Skip to content

On-screen keyboard & in-app IME (1.70…1.83)

Win11 / Fluent touch keyboard chrome we own. This is not Qt Virtual Keyboard, and it is not a hardware-shortcut cookbook (keyboard.md).

Status: 1.85 a11y wave 3 (IME candidate live region). OSK/IME still experimental. Next: 2.01+ green soak / promote (parked until after 2.00). 2.00 is after 1.90 (ROADMAP.md, checkpoint-190).
License: OSK chrome is this repo (Apache-2.0). SIL Keyman Core (MIT) for layouts. Pinyin tables are mozillazg/pinyin-data (MIT) — NOTICE-pinyin.md. Japanese is a Hepburn romaji→kana map (no kanji — no MIT reading lexicon). Korean is Unicode hangul composition (not a lexicon). Keyman cs_pinyin IMX is not used.

Slice What ships
1.70 Win11 dock + en-US letters/symbols (builtin inject)
1.71 Keyman Core + extra layouts (de/fr/es/ru/ar) — still no candidate window
1.72 Chinese IME — pinyin composition + our candidate bar
1.73 Full in-app IME — ja romaji/kana, ko hangul, emoji layer, shared candidate host
1.74 Soak / harden — Gallery checklist + a11y + romaji fixes; still experimental
1.75 Extra documented Keyman .kmx — en-GB / it / pt / pl / sv / tr (named subset)
1.76 IME deepen (MIT): pinyin prefix phrases, hangul peel/Space; ja kanji skipped
1.77 App-scoped hardware keys → same engine
1.78 Long-horizon checkpoint
1.79 Wayland field harden (portal parent + CapsLock); OSK still experimental
1.80 Win11 default touch layout chrome (Esc/Tab/dual Shift/lang chip/hints)
1.81 Win11 behavior vs Win10: long-press, size modes, clipboard, emoji tabs
1.82 Floating OnScreenKeyboardWindow + Windows system-wide SendInput (floating defaults on)
1.83 Floating no-activate soak + honest UIPI / UWP / games limits
1.84 Consumer host: examples/floating-osk
1.85 ImeCandidateBar live-region announce (a11y wave 3); still experimental
2.58 App dock recipe — sharedEngine, focus return, floating candidates, AnnotatedScrollBar.imeEngine — osk-in-apps-258.md
1.87 Green soak + promote (planned; slip if not green)

Why this split

Qt Virtual Keyboard is GPL-3.0 or commercial. QWinUI3 is Apache-2.0. Bootstrap::configureEnvironment already clears QT_IM_MODULE so desktop kits do not pull that GPL plugin (packaging-consumer.md strip notes).

A MIT QML keyboard that also ships its own look (SomcoKeyboard, OpenVirtualKeyboard, UnivKbd) would fight Theme tokens and Win11 chrome. We only want their idea (in-process OSK), not their UI.

Piece Owner License Role
Key caps, layers, acrylic dock, Theme QWinUI3 QML Apache-2.0 Win11-style panel
Key → text (shift, caps, dead keys, layouts) SIL Keyman Core (libkeymancore) MIT Engine only — no UI. Builtin en-US fallback if Core is not fetched
Insert into the focused control QWinUI3 C++ adapter Apache-2.0 QInputMethodEvent / a few QKeyEvents
Qt Virtual Keyboard / QT_IM_MODULE=qtvirtualkeyboard — GPL / commercial Forbidden

Engine source: keymanapp/keyman core/ (MIT).
API: Keyman Core (km_core_process_event, actions). Layouts: keymanapp/keyboards (community .kmx, MIT).
Notice: NOTICE-Keyman.md.

Architecture

  focused TextField / TextArea
           ▲
           │  QInputMethodEvent (commit / backspace)
           │  QKeyEvent for Enter / Tab / Backspace
           │
  KeyboardEngine  (C++, QML_ELEMENT)     ← our adapter
           │
           │  km_core_process_event → libkeymancore (MIT, static)  — layouts
           │  PinyinLexicon / RomajiKana / HangulComposer         — in-app IME
           │  fallback: builtin en-US if Core sources are missing
           ▼
  OnScreenKeyboard.qml   ← our Fluent / Win11 chrome (Theme tokens)
  ImeCandidateBar.qml    ← shared candidate strip (zh/ja/ko)

Keep QT_IM_MODULE unset. Do not ship a platforminputcontexts plugin in 1.70…1.77 — an in-window dock is enough to theme, test, and stay off the GPL IM module path. Extra languages are more .kmx (1.75) or MIT IME tables (1.76), not a second engine.

Engine choice

Pick: SIL Keyman Core. It is a C API keyboard processor: load a keyboard, feed virtual keys, get insert/delete actions. Windows is first-class. UI is explicitly a “platform layer” the consumer writes.

Candidate License UI? Windows Why not (or why)
Keyman Core MIT No Yes Use this
libxkbcommon MIT No Weak Hardware XKB/Wayland keymaps, not a touch OSK
Qt Virtual Keyboard GPL / commercial Yes Yes License conflict
SomcoKeyboard / OpenVirtualKeyboard MIT Yes (QML) Yes We own chrome; do not vendor their QML

How we build Core: sources are vendored under third_party/keyman (core/ + common/ only — in git clone). QWINUI3_FETCH_KEYMAN=ON is only a fallback if that tree is deleted (scripts/fetch_keyman_core.py). CMake target qwinui3_keymancore is static, KM_CORE_LIBRARY_STATIC, KMN_NO_ICU=1. NFC/NFD uses Qt (src/extras/keyman_shims/util_normalize_qt.cpp). LDML regex is stubbed — basic .kmx packs do not need it. Do not vendor the Keyman monorepo UI.

UI we write (Win11 — not Win10 classic)

Follow Windows 11 Touch Keyboard. Explicitly not the Win10 classic “full” keyboard (always-visible number row, flatter keys, different emoji/clipboard).

1.80 layout chrome + 1.81 behavior:

Row Keys
Header Settings · grab bar · close (+ emoji / clipboard tools)
1 Esc · q–p with 1–0 hints · Backspace
2 Tab · a–l · ;: · Enter
3 Shift · z–m · ,; .: · ?! · Shift
4 &123 · Ctrl · Win · Alt · lang chip (英/中/あ/한/…) · Space · mic · ← →
Behavior (1.81) Win11 expectation
Long-press letter with hint Inserts the digit (q→1)
Long-press punctuation Secondary-character flyout
keyboardSize small / default / wide (Settings sheet)
Clipboard tool Paste strip for current clip (OS history stays OS-owned)
Emoji Category chips + grid
Key chrome Rounder radii, press scale, soft raise — not Win10 flat tiles
Mic / Win / Ctrl / Alt Chrome / banner only in-app
Layer Behavior
Letters Physical US VKs; labels from previewVk; Shift latch / Caps; Keyman / IME backends
Symbols Numbers + punctuation (&123 / abc) + long-press alts
Lang chip Cycles en-US/GB · de/fr/es/it/pt/pl/sv/tr · ru/ar · zh/ja/ko
Emoji Category tabs — commitText only, no engine

Tokens: Theme.bgAcrylic / fillControl / local Win11 key radius (~8dp) / Theme.dp(48) default hit size. Reuse FluentIcons (Backspace / ReturnKey / Microphone / Emoji / …).

Suggested Extra: OnScreenKeyboard (experimental). Host in Overlay.overlay, a shell footer, or OnScreenKeyboardWindow (floating). closeRequested / settingsRequested for host wiring.

Floating window + system-wide (1.82)

OnScreenKeyboardWindow {
    id: oskWin
    // Windows: systemWide defaults ON. Dock stays in-app unless you opt in.
}
// oskWin.openFloating()
Mode Behavior
Dock (OnScreenKeyboard in footer) In-app QInputMethodEvent (systemWide default off)
Floating window Always-on-top tool; grab bar drags; WindowHelper.setNoActivate so focus stays on the target
systemWide (Windows) Default on for OnScreenKeyboardWindow; commits / keys use SendInput; IME preedit stays on the OSK candidate bar
Linux / Wayland Floating OK; supportsSystemWide === false (no desktop inject)

Honesty: experimental, Windows-first. Not a full TSF/IMM desktop IME. Elevated apps, UIPI, many UWP hosts, and some games ignore SendInput. Set systemWide: false on the floating host to force in-app-only.

1.83 (shipped) — field harden

Shipped

  • WindowHelper.setNoActivate: WS_EX_NOACTIVATE plus WM_MOUSEACTIVATE → MA_NOACTIVATE (first tap / grab bar)
  • Floating window does not raise() (that can foreground the OSK)
  • Long-press alt Popup uses in-item type on Qt 6.8+ so it does not spawn a focus-stealing HWND
  • Gallery soak: floating vs dock; IME preedit on the bar; Backspace / Enter / arrows inject when not composing

Out

  • Linux system-wide inject
  • Promote to stable (1.87)
  • Full TSF / IMM

1.84 (shipped) — consumer host

Shipped

  • examples/floating-osk — StandardWindow + OnScreenKeyboardWindow.openFloating()
  • systemWide: Qt.platform.os === "windows"
  • Packaging: Keyman Core in third_party/keyman (clone); WebView2 still optional NuGet
OnScreenKeyboardWindow {
    id: oskWin
    systemWide: Qt.platform.os === "windows"
}
// oskWin.openFloating()

Out

  • New .kmx / IME languages
  • vcpkg/Conan (1.88)

1.85 (shipped) — IME live region

Shipped: ImeCandidateBar announces paged candidates (or preedit) on composeChanged without taking focus (Accessible.announce on Qt 6.8+). Part of a11y wave 3 — accessibility.md. Still experimental.

Out

  • Promote to stable (1.87)

Language & IME ladder

Keyman Core already knows thousands of community keyboards. Extending languages is load another .kmx + draw the matching chrome, not rewrite Qt Virtual Keyboard.

Kind Examples When What we add
Direct layouts en-US/GB, de, fr, es, it, pt, pl, sv, tr, ru, ar 1.71 + 1.75 Globe switcher; dead keys via Core; RTL mirroring
Composition IME zh-Hans pinyin 1.72 shipped Preedit + candidate strip (QML we write); MIT pinyin-data
More IMEs ja romaji/kana, ko hangul 1.73 shipped Same candidate host; hangul compositor + romaji map (not Keyman IMX)
Soak zh / ja / ko + 1.71 layouts 1.74 shipped Gallery checklist + a11y; still experimental (not promote-green)
Extra layouts more MIT .kmx 1.75 shipped Named subset (en-GB/it/pt/pl/sv/tr) + BYO recipe; not CJK IMX
IME deepen MIT tables only 1.76 shipped Prefix phrases + hangul polish; ja stays kana (no MIT kanji)
Not this product Handwriting, dictation, cloud lexicon, OS-wide IME Parking lot —

Chinese / CJK needs a candidate UI we own (Win11). Keyman Core does not run Chinese IMX, Japanese Mozc, or Korean dictionaries. zh uses MIT pinyin-data; ja is romaji→kana; ko is 2-beolsik hangul. Kanji conversion is out (no GPL Mozc, no hand-written 词库).

Honest limit: this is an in-app IME for QWinUI3 fields. It does not replace Microsoft Pinyin for the whole desktop, and 1.73–1.76 will not match a cloud IME’s phrase quality.

System IME remains available alongside the panel until a later minor explicitly documents otherwise.

1.70 (shipped)

In

  • Experimental OnScreenKeyboard + KeyboardEngine inject adapter
  • en-US letters + Shift/Caps + symbols; dark/light (builtin backend)
  • Gallery footer dock + this recipe
  • Same inject path Keyman Core uses in 1.71 (engine.backend)

Out

  • Qt Virtual Keyboard / any GPL IM plugin
  • Vendoring third-party QML keyboards
  • Extra layouts / CJK IME (1.71…1.76)
  • Handwriting, dictation
  • Global SendInput into other processes (security)
  • Promoting to stable in the same minor

1.71 (shipped)

In

  • Static-link SIL Keyman Core (qwinui3_keymancore) when sources are fetched
  • Bundled .kmx: basic_kbdus, basic_kbdgr, basic_kbdfr, basic_kbdes, basic_kbdru, basic_kbda1
  • Globe key + Gallery ComboBox; LayoutMirroring on Arabic
  • KeyboardEngine.backend is "keyman" when Core is linked; "builtin" otherwise

Out

  • Candidate window / pinyin (1.72)
  • ICU / meson (we use KMN_NO_ICU + Qt normalize)
  • Qt Virtual Keyboard

Add a layout pack

  1. Drop a MIT .kmx in src/extras/QWinUI3/Extras/keyboards/.
  2. Add the file to qt_add_resources in src/extras/QWinUI3/Extras/CMakeLists.txt.
  3. Map a layout id → filename in KeyboardEngine (kLayoutIds + kmxResource).

1.72 (shipped)

In

  • zh-Hans on the globe; engine.backend === "pinyin"
  • Preedit via QInputMethodEvent + ImeCandidateBar (Theme tokens)
  • Lexicon generated from mozillazg pinyin-data / phrase-pinyin-data (MIT) — not a hand-written table, not GPL libpinyin, not Keyman IMX
  • Space / 1–9 / tap confirm; honest limit: in-app, not Microsoft Pinyin quality

Out

  • ja / ko (1.73 shipped)
  • Cloud lexicon / handwriting
  • Replacing the desktop OS IME

1.73 (shipped)

In

  • ja-JP on the globe; engine.backend === "romaji" — Hepburn longest-match → hiragana; katakana as second candidate; Space / 1–2 / tap confirm
  • ko-KR on the globe; engine.backend === "hangul" — 2-beolsik on US VKs; Unicode syllable compose; incomplete cluster as preedit
  • Shared ImeCandidateBar for zh/ja/ko
  • Emoji layer (FluentIcons.Emoji); commitText only
  • Gallery ComboBox language matrix (en/de/fr/es/ru/ar/zh/ja/ko)
  • Shipped Keyman packs remain the 1.71 set (basic_kbdus/gr/fr/es/ru/da1). Further .kmx is bring-your-own (see 1.71 “Add a layout pack”). ja/ko are not extra Keyman packs

Out

  • OS-wide IME / platforminputcontexts
  • Kanji / hangul-word dictionaries (Mozc, Anthy, libhangul dicts)
  • Cloud lexicon / handwriting / dictation
  • Promote to stable

1.73 architecture:

  TextField  ←  QInputMethodEvent (preedit + commit)
  ImeCandidateBar.qml     ← our Win11 candidate strip
  OnScreenKeyboard.qml    ← same Fluent dock + emoji layer
  KeyboardEngine          ← pinyin (MIT) / romaji-kana / hangul / km_core_process_event
  libkeymancore (MIT)     ← layouts only; CJK is not Keyman IMX

Consumer notes (when shipped)

  • Link qwinui3_extras as today; OnScreenKeyboard is experimental. Copy examples/osk-dock/](../examples/osk-dock/) (**2.58**) for embedded footer, or [examples/floating-osk/` (1.84) for floating desktop input — not the Gallery.
  • Core ships in third_party/keyman with the clone; configure only fetches if that tree is missing (QWINUI3_FETCH_KEYMAN).
  • Strip Qt Virtual Keyboard from windeployqt trees as already documented.
  • Through 1.71 this panel is a touch OSK; 1.72 adds in-app pinyin; 1.73 adds ja/ko + emoji. System IME (Microsoft Pinyin, etc.) stays the desktop CJK default.
  • KeyboardEngine.backend is "pinyin" / "romaji" / "hangul" on those layouts, "keyman" when Core is linked for direct layouts, "builtin" if you skipped the fetch.

1.74 (shipped) — soak

Verdict: soak written for manual Gallery verification. Not promote-green. OnScreenKeyboard stays experimental.

Shipped

  • Gallery On-screen keyboard language-matrix checklist (en/de/fr/es/ru/ar/zh/ja/ko + emoji + backend readout)
  • ImeCandidateBar accessible names for composition, candidates (Candidate N …), page buttons; description mentions Space / 1–9
  • Romaji: finalize trailing n → ん on display / candidates / pick; small kana (xtu/xa/…), ye/wi/we, v*/f*
  • keyboards/README.md lists shipped .kmx vs BYO three-step; IME backends named

Manual checklist (Gallery footer dock)

  • [ ] en-US letters / Shift / Caps / symbols (keyman or builtin)
  • [ ] de / fr / es / ru labels; ar RTL
  • [ ] 中文 nihao + Space/1–9; nv → 女
  • [ ] 日本語 konnichiwa; trailing n → ん; xtu → っ; hiragana + katakana
  • [ ] 한국어 2-beolsik dkssud → 안녕; Shift doubles; incomplete cluster preedit
  • [ ] Emoji layer; keys never steal focus
  • [ ] Candidate bar page buttons when many pinyin hits

Out

  • Promote to stable
  • Extra .kmx (1.75 shipped) / kanji (1.76) / OS IME

1.75 (shipped) — extra Keyman packs

Shipped

  • Named MIT subset (not every keyboard): basic_kbduk (en-GB), basic_kbdit, basic_kbdpo, basic_kbdpl, basic_kbdsw, basic_kbdtuq (tr-TR Turkish-Q)
  • Globe / Gallery ComboBox; full shipped table in src/extras/QWinUI3/Extras/keyboards/README.md
  • Re-fetch: python scripts/fetch_keyman_keyboards.py
  • Still engine.backend === "keyman" for these; Arabic RTL and 1.71 packs unchanged

Out

  • Vendoring the whole keymanapp/keyboards tree
  • CJK via Keyman IMX
  • IME deepen (1.76)

1.76 (shipped) — IME deepen (MIT-only)

Shipped

  • zh: regenerated mozillazg MIT tables (phrases 2–6 chars, more candidates); prefix phrase lookup ( iha → 你好); consumeLength for partial picks
  • ko: Backspace peels compound vowels (ㅘ/ㅙ/ㅚ/ㅝ/ㅞ/ㅟ/ㅢ) and double finals; Space commits + inserts a word break; Caps does not double jamo (Shift only)
  • ja: extra Hepburn rows (thi/dhi/ts/wh/fyu…); kanji not shipped — JMdict / KANJIDIC are CC-BY-SA, not MIT. Gap is intentional under the MIT-only rule

Out

  • Mozc / Anthy / libpinyin / hand-written 词库
  • Promote to stable

1.77 (shipped) — app hardware input

Shipped

  • KeyboardEngine.hardwareInput (default on) — physical keys in this process feed processVk / Keyman (AltGr supported)
  • Candidate keys: 1–9, Esc cancel, PageUp/PageDown page
  • Ctrl / Meta shortcuts pass through; Gallery Switch to disable
  • Honest limit: in-app only. No SendInput into other processes. Not a desktop IME replacement.

Out

  • OS-wide / system input method
  • platforminputcontexts plugin

Voice & handwriting (cross-platform)

Optional panels wired into OnScreenKeyboard — Windows + Linux, in-process libraries (no command-line helpers).

Feature Windows Linux
Voice Vosk (libvosk) or SAPI in-proc Vosk + Qt Multimedia capture
Handwriting Zinnia (libzinnia) or Windows Ink Zinnia in-process
Pinyin learning OskUserLexicon (local QSettings) Same

Setup: on-screen-keyboard-voice-handwriting.md.