CJK IME popup display at wrong position

[xland]

OS: Windows 10
RmlUi: master branch 2022-05-09

CJK(Chinese Japanese Korean) IME popup is displayed on the right bottom side of the screen when input starts.
(It should display near by the input element)

CEF has the same issue
https://bitbucket.org/chromiumembedded/cef/issues/3313/windows-ime-popup-displayed-in-wrong
(Is this info helpful? https://en.sfml-dev.org/forums/index.php?topic=22659.msg158709#msg158709)

And RmlUi Input elemet accept CJK character very slow.

[mikke89]

Hi.
Thanks for reporting. I have never used IME myself, so someone else will have to take this upon themselves. I'll happily review any pull requests.

I am guessing this has more to do with the samples and shell, does it require any changes to the core library?

And RmlUi Input elemet accept CJK character very slow.

Could you elaborate, slow in what way? What can we do to mitigate it?

[xland]

@mikke89

I spent two days looking for a solution.
I found that ImmAssociateContextEx,ImmGetContext,ImmSetCompositionWindow,ImmSetCandidateWindow
and ImmReleaseContext windows APIs are helpful to fix this issue.
(GUI framework like FLTK,ImGui,GTK use these APIs to fix this issue)
But these APIs need the input cursor position and size(not mouse cursor).
And I can not get them by RmlUi.
Could you give me some suggesstions?
I am happy to submit a PR when I solve it.

This is what FLTK do:

static void get_imm_module() {
  s_imm_module = LoadLibrary("IMM32.DLL");
  if (!s_imm_module)
    Fl::fatal("FLTK Lib Error: IMM32.DLL file not found!\n\n"
              "Please check your input method manager library accessibility.");
  flImmAssociateContextEx = (flTypeImmAssociateContextEx)GetProcAddress(s_imm_module, "ImmAssociateContextEx");
  flImmGetContext = (flTypeImmGetContext)GetProcAddress(s_imm_module, "ImmGetContext");
  flImmSetCompositionWindow = (flTypeImmSetCompositionWindow)GetProcAddress(s_imm_module, "ImmSetCompositionWindow");
  flImmReleaseContext = (flTypeImmReleaseContext)GetProcAddress(s_imm_module, "ImmReleaseContext");
}



void Fl_GDI_Graphics_Driver::set_spot(int font, int size, int X, int Y, int W, int H, Fl_Window *win)
{
  if (!win) return;
  Fl_Window* tw = win->top_window();
  if (!tw->shown())
    return;
  HIMC himc = flImmGetContext(fl_xid(tw));
  if (himc) {
    COMPOSITIONFORM cfs;
    float s = scale();
    cfs.dwStyle = CFS_POINT;
    cfs.ptCurrentPos.x = int(X * s);
    cfs.ptCurrentPos.y = int(Y * s) - int(tw->labelsize() * s);
    // Attempt to have temporary text entered by input method use scaled font.
    // Does good, but still not always effective.
    Fl_GDI_Font_Descriptor *desc = (Fl_GDI_Font_Descriptor*)font_descriptor();
    if (desc) SelectObject((HDC)gc(), desc->fid);
    MapWindowPoints(fl_xid(win), fl_xid(tw), &cfs.ptCurrentPos, 1);
    flImmSetCompositionWindow(himc, &cfs);
    flImmReleaseContext(fl_xid(tw), himc);
  }
}

[mikke89]

Nice to see you looking into this.

Here are some pointers to get you started:

To get the text element currently being edited you can call Context::GetFocusElement(). There is also SystemInterface::ActivateKeyboard() which is called when a text box receives focus.

So the focused element itself would be either an ElementFormControlInput or ElementFormControlTextArea. You could use rmlui_dynamic_cast to try to cast to one of these types. Both of these types use an underlying WidgetTextInput which contains the cursor position (cursor_position). I guess the next step is to enable access to that one somehow.

Let me know if I can help out some more.

[xland]

@mikke89

I found that cursor_position is the relative position,
How can I translate the cursor_position to an absolute position of the screen.
Thanks

[mikke89]

Right, yes, you have to add the absolute position to it. I see the following in the source code:

	Vector2f text_translation = parent->GetAbsoluteOffset() - Vector2f(parent->GetScrollLeft(), parent->GetScrollTop());
	/* ... */
	cursor_geometry.Render(text_translation + cursor_position);

So replicating this should solve it.

If you want to position it below the text cursor, I guess you could also add cursor_size to achieve that.

[xland]

#305 PR fixed this issue.