In case someone wonders what's "special" about it:
user perspective: As far as I know, it's the only tool offering somewhat reliable emoji input using faked X11 keyboard events. "x11-emoji-picker" comes close, but delegates faking events to "xdotool", with weird consequences like having to restart it to direct emojis to a different window ;)
hacker perspective: It directly integrates xcb in a different event loop (here based on pselect()) and actually works – xcb is IMHO pretty great except for pretty much enforcing its own async model, internally using poll() and reading from the X server into buffers at many occassions, hidden from the consumer. If all your app does is communicate with the X server, you just happily use xcb's model. As soon as you need different things (e.g. local timers), you have a problem. The typical way to address that is dedicating another thread to xcb (AFAIK also done that way by toolkits offering an X11 backend). I hated that idea enough to be stubborn about doing it directly in my main thread, and with quite some trial-and-error, finally succeeded.
Not sure you really mean the compose mechanism provided by Xlib (and xkbcommon etc ...)? If so, this wouldn't work for all these emojis that are represented by grapheme clusters, ZWJ-sequences or are just "qualified". The result of a compose sequence is limited to a single "key symbol", which can only encode a single unicode codepoint.
Even for those emojis consisting of a singe codepoint, it would really miss the point. There are LOTS of emojis available, an emoji keyboard (or emoji picker) shows them organized in groups and allows searching for them.
> If so, this wouldn't work for all these emojis that are represented by grapheme clusters, ZWJ-sequences or are just "qualified". The result of a compose sequence is limited to a single "key symbol", which can only encode a single unicode codepoint.
I use the compose key as an abbreviation expander among other things. So far this works fine in all applications.
Though I don’t know if we’re talking about something beyond typing the XCompose sequence on the keyboard. If we’re talking about sending programmatic keypresses to applications like in this submission then I don’t know.
Thanks for the pointer, this made me have another closer look at xkbcommon's API. Probably not too relevent for the tool I'm currently writing, but I'll keep that in mind: Instead of a key symbol, you can also obtain an utf-8 encoded string. Nice.
Still, in practice, it will only work with some applications...
i enter the five emojis i ever need to use by pressing <multikey>+u and then typing the emoji word. i think i can use skin color thumbs up just fine.
but yeah, no search. but also no extra windows. when i asked i assumed compose + something like an IME could solve both with the search/window being optional.
Xcompose only offers sequences started with the "compose" key (whatever it is mapped to) and resulting in a single X key-symbol, so to type an emoji needing multiple uniciode codepoints that way, you'd have to enter multiple "compose sequences". Whatever you describe here is not simple Xcompose. AFAIK, GTK offers entering unicode codepoints in a similar way ...
Of course you can do a lot of things with "input methods" (like popping up some "picker" only when wanted). But then you'll depend on X clients being aware of them in some way, e.g. by implementing the XIM protocol. Lots of applications have that (even xterm), so it might be what you prefer and that's fine. My method with faked key press events is definitely hacky, but will work with any X client, that's the point of it ;)
Indeed. In my experience you can use whatever mappings you want. Just grab a writing script that you never type in and go ham : `<Ethiopian_B> …` (Note: made up keysym).
Are there any major apps that don't support XIM? An IME is required for CJK languages, so there shouldn't be apps that meet (in common use && have CJK users && not supporting IME) criteria.
Implementation is beyond my ability, but if en-US locale were to finally adopt IME, I think it should be semi-trivial to include an optional dictionary file trained with emoji :tofu_on_fire: notation.
(or, as it's spelled through my .XCompose, shift-altgr 3 3)
I think there was a time when compose sequences could only emit a single character, at least in some contexts. But as far as i can tell, every text box in every app on my machine supports multi-character sequences.
One of the major flaws in X11 is its poorly designed keyboard input system. When a key is pressed, the keypress event sends a "keycode" - an 8-bit number that references the current layout. This means you're limited to injecting characters that are present in the current layout.
The implications of this design are frustrating. For instance, if you're connecting to a remote system via VNC and the client and server have different keyboard layouts, you'll run into all sorts of issues. Similarly, how do you create an on-screen keyboard that can inject keypresses for characters not available in the current layout? And what if you want to programmatically send some text, but the user has the wrong layout active? It's a mess.
I guess because of this in Debian most of characters in on-screen keyboard didn't work (they "fixed" it by showing only characters present in active layouts instead of fixing the root issue).
The common workaround for this is an ugly hack: you modify the keyboard layout, find unused spots, add desired characters, send key press event and restore the keyboard layout back. See the code: [2]
Also, Wayland which was supposed to get rid of legacy problems, seems to have inherited this ugly design. Also, there seem to be no sane API for managing layouts or switching them programatically, or subscribing to layout change event. Also, you cannot use modifiers like Ctrl to switch layouts because then combinations like Ctrl + C stop working. Keyboard APIs on Linux are broken in the worst way possible since beginning, probably because most developers use only ASCII and do not have experience using multiple layouts.
A better idea would be to allow to send arbitrary Unicode strings and maybe integrate regular input and IME input (input system for typing Asian characters).
> Also, there seem to be no sane API for managing layouts or switching them programatically, or subscribing to layout change event
X11 has XkbMapNotify/XkbStateNotify.
Wayland has a wl_keyboard.keymap event.
> A better idea would be to allow to send arbitrary Unicode strings and maybe integrate regular input and IME input (input system for typing Asian characters).
Not particularly, the difficulty here is that some clients want text input, and some clients really do want key events (e.g. think games where holding W does not really have much to do with the Unicode code point 'w'). This was discussed for a long time, and the current design was decided as the best option.
IME systems do exist and already work just fine; they are integrated client-side. IME systems cannot really be integrated into the protocol since many of them involve custom UI.
> Not particularly, the difficulty here is that some clients want text input, and some clients really do want key events
I understand this point. For this case it makes sense to send both key code (what character does the key map in Latin layout) and translated code (which characters will be printed when key is pressed in current layout). It seems an easier solution than broadcast large structures to every client and let them have each own implementation for translating codes.
Regarding IME, I meant not integrating IME client into a Wayland server but instead unify the API and events that IME uses to insert text with API and events used to notify about regular keypresses.
> Similarly, how do you create an on-screen keyboard that can inject keypresses for characters not available in the current layout?
I switched the keyboard layout on the fly, on key press, if needed. That works... mostly. Chromium and Chromium-based apps know better what layout I am using, so they will misinterpret some inputs despite having a key map already.
And then you realize that you can't use a physical keyboard at the same time, because key maps go out of sync while keys are pressed on both. I talked to a Wayland dev about having separate keyboards with separate layouts, but the answer was basically "it's an incompatible change, and it's too late to fix this" (it was in an issue tracker, but no link). So the only way to have a non-input-method on-screen keyboard is to limit yourself artificially to the current layout. Which, of course, is an oft requested feature I will never implement.
> A better idea would be to allow to send arbitrary Unicode strings and maybe integrate regular input and IME input (input system for typing Asian characters).
Isn't Mac OS do something like that? I agree this is the way to go. But the stumbling block is - again - that applications like Chromium won't implement this. I created the text-input-v3 protocol some 4 years ago, and it's still basically only used in GNOME.
But with new funding from NLNet I'm gathering a special ops team to push input methods again this year :)
> most developers use only ASCII and do not have experience using multiple layouts.
I'm getting that impression as well after discussing the topic of internationalization on Mastodon: using languages other than English is undervalued by open source devs. I mean, how often do you find variables named in Spanish or Russian in open source software? It's a very anglocentric bubble.
> I'm getting that impression as well after discussing the topic of internationalization on Mastodon: using languages other than English is undervalued by open source devs. I mean, how often do you find variables named in Spanish or Russian in open source software? It's a very anglocentric bubble.
Why is that a good thing? I get the idea that a common language is beneficial, but the flip side is the knowledge and effort of the people who know another language. That's lost due to never being opened (I guess that's more of an indictment of the open source community being not interlinguistic).
So, let's look at this group of people knowledgeable in programming et al, but not english. Sure, this group exists. It's relatively tiny though. English is the language in any (modern, IOW except where it didn't replace latin yet) science, including CS. The vast majority of other on-topic literature is english. It's hard to learn a decent amount of stuff concerning programming without knowing english.
You can't randomly mix languages in source code, and any other choice of common language than english would exclude a LOT more people from the project.
All of this isn't related to i18n/l10n of your application at all. People not knowing english is a much more relevant factor when talking about user interfaces. I actually plan to localize my Xmoji tool eventually, it's just postponed until more important stuff is done (I mean, I assume most people would get together enough english to be able to use an emoji keyboard after all, but of course, especially seaching would benefit from l10n).
I guess my initial reaction was wrong: not having code in non-English languages doesn't accurately represent developer sentiments. There's a lot of translation efforts in open source, but again, this is not a good proxy for the sentiment because we don't know how many translators (who care about non-English) set project direction and design protocols.
Still, an anglocentric bubble diminishes internationalization, and I disagree that it's a good thing.
For Wayland, keyboard input is "out of scope". (edit: Not entirely, just verified it does forward very basic events, but it's a thing wayland doesn't want to handle, while X11 originally had its own keyboard/mouse/etc drivers) What's typically used is XKB (inherited from X11) with a different backend ([edit: raw events]). So yes, in practice, you'll have the same broken design.
> Also, there seem to be no sane API for managing layouts or switching
> them programatically
Layouts only exist as "data" as far as the X server is concerned, clients must fetch them and map themselves from the key codes. Libraries like xkbcommon (or even grandfather Xlib) do the job for you. That said, there are APIs to modify the mapping (by publishing messages/events) as you wish. The ugliness is, apart from the fact that you're forced to fiddle with the mapping at all, that you can't guarantee another client will process everything in sequence. It might apply a new mapping before processing all its queued key press events. That's why my code adds delays between fiddling with the mapping and sending the events.
Yes, it's extremely ugly. Still, at least for me, it works. Try it out ;)
And yes, Windows is doing better here, there's a Unicode-flavor of keyboard events available.
It's funny at the end he says "and don't ever suggest me I use Linux, just don't." I felt he expected it to be really hard project on Linux, while iterally his project is a single keyboard layout file.
On Gnome there is the "Characters" application which is pretty handy. I do want to make a few improvement to search, but otherwise it's pretty good. You copy the unicode character to the clipboard and can then paste it wherever you need.
Neat, thanks for sharing! This just replaced "Characters" for me.
Seems really well done, and most importantly the search works well and it has "recents". I do wish it wouldn't automatically close itself after clicking/copying an emoji, but I try not to let perfect be the enemy of good :-)
I'm very glad people made the push! My configuration has been much nicer being based on Wayland for the last 4 years or so than it was on X. Screentearing and limited refresh rates on mixed Hz setups are now a thing of the past :)
By that measure we'd still be using DOS these days (which was also productive and usable... and indeed the initial backlash against this nrefangled "Windows 95" thing kept going for a while, not very dissimilar to the X11 vs Wayland debates)
I mean you're free to fork and continue developing X11; right now there is nobody with both the capability and the desire to do so.
I'd wager that once I get hardware made in 2024, Wayland may work well for me (though in its defense it does work fine on my one machine with an Intel integraded GPU), but for now none of my (very old) discrete GPUs work reliably with Wayland, with 2 GPUs and 3 drivers (nvidia vs nouveau for my old GeForce and "radeon" (not amdgpu) for my old AMD card) causing 3 symptoms:
1. Crashes immediately on login
2. Black Screen
3. Kind-of sort-of works, but sometimes the screen just freezes for no reason and sometimes switching VTs fixes it sometimes not.
The era of a single machine is over. We need remote rendering for services on datacenter fleets without GPUs, so X11 is more often replaced by Javascript for a browser (with support for a user's local GPU) than by Wayland.
x11 is depreciated. It has no active maintainers and barely even qualifies for "maintenance mode" status; the push to remove Xorg can be justified by enumerating the security issues and nothing else.
Strictly speaking Linux is "productive and usable" with nothing but a terminal multiplexer and a shell to work with. With expectations as high as they are in 2024, I don't think former Windows or Mac users will feel at-home with an x11 session. Switching away from bazaar-style software development is a prerequisite for the Year of the Linux Desktop.
Cathedral-style development doesn't necessarily mean closed-source, but instead reflects the less-modular nature of Wayland in relation to x11. There aren't multiple desktops that are all using the same display server; instead each desktop implements it themselves around a common spec. Plug-and-play software has fewer and more restrictive interfaces to rely on. Modern desktop Linux is decidedly pared-back, which is a good thing when you consider how scarily open Linux is in the right hands.
"sole advantage" isn't correct either - there's a plethora of reasons to use Linux. In the enterprise, people pay companies money to keep their Linux away from bazaar-level patches and randomly packaged repos. More casually, a lot of people don't use desktop Linux for a particularly advanced purpose and just treat it like a Mac/Windows/Chrome machine with fewer advertisements. Some people do very much get a lot of value out of the bazaar-side of Linux, but the comparison between the two styles wouldn't exist at all if Linux didn't entertain both philosophies.
Building upon 40+ year old, stable technology that, compared to Wayland, still seems to be in popular demand [1, 2] is the only sensible thing for the lazy maintainer/dev. That's how I felt when I wrote X11-compatible EF*CK emoji keyboard [3].
I just use IBus, since I was already using it for Japanese IME input. It has its own emoji picker but has been kinda buggy before, instead I just use the "Typing Booster" keyboard option. I can type "wolf" and it'll popup a menu including the emoji or related emojis; I guess "cat:" is a better example since there's a lot more (e.g. shows tiger eventually) and I can scroll through them.
Also, I am curious: why in X11 and Wayland keyboard code translation happens on the client side? Wouldn't it make more sense if the layout was stored only on the server and server did the translation? There would be no need to broadcast layout changes, for example.
I had thought to design the window system and operating system, that the keyboard manager will handle the keyboard translation and can be set to several modes by application programs. Only the server knows the keyboard layout, although clients can tell the keyboard translation modes:
- Text input with large character sets (Extended TRON Code)
- Text input with small character sets (the application program can select which one (although it is not guaranteed to be available, unless the user installs an appropriate keyboard layout); e.g. APL character set)
- Command mode
- Game mode
- Hybrid mode
This way it can handle IME as well, and also specialized layouts for purposes (such as APL programming), and can handle nearly raw keyboard events when desired (although, for purposes of security and others, some combinations might still be handled by the operating system or by the window manager).
I don't think you need translation modes; you could simply send both raw events and translated events simultaneously and let client choose what it needs.
I do not agree. Translation modes will be necessary in order to handle IME properly, as well as for handling specialized character sets. (It may also mitigate some types of spying, possibly.)
The only thing I can think of is that some clients might want raw keyboard events. It does seem to me (an outsider) like Keysyms are the better level of abstraction for input for the overwhelming majority cases, and the translation should happen as early as possible in the server, and that's even before ignoring things like the asynchronous nature of xlib making keyboard remapping &c. unreliable for sending key events outside of the primary keymap.
But you can send both raw key codes and translated character code and let client decide what it needs (for example, for hotkeys like Ctrl + C you will probably prefer a raw key code so that it works independent of selected input language).
> As far as I know, it's the only tool offering somewhat reliable emoji input using faked X11 keyboard events.
A few years ago I wrote https://gitlab.com/finnoleary/zhmenu (excuse the deadname in the URL there). I'm reasonably sure it partially counts? I'd meant to build an emoji database for it but never got around to it.
Is there an issue with Wayland for your setup? I run Gnome + Wayland on Fedora, with two monitors with different sizes and different refresh rates - eerything seems to run pretty fine, including gaming?
Not OP but the last time I tried Wayland was about a year ago (also on Fedora) and I had similar issues using multiple monitors with different refresh rates, ended up switching to x11 which resolved it (using an nvidia card with nonfree drivers if that matters). But it seems I'll have to give Wayland another try if they've addressed it since then, I hadn't been keeping up with it.
We're in 2024Q3, things have improved. Nvidia helped out. (X)Wayland/KDE/Qt/GNOME bugs were fixed.
Apart from that, use X11 as long as you wish!
I'm just glad we can finally improve the open source desktop-environment infrastructure to match those of Windows and macOS: we've been stuck for too long.
Indeed. Security (all apps get all keypresses and hard to implement correct behaving lock screens) and multi-monitor with factional scaling per monitor; those were the things that apparently were to super-duper hard to fix in X11.
Maybe a years or two until Wayland gets to full feature parity (except for features Wayland specifically does not have like network support and drawing window borders etc.)
In those same two year I expect all app I use to be running native on Wayland as well. (IntelliJ being the main app I'm waiting for and they are making progress)
Any closed source app could easily contain a keylogger: great feature!
If we want people to run closed source on Linux, we need facilities to sandbox these apps like on other OSes. Flatpak is moving in this direction. Wayland's security also helps.
I think I did managed to wrangle fcitx into some reasonable emoji input in the past but at some point it broke / I lost it and configuring that tool is just an unpredictable mess...
Using input methods is the "technically best" solution of course, but it needs configuration and some application support, therefore I wanted something working with "plain X" :)
Sending fake key events is unfortunately "hacky". You need to temporarily switch the keyboard mapping, and many X applications have races between applying a new mapping and processing key press events, that's why I added (now configurable, new release soon) delays... and also, I found applications (e.g. chromium) having issues to correctly combine ZWJ sequences from individual key events, so I added more "hacks" to help with that, also just made configurable.
To sidestep these issues, my tool also offers transfer via "primary selection": Just middle-click an emoji to select it, and middle-click again where you want to insert it. Downside: Two clicks instead of just one.
It shows an overlay fuzzy-searchable list of all emojis, and it writes selected one to the input regardless of the toolkit. assign it to a keyboard shortcut and you're good to go.
the ~/.local/share/emojis/emojis file is just a list of all emojis:
cat ~/.local/share/emojis/emojis | head -n 3
Grinning Face
Grinning Face with Big Eyes
Grinning Face with Smiling Eyes
(HN is removing the emojis but each line starts with the emoji self, followed by the description)
Note input is "out of scope" for wayland and XKB is used there for keyboard input as well. I just had a quick glance at "wtype" source and indeed, it fiddles with the keymap, using xkbcommon. So, assuming a suitable X11 replacement for this "wofi" tool, you could do exactly the same on X11 using "xdotool".
You know, we should add an emojis dictionary to http://wordlist.aspell.net/ for exactly this purpose. Why not have a /usr/share/dict/emoji just like we have /usr/share/dict/words?
Unicode is OK (flawed but OK), emoji in Unicode suck.
The worst part is Unicode breaking existing documents by retroactively converting some common symbols emoji-default, despite supposed stability guarantees.
The second worst part is the emoji combining sequences becoming an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.
The goal of Unicode is to ascribe semantic, machine-readable/indexible identifiers (codepoints) to “things that appear in what are conventionally considered plaintext documents” so that “text” can be handled in a standard way across systems like screen-readers, IMEs, LLMs, search engines, etc; and so that we don’t need to depend on some particular pictorial representation / decoding of an image surviving into the future to decode it[1], because the (open, widely replicated) Unicode standard and database files encode a description for each codepoint, and a semantics for each codepoint (things like collation, joining, capitalization, etc.)
[1] Consider if your vector image representing a soldering iron is someone’s IP, and they rescind licensing for use/redistribution of it. Poof goes all the (legal) copies of your emoji, leaving future historians scratching their heads about what was supposed to be there in the text, and what meaning it contributed. (A concrete case of this actually happening — though not with vector images, but rather stock sound effects: https://roblox.fandom.com/wiki/Roblox_death_sound)
What the GP is complaining about isn't Unicode in general but Unicode Emoticons[1]. Characters can represent any thought, but emojis represent only what the committee agrees to, and the committee seems quite political.
What they should have done is to just have two characters <START_EMOTICON> and <END_EMOTICON>. And you could have text like:
I'm not doing it <START_EMOTICON>pouting<END_EMOTICON>
If the renderer supports the "pouting" emoji it would replace the text and if it doesn't it would just render:
I'm not doing it *pouting*
Everyone would be free to create emojis. You could pick your own emoji provider. So if the emoji doesn't exist locally it would be fetched from `http://emojiprovider.tld/pouting`. If you don't like it you can install another one.
It's ridiculous that there's no "pouting" emoji but there 6 emojis for pregnant men.
[1]: Yes, they're called emoticons in Unicode, not emojis. The term emoji entered English later.
Ya let's integrate gpu-accelerated emoji generation into unicode and require it everwhere, we can call it UTF-8B and standardize on 8GiB per character until that turns out not to be enough.
UTF-8 is innocent here, it's just a (very clever and useful) encoding of Unicode. The problem is adding junk codepoints based on current political ideology, not how they are encoded.
If you use a consortium governed character set, you get the problems of government by consortium. Probably still better to have a versioned universal set than so many to choose from.
Maybe ability to turn skin color brown? Or maybe the poster is mad that some country flag was or wasn't included? The only other distant possibility is how some character sets turn the gun emoji into a water pistol, but that's not on the Unicode consortium. I can't think of anything else that is political in the set.
I dont want to supply munitions to the culture war, but I would like to add a counterpoint.
Adding skin tone modifiers to emojis was a bit odd to me, since I view them as signifying emotions rather than people. Maybe that's why, of the six Fitzpatrick scale[0] skin tones they drew from, the only one not added was mine.
Similarly odd to me (to the point of appearing performative) was having a male/female/ambiguous variant on every job emoji and a separate option for every two-child two-parent family "gender" permutation. That's not how I view language as working, particularly because you're not going to be able to cover all valid families that way. It makes more sense to me, if communication rather than tokenism is the goal, to have a couple of representative emojis that convey the general concept, and then specify whatever you want about the relevant people afterward.
> you're not going to be able to cover all valid families that way
In fact the emoji committee backpedaled on family permutations for exactly this reason, and now recommends (exactly as you suggest) "symbolic" family glyphs and juxtaposition of existing people-emoji to describe families in detail.
Even the new proposal still has problems. It still has all of the variations between 1 and 2 parents and 1 and 2 children. IMHO it should all be collapsed down to just two Emoji:
1. Two people without children
2. Two people with children.
The former being short for "just the parents" or childless couples and the latter encompassing all families with kids regardless of the numbers. This is a compromise answer, but I think it serves the purpose best given the intended use of emojis.
And even better solution would be to figure out some iconography that would denote "family" without explicitly depicting the people, but I'm at a loss on that one. I mean the people are what families are all about, it's hard to divorce the concept.
user perspective: As far as I know, it's the only tool offering somewhat reliable emoji input using faked X11 keyboard events. "x11-emoji-picker" comes close, but delegates faking events to "xdotool", with weird consequences like having to restart it to direct emojis to a different window ;)
hacker perspective: It directly integrates xcb in a different event loop (here based on pselect()) and actually works – xcb is IMHO pretty great except for pretty much enforcing its own async model, internally using poll() and reading from the X server into buffers at many occassions, hidden from the consumer. If all your app does is communicate with the X server, you just happily use xcb's model. As soon as you need different things (e.g. local timers), you have a problem. The typical way to address that is dedicating another thread to xcb (AFAIK also done that way by toolkits offering an X11 backend). I hated that idea enough to be stubborn about doing it directly in my main thread, and with quite some trial-and-error, finally succeeded.