Emacs & Hebrew

[Eli Zaretskii]

> From: Omer Zak <w1 at zak.co.il>
> Date: Thu, 14 Jun 2012 22:27:22 +0300
> 
> Once we start making BiDi rendering mode dependent upon nitpicking
> details of the particular text displayed in a buffer, it is a losing
> game.  There are so many special cases, you are bound to lose some
> pathological corner cases.

You are, in effect, saying that there's no hope for Emacs to support
programming languages, something it already does quite well.  Because
the same machinery that is used now to find comments and strings,
fontify them correctly, and support various specialized commands for
them -- the same machinery is what is needed to reorder those same
comments and strings.

Maybe I'm missing something, but then please give specific examples
why you think this is a losing game.

> And According to Behdad Esfahbod, the very Unicode BiDi algorithm itself
> fails to correctly handle all kinds of corner cases in rendering Arabic
> text.

Irrelevant.  The issue here is how to reorder comments and strings in
a program source code without getting jumbled display due to
punctuation characters surrounding those comments and strings, which
are not part of the comments/strings.  If the current UBA cannot
handle some script well enough, then this pertains to reordering the
body of the comment/string itself, and fixing that is a separate issue
that should be pursued with the Unicode Consortium, not with Emacs
developers.

> How, for example, should we handle a Perl code snippet having Nadav
> Har'EL's example, which is embedded in Hebrew text (say, a chapter in a
> Perl textbook written in Hebrew)?

By putting text properties on each portion of that text guiding
the reordering engine which portions to reorder and with what base
embedding direction.  Typically, a textbook with embedded code
snippets has those snippets marked by some markup, like @code.. at end code
in Texinfo; these can be used to place the text properties as required.

In any case, even if the use case you describe is hard to handle, it
doesn't yet mean that we shouldn't handle simpler cases.  Emacs is a
programmer's editor, so rendering correctly a program source code is
something that it should do well, even if it has problems with code
embedded in plain text.  If MS Studio does it, it would be a shame if
Emacs didn't, don't you think?

> The solution that I propose is:
> - Turn off BiDi by default in all programming language major modes.

That doesn't provide solution for displaying comments and strings in a
legible form.

> In both cases, provide an easy way to display a marked text snippet in
> the opposite BiDi rendering mode.

Isn't that what I describe above?  And if so, what are we exactly
arguing about? is that about who marks the text to be reordered, where
I think Emacs should know that automagically, while you want to
place that burden on the user?

> - When the default is to turn off the BiDi mode, the display of text
> after BiDi rendering can be an uneditable pop-up window.
> - When the default is to turn on BiDi mode, then when the user wants to
> see the text rendered in logical ordering (without BiDi), he should be
> able to edit it in this mode (and with an easy way to insert
> directionality modifying/overriding special characters) - I expect it to
> be used to clean up places where the BiDi rendering engine messed up the
> text.

These are separate features, not directly related to display of
program source code.  They can be easily implemented, if the consensus
is that they are needed, since the infrastructure for that already
exists in Emacs.  By contrast, selectively reordering portions of a
buffer is not yet possible, so that is where my work will happen in
the near future.