Call for testers: the features/str-metrics branch

[Jean-Marc Lasgouttes]

Hello,

I finally think that the str-metrics branch is ready for a first round
of testing. I would like to know everything that does not work
especially (but not limited to) RTL writing.

Once this preliminary testing has been done, I'll remove a truckload of
dead code (testing and code that I think is now not needed), rebase the
branch properly and propose it for inclusion.

One new feature of the branch is that all the hand-made handling of
Hebrew and Arabic characters is now unused. This is good news for Urdu
or other scripts.

If you do not try it out now, the risk is that you'll discover issues
when it lands on master :)

Feel free to ask if you are not sure of how to test the branch.

JMarc

[Richard Heck]

Post by Jean-Marc Lasgouttes
Hello,
I finally think that the str-metrics branch is ready for a first round
of testing. I would like to know everything that does not work
especially (but not limited to) RTL writing.
Once this preliminary testing has been done, I'll remove a truckload
of dead code (testing and code that I think is now not needed), rebase
the branch properly and propose it for inclusion.
One new feature of the branch is that all the hand-made handling of
Hebrew and Arabic characters is now unused. This is good news for Urdu
or other scripts.
If you do not try it out now, the risk is that you'll discover issues
when it lands on master :)
Feel free to ask if you are not sure of how to test the branch.

Is this otherwise up to date as far as the latest file format is
concerned, etc?

Richard

[Jean-Marc Lasgouttes]

Post by Richard Heck
Is this otherwise up to date as far as the latest file format is
concerned, etc?

Yes, I merged recently with master.

JMarc

[Benjamin Piwowarski]

Hi,

I was able to build LyX (after a merge with master) and everything seem to work (did not try with RTL writing). What should we pay attention to while testing?

benjamin
On 30 Apr 2014 at 18:37:36 , Jean-Marc Lasgouttes (***@lyx.org) wrote:

Hello,

I finally think that the str-metrics branch is ready for a first round
of testing. I would like to know everything that does not work  
especially (but not limited to) RTL writing.

Once this preliminary testing has been done, I'll remove a truckload of
dead code (testing and code that I think is now not needed), rebase the
branch properly and propose it for inclusion.

One new feature of the branch is that all the hand-made handling of
Hebrew and Arabic characters is now unused. This is good news for Urdu
or other scripts.

If you do not try it out now, the risk is that you'll discover issues
when it lands on master :)

Feel free to ask if you are not sure of how to test the branch.

JMarc

[Jean-Marc Lasgouttes]

Post by Benjamin Piwowarski
Hi,
I was able to build LyX (after a merge with master) and everything seem
to work (did not try with RTL writing). What should we pay attention to
while testing?

Everything about display and editing on screen; If I had more precise
ideas, I would have tested them ;)

JMarc

[Scott Kostyshak]

On Wed, Apr 30, 2014 at 12:36 PM, Jean-Marc Lasgouttes

Post by Jean-Marc Lasgouttes
Hello,
I finally think that the str-metrics branch is ready for a first round of
testing. I would like to know everything that does not work especially (but
not limited to) RTL writing.

It works well for me and the only (very minor) thing I've noticed so far is:
When I select an ERT box, the text inside seems to jump a little bit.
Here is a screencast of master built at 0c0e16c6:
https://www.dropbox.com/s/co98z262kwi0l6r/master-2014-04-22.ogv
(you should see no jumping)
And here is the screencast doing the same thing but with the str-metrics branch:
https://www.dropbox.com/s/i9x9ab060p1rry5/str-metrics.ogv
(here is where I see jumping).

Can you reproduce?

Scott

[Jean-Marc Lasgouttes]

Post by Scott Kostyshak
When I select an ERT box, the text inside seems to jump a little bit.
https://www.dropbox.com/s/co98z262kwi0l6r/master-2014-04-22.ogv
(you should see no jumping)
https://www.dropbox.com/s/i9x9ab060p1rry5/str-metrics.ogv
(here is where I see jumping).

I'll have a look.

JMarc

[Jean-Marc Lasgouttes]

Post by Scott Kostyshak
When I select an ERT box, the text inside seems to jump a little bit.
https://www.dropbox.com/s/co98z262kwi0l6r/master-2014-04-22.ogv
(you should see no jumping)
https://www.dropbox.com/s/i9x9ab060p1rry5/str-metrics.ogv
(here is where I see jumping).

I did not manage to reproduce. Do you have a file to try is out?

JMarc

[Scott Kostyshak]

Post by Jean-Marc Lasgouttes

Post by Scott Kostyshak
When I select an ERT box, the text inside seems to jump a little bit.
https://www.dropbox.com/s/co98z262kwi0l6r/master-2014-04-22.ogv
(you should see no jumping)
https://www.dropbox.com/s/i9x9ab060p1rry5/str-metrics.ogv
(here is where I see jumping).

I did not manage to reproduce. Do you have a file to try is out?

I will send you the file when I get home. I will also check that the
build logs are the same. I took care to use the latest commit from
master that you merged in for the comparison and the same build
script, but maybe somehow a CMake flag was added/subtracted.

Scott

[Scott Kostyshak]

Post by Scott Kostyshak

Post by Jean-Marc Lasgouttes
I did not manage to reproduce. Do you have a file to try is out?

I will send you the file when I get home. I will also check that the
build logs are the same. I took care to use the latest commit from
master that you merged in for the comparison and the same build
script, but maybe somehow a CMake flag was added/subtracted.
Scott

See attached for the example. I checked and that build logs are the
same. Attached is my output of CMake. I've also tried with my normal
preferences and with default preferences. I've also confirmed that the
output of ldd on the binaries is the same. And I reproduced the same
difference in behaviors on a different machine (although a very
similar setup).

Scott

[Jean-Marc Lasgouttes]

Post by Scott Kostyshak
See attached for the example. I checked and that build logs are the
same. Attached is my output of CMake. I've also tried with my normal
preferences and with default preferences. I've also confirmed that the
output of ldd on the binaries is the same. And I reproduced the same
difference in behaviors on a different machine (although a very
similar setup).

Strangely enough, I cannot reproduce here on an iMac. I'll have to chek
later on linux.

JMarc

[Jean-Marc Lasgouttes]

Post by Scott Kostyshak

Post by Scott Kostyshak

Post by Jean-Marc Lasgouttes
I did not manage to reproduce. Do you have a file to try is out?

I will send you the file when I get home. I will also check that the
build logs are the same. I took care to use the latest commit from
master that you merged in for the comparison and the same build
script, but maybe somehow a CMake flag was added/subtracted.
Scott

See attached for the example. I checked and that build logs are the
same. Attached is my output of CMake. I've also tried with my normal
preferences and with default preferences. I've also confirmed that the
output of ldd on the binaries is the same. And I reproduced the same
difference in behaviors on a different machine (although a very
similar setup).

This is now ticket #9114.

JMarc

[Vincent van Ravesteijn]

Post by Jean-Marc Lasgouttes
Hello,
I finally think that the str-metrics branch is ready for a first round
of testing. I would like to know everything that does not work
especially (but not limited to) RTL writing.
Once this preliminary testing has been done, I'll remove a truckload
of dead code (testing and code that I think is now not needed), rebase
the branch properly and propose it for inclusion.
One new feature of the branch is that all the hand-made handling of
Hebrew and Arabic characters is now unused. This is good news for Urdu
or other scripts.
If you do not try it out now, the risk is that you'll discover issues
when it lands on master :)
Feel free to ask if you are not sure of how to test the branch.
JMarc

The first thing I notice is that painting of some 'simple' ligatures
seems to be broken in master as well as in LyX 2.0.7. I really think
this worked before. Your branch fixes this issue.
str-metrics: master:


Next, the position of the cursor seems to be computed by cutting the
string in two parts. Because of this, the width of the character before
the cursor is computed as its width in the unconnected form. This can be
seen in the following:

Here the cursor is positioned too far to the left :
str-metrics:Master:

and here, the cursor actually moves right, when moving logically to the
left:
pos:3 pos:4

The following shows a clue on this cursor positioning.
pos: 3 pos:4

The last image shows a correct cursor position because the character
before the cursor has the same width in the unconnected form.



When selecting the first character, I see two things:

1) the connection between characters is lost (in master this is not the
case)
2) the first character actually moves to the right on selection (while
you would expect that the rest of the word moves to the left because the
first character became wider on selection)

str-metrics: master:

Vincent

[Jean-Marc Lasgouttes]

Post by Vincent van Ravesteijn
1) the connection between characters is lost (in master this is not the
case)
2) the first character actually moves to the right on selection (while
you would expect that the rest of the word moves to the left because the
first character became wider on selection)

OK, I have an idea for having correct selections without loosing the
Color_selectiontext enum: we could draw the complete string as selected
and non-selected, but use clipping to make sure that only the right part
of the selection is visible. It will be a bit tricky, but it is doable.

However, the question is: do we want to keep this Color_selectiontext
thing? It makes all selection lose color, which is not very helpful.
Therefore, we could just decide to forget about that and just use
Color_selection for the background. Actually, I haven't found any
wordprocessor-like application that uses a selection background color.

Pavel, Enrico, as users of weird color schemes, I need your input on that.

The bigger problem will be cursor positioning, but I need more
information from people who understand Arabic writing to progress on that.

I just noticed that we have no Arabic document (just a farsi splash.lyx)
in our doc. This is not very encouraging. Do we even have Arabic writers
as users?

[Enrico Forestieri]

Post by Jean-Marc Lasgouttes
However, the question is: do we want to keep this
Color_selectiontext thing? It makes all selection lose color, which
is not very helpful. Therefore, we could just decide to forget about
that and just use Color_selection for the background. Actually, I
haven't found any wordprocessor-like application that uses a
selection background color.

Well, all other wordprocessor-like applications also have a boldify button
and many other things LyX is currently lacking ;)

Post by Jean-Marc Lasgouttes
Pavel, Enrico, as users of weird color schemes, I need your input on that.

I like very much the way LyX shows a selection in contrast to any other
similar application, but if this is going to cause problems (I don't
know what you mean by "all selection lose color" but, if it is what I
think, it actually is what I experience when I try 'Use system colors')
I am not going to insist... so much ;)

[Jean-Marc Lasgouttes]

Post by Enrico Forestieri
Well, all other wordprocessor-like applications also have a boldify button
and many other things LyX is currently lacking ;)

To say things differently, I searched without success for an application
that uses a specific text color for selection _and_ is able to handle
ligatures/kerning properly. Just try the str-metrics branch and play
with selection and you will understand what the problem is.

Post by Enrico Forestieri
I like very much the way LyX shows a selection in contrast to any other
similar application, but if this is going to cause problems (I don't
know what you mean by "all selection lose color" but, if it is what I
think, it actually is what I experience when I try 'Use system colors')
I am not going to insist... so much ;)

When one selects in LyX; no matter whether there is changed text, maths,
or whatever, suddenly all the text changes to black (or white). I find
that this is a loss of information.

Actually I think I have found a way to keep our ugly selection colors
and avoid dancing text. But it will require some time to experiment. I
will maybe propose an alternative solution where selection text is the
same as the normal text color.

JMarc

[Enrico Forestieri]

Post by Jean-Marc Lasgouttes

Post by Enrico Forestieri
Well, all other wordprocessor-like applications also have a boldify button
and many other things LyX is currently lacking ;)

To say things differently, I searched without success for an
application that uses a specific text color for selection _and_ is
able to handle ligatures/kerning properly. Just try the str-metrics
branch and play with selection and you will understand what the
problem is.

Sorry, I don't know how to do that. After launching "git gui" I select
Branch->Checkout but I cannot find any str-metrics branch there.
I initially tried to learn git but then gave up. I am more proficient
with applying and removing patches than messing with git branches.
Every time I tried the "git way" I found myself spending time in
solving problems that I would not have had.

Post by Jean-Marc Lasgouttes

Post by Enrico Forestieri
I like very much the way LyX shows a selection in contrast to any other
similar application, but if this is going to cause problems (I don't
know what you mean by "all selection lose color" but, if it is what I
think, it actually is what I experience when I try 'Use system colors')
I am not going to insist... so much ;)

When one selects in LyX; no matter whether there is changed text,
maths, or whatever, suddenly all the text changes to black (or
white). I find that this is a loss of information.

But the text is already black, right? ;)

Post by Jean-Marc Lasgouttes
Actually I think I have found a way to keep our ugly selection
colors and avoid dancing text. But it will require some time to
experiment. I will maybe propose an alternative solution where
selection text is the same as the normal text color.

Seriously, I don't want to waste your time if that is so difficult,
but you would have all my appreciation if you succeed ;)

[Scott Kostyshak]

Post by Enrico Forestieri

Post by Jean-Marc Lasgouttes

Post by Enrico Forestieri
Well, all other wordprocessor-like applications also have a boldify button
and many other things LyX is currently lacking ;)

To say things differently, I searched without success for an
application that uses a specific text color for selection _and_ is
able to handle ligatures/kerning properly. Just try the str-metrics
branch and play with selection and you will understand what the
problem is.

Sorry, I don't know how to do that. After launching "git gui" I select
Branch->Checkout but I cannot find any str-metrics branch there.
I initially tried to learn git but then gave up. I am more proficient
with applying and removing patches than messing with git branches.
Every time I tried the "git way" I found myself spending time in
solving problems that I would not have had.

Taken from http://wiki.lyx.org/Devel/LyXGit :

git remote add features ***@git.lyx.org:features.git (read/write
access for developers)
git remote add features git:features.git (anonymous access for users)
Use git fetch features (or git fetch --all) to get updates to the
feature branches.

Then you should be able to checkout str-metrics.

Scott

[Enrico Forestieri]

Post by Scott Kostyshak
access for developers)
git remote add features git:features.git (anonymous access for users)
Use git fetch features (or git fetch --all) to get updates to the
feature branches.
Then you should be able to checkout str-metrics.

Thanks. However, I have a bad recollection of git branch dances.
I prefer using the following if it achieves the same effect:

git clone ***@git.lyx.org:features.git lyx-features
cd lyx-features
git checkout str-metrics

[Enrico Forestieri]

Post by Jean-Marc Lasgouttes
To say things differently, I searched without success for an
application that uses a specific text color for selection _and_ is
able to handle ligatures/kerning properly. Just try the str-metrics
branch and play with selection and you will understand what the
problem is.

I just tried it. I loaded an old document and searched for ligatures
such as "ff" or "fi" but everything I found does not seems to be a
ligature and I don't see any of the reported problems.

The text changes color only when "Use system colors" is selected, but
it suffices not doing that for avoiding it ;)

Even in this respect I don't see any difference with master, if not
for the fact that in the str-metrics branch the selection seems to
be slightly lagging behind the cursor.

Out of curiosity I then tried libreoffice and noticed that the selection
is very similar to the LyX way when "Use system colors" is not checked.

I used Qt 5.2.1 but I don't think it makes any difference.

[Jean-Marc Lasgouttes]

Post by Enrico Forestieri

Post by Jean-Marc Lasgouttes
To say things differently, I searched without success for an
application that uses a specific text color for selection _and_ is
able to handle ligatures/kerning properly. Just try the str-metrics
branch and play with selection and you will understand what the
problem is.

I just tried it. I loaded an old document and searched for ligatures
such as "ff" or "fi" but everything I found does not seems to be a
ligature and I don't see any of the reported problems.

Try to use a modern font like Dejavu Serif, or maybe Garamond on
windows. Do you see some kerning? For example, in the string "AV", the
bottom of the A should overlap with the top of the V. In such a case,
when a selection ends between the A and the V, two strings have to be
drawn separately and thus the kerning is broken. This causes the part
starting with V to nudge a bit to the right.

Post by Enrico Forestieri
The text changes color only when "Use system colors" is selected, but
it suffices not doing that for avoiding it ;)

Actually, the text always changes color. In you case it changes from
balck to black. But math changes from blue to black. Change tracking
becomes black too. ERT text becomes black too. Do you see what I mean?

Post by Enrico Forestieri
Even in this respect I don't see any difference with master, if not
for the fact that in the str-metrics branch the selection seems to
be slightly lagging behind the cursor.

Interesting, I'll have a look. I have not done any profiling yes. Are
you sure that the lag is worse than with master?

Post by Enrico Forestieri
Out of curiosity I then tried libreoffice and noticed that the selection
is very similar to the LyX way when "Use system colors" is not checked.

Up to a certain point (see above), yes.

Post by Enrico Forestieri
I used Qt 5.2.1 but I don't think it makes any difference.

Probably not indeed.

Thanks for trying it out.

JMarc

[Enrico Forestieri]

Post by Jean-Marc Lasgouttes

Post by Enrico Forestieri
I just tried it. I loaded an old document and searched for ligatures
such as "ff" or "fi" but everything I found does not seems to be a
ligature and I don't see any of the reported problems.

Try to use a modern font like Dejavu Serif, or maybe Garamond on
windows. Do you see some kerning? For example, in the string "AV",
the bottom of the A should overlap with the top of the V. In such a
case, when a selection ends between the A and the V, two strings
have to be drawn separately and thus the kerning is broken. This
causes the part starting with V to nudge a bit to the right.

So, this depends on the font, apparently. Now I see both kerning and
ligatures when using Dejavu Serif (even on Linux). However, placing
the cursor between 'A' and 'V' does not modify the kerning, as well
as placing the cursor between 'f' and 'i' does not break the ligature.
This occurs only when selecting. Thus, when extending the selection,
mimicking what is done when placing the cursor should avoid those
artifacts (but I am speaking out of ignorance here, admittedly).

Post by Jean-Marc Lasgouttes

Post by Enrico Forestieri
The text changes color only when "Use system colors" is selected, but
it suffices not doing that for avoiding it ;)

Actually, the text always changes color. In you case it changes from
balck to black. But math changes from blue to black. Change tracking
becomes black too. ERT text becomes black too. Do you see what I mean?

It seems we tend to describe what happens in our own world. In my world
"Use system colors" is never checked, while in your world it is always
checked. Now, in my world, nothing of what you describe above happens.
Math, change tracking and ERT maintain their colors when selecting.
The only text that changes to black is the text whose color is manually
changed. However, if you don't change color for every word you write,
this is barely a problem.

Now I think I understand what you are trying to accomplish. You are
trying to make your world more similar to mine. If this is the case,
I would have no problem in migrating, because those differences were
the primary cause of my dislike for "Use system colors". However, I
would like to warn you of the fact that the background of a selection
is system dependent. For example, on Solaris I get a hideous dark blue
and the text would be illegible if it remains black. Maybe this is why
the colors are changed when the system ones are followed.

You know better than me the technical problems caused by not checking
"Use system colors", but I would say that it is the cause of your other
problems. The above (unknown to me) technical problems could maybe
solved more easily by ditching that checkbox ;)

Post by Jean-Marc Lasgouttes

Post by Enrico Forestieri
Even in this respect I don't see any difference with master,

Though, I just noticed an ugly difference with respect to LyX 2.0.
The background of math remains white when selecting. This does not
happen in 2.0, where the math background is set to the right color.
What changed here?

Post by Jean-Marc Lasgouttes

Post by Enrico Forestieri
if not for the fact that in the str-metrics branch the selection seems
to be slightly lagging behind the cursor.

Interesting, I'll have a look. I have not done any profiling yes.
Are you sure that the lag is worse than with master?

It seems that the lag is caused by Qt. With Qt4 there is no lag at all
and, however fast I move the cursor, the selection follows instantly.
With Qt5 there is a noticeable lag and it seems like playing to that
old game where you have a growing snake following your moves...

[Jean-Marc Lasgouttes]

Post by Enrico Forestieri
So, this depends on the font, apparently. Now I see both kerning and
ligatures when using Dejavu Serif (even on Linux). However, placing
the cursor between 'A' and 'V' does not modify the kerning, as well
as placing the cursor between 'f' and 'i' does not break the ligature.
This occurs only when selecting. Thus, when extending the selection,
mimicking what is done when placing the cursor should avoid those
artifacts (but I am speaking out of ignorance here, admittedly).

Placing the cursor is just a matter of drawing a blinking thing at some
position. When there is a selection, however, the selection text is
painted with its own color. This has been like that for ages. The only
difference with the "system colors" thing you love to hate is that there
is a different set of colors, and thus different things happen. But the
code is the same.

I would think that this code with a different selected text color has
been coded from the start by Matthias to allow for a monochrome mode, at
the time when running on a Sun3/60 made sense.

Post by Enrico Forestieri
It seems we tend to describe what happens in our own world. In my world
"Use system colors" is never checked, while in your world it is always
checked. Now, in my world, nothing of what you describe above happens.
Math, change tracking and ERT maintain their colors when selecting.

I agree about math, in 2.1, although I thought that it changed. But try
again with ERT, I doubt that it stays red.

Post by Enrico Forestieri
Now I think I understand what you are trying to accomplish. You are
trying to make your world more similar to mine.

No, I am just trying to make the current code work in a word where we
respect ligatures and kerning. This is completely irrelevant to your
nightmares about colors. The fact is that, if I want two pieces of the
same word in different colors, I have to break kerning or break a
ligature. If the word is in arabic, it can have very important consequences.

Post by Enrico Forestieri
Though, I just noticed an ugly difference with respect to LyX 2.0.
The background of math remains white when selecting. This does not
happen in 2.0, where the math background is set to the right color.
What changed here?

I do not know. I think we have a bug, but is is not related to my
str-metrics branch AFAIK.

Post by Enrico Forestieri
It seems that the lag is caused by Qt. With Qt4 there is no lag at all
and, however fast I move the cursor, the selection follows instantly.
With Qt5 there is a noticeable lag and it seems like playing to that
old game where you have a growing snake following your moves...

That's not good.

JMarc

[Enrico Forestieri]

Post by Jean-Marc Lasgouttes

Post by Enrico Forestieri
So, this depends on the font, apparently. Now I see both kerning and
ligatures when using Dejavu Serif (even on Linux). However, placing
the cursor between 'A' and 'V' does not modify the kerning, as well
as placing the cursor between 'f' and 'i' does not break the ligature.
This occurs only when selecting. Thus, when extending the selection,
mimicking what is done when placing the cursor should avoid those
artifacts (but I am speaking out of ignorance here, admittedly).

Placing the cursor is just a matter of drawing a blinking thing at
some position. When there is a selection, however, the selection
text is painted with its own color. This has been like that for
ages. The only difference with the "system colors" thing you love to
hate is that there is a different set of colors, and thus different
things happen. But the code is the same.

Oh... I thought there were more subtle differences.

Post by Jean-Marc Lasgouttes
I would think that this code with a different selected text color
has been coded from the start by Matthias to allow for a monochrome
mode, at the time when running on a Sun3/60 made sense.

They had some colors, at least. The Mac's still had B&W displays ;)

Post by Jean-Marc Lasgouttes

Post by Enrico Forestieri
It seems we tend to describe what happens in our own world. In my world
"Use system colors" is never checked, while in your world it is always
checked. Now, in my world, nothing of what you describe above happens.
Math, change tracking and ERT maintain their colors when selecting.

I agree about math, in 2.1, although I thought that it changed. But
try again with ERT, I doubt that it stays red.

Yes, you are right, I overlooked that.

Post by Jean-Marc Lasgouttes

Post by Enrico Forestieri
Now I think I understand what you are trying to accomplish. You are
trying to make your world more similar to mine.

No, I am just trying to make the current code work in a word where
we respect ligatures and kerning. This is completely irrelevant to
your nightmares about colors. The fact is that, if I want two pieces
of the same word in different colors, I have to break kerning or
break a ligature. If the word is in arabic, it can have very
important consequences.

It seems to me we are playing a sort of "comedy of errors". I don't
want that the text changes color, you don't want that too. I am getting
confused about the fact that we are disagreeing on something...

Post by Jean-Marc Lasgouttes

Post by Enrico Forestieri
Though, I just noticed an ugly difference with respect to LyX 2.0.
The background of math remains white when selecting. This does not
happen in 2.0, where the math background is set to the right color.
What changed here?

I do not know. I think we have a bug, but is is not related to my
str-metrics branch AFAIK.

This was changed at http://www.lyx.org/trac/changeset/7765b1c9/lyxgit
and the attached patch restores the old behavior. I think it should
be applied because this particular change seems to be gratuitous.
Richard, that would also be needed for the stable branch.

[Richard Heck]

Post by Enrico Forestieri

Post by Jean-Marc Lasgouttes

Post by Enrico Forestieri
Though, I just noticed an ugly difference with respect to LyX 2.0.
The background of math remains white when selecting. This does not
happen in 2.0, where the math background is set to the right color.
What changed here?

I do not know. I think we have a bug, but is is not related to my
str-metrics branch AFAIK.

This was changed at http://www.lyx.org/trac/changeset/7765b1c9/lyxgit
and the attached patch restores the old behavior. I think it should
be applied because this particular change seems to be gratuitous.
Richard, that would also be needed for the stable branch.

Yes, thanks. I just miscopied the other change. I've applied the fix.

Richard

[Helge Hafting]

Post by Jean-Marc Lasgouttes
Try to use a modern font like Dejavu Serif, or maybe Garamond on
windows. Do you see some kerning? For example, in the string "AV", the
bottom of the A should overlap with the top of the V. In such a case,
when a selection ends between the A and the V, two strings have to be
drawn separately and thus the kerning is broken. This causes the part
starting with V to nudge a bit to the right.

The strings don't have to be drawn separately. Another way is to draw
the complete string twice, each time in different colors.

This yields two bitmaps that can be combined to a bicolor string with no
changes to kerning or ligatures. Use part of one bitmap before the
cursor position, and the other bitmap after the cursor position.
Hopefully, the windowing system can do this with clipping, so it won't
be necessary to actually handle these bitmaps.

Helge Hafting

[Stephan Witt]

Post by Jean-Marc Lasgouttes
Try to use a modern font like Dejavu Serif, or maybe Garamond on
windows. Do you see some kerning? For example, in the string "AV", the
bottom of the A should overlap with the top of the V. In such a case,
when a selection ends between the A and the V, two strings have to be
drawn separately and thus the kerning is broken. This causes the part
starting with V to nudge a bit to the right.

The strings don't have to be drawn separately. Another way is to draw the complete string twice, each time in different colors.
This yields two bitmaps that can be combined to a bicolor string with no changes to kerning or ligatures. Use part of one bitmap before the cursor position, and the other bitmap after the cursor position. Hopefully, the windowing system can do this with clipping, so it won't be necessary to actually handle these bitmaps.

Currently the drawing of LyX is not appropriate for high resolution displays.

I'm afraid your suggestion is not suitable to improve this.

Stephan

[Helge Hafting]

Post by Stephan Witt
Currently the drawing of LyX is not appropriate for high resolution displays.
I'm afraid your suggestion is not suitable to improve this.

I didn't know that. In that case, the "background only" selection seems
like a good alternative.

The trick then, will be to find a selection background color that has
sufficient contrast to any text color that has good contrast to the
normal background. So it will have to be a light color, but one that is
sufficiently different from the unselected background.

Helge

[Jean-Marc Lasgouttes]

Post by Stephan Witt
Currently the drawing of LyX is not appropriate for high resolution displays.
I'm afraid your suggestion is not suitable to improve this.

This is something unrelated to str-metrics branch, right? I agree that
we will have to fix this soon. Do we have an open bug? Is it only a
mac/retina issue, or is it also true on windows/linux hi definition
displays.

JMarc

[Stephan Witt]

Post by Jean-Marc Lasgouttes

Post by Stephan Witt
Currently the drawing of LyX is not appropriate for high resolution displays.
I'm afraid your suggestion is not suitable to improve this.

This is something unrelated to str-metrics branch, right?

Text rendering shouldn't go to bitmaps and later copied to screen. This is what I wanted to avoid.

Post by Jean-Marc Lasgouttes
I agree that we will have to fix this soon. Do we have an open bug?

There are some, but I don't know of a bug mentioning the problem with text in work area.

Post by Jean-Marc Lasgouttes
Is it only a mac/retina issue, or is it also true on windows/linux hi definition displays.

I don't know how hi-resolution on windows/linux look.

Stephan

[Jean-Marc Lasgouttes]

Post by Stephan Witt

Post by Jean-Marc Lasgouttes

Post by Stephan Witt
Currently the drawing of LyX is not appropriate for high resolution displays.
I'm afraid your suggestion is not suitable to improve this.

This is something unrelated to str-metrics branch, right?

Text rendering shouldn't go to bitmaps and later copied to screen. This is what I wanted to avoid.

This happens only with use_qimage==true, doesn't it? Can you try to set
this to false and report on correctness and performance?

Post by Stephan Witt

Post by Jean-Marc Lasgouttes
I agree that we will have to fix this soon. Do we have an open bug?

There are some, but I don't know of a bug mentioning the problem with text in work area.

Could you create one, along with some screenshots ?


JMarc

[Stephan Witt]

Post by Stephan Witt

Post by Jean-Marc Lasgouttes

Post by Stephan Witt
Currently the drawing of LyX is not appropriate for high resolution displays.
I'm afraid your suggestion is not suitable to improve this.

This is something unrelated to str-metrics branch, right?

Text rendering shouldn't go to bitmaps and later copied to screen. This is what I wanted to avoid.

This happens only with use_qimage==true, doesn't it? Can you try to set this to false and report on correctness and performance?

It makes no difference for me. This is on master. I can test it on str-metrics branch later.
But it may be the high resolution text rendering is possible with use_qimage==false only.

I know there are Windows people with pleasing results. I don't know what the difference is.
Perhaps Qt handles it differently or Windows itself...

Post by Stephan Witt

Post by Jean-Marc Lasgouttes
I agree that we will have to fix this soon. Do we have an open bug?

There are some, but I don't know of a bug mentioning the problem with text in work area.

Could you create one, along with some screenshots ?

I made one: http://www.lyx.org/trac/ticket/9130

Stephan

[Jean-Marc Lasgouttes]

Post by Stephan Witt

This happens only with use_qimage==true, doesn't it? Can you try to set this to false and report on correctness and performance?

It makes no difference for me. This is on master. I can test it on str-metrics branch later.
But it may be the high resolution text rendering is possible with use_qimage==false only.

It should be the same with str-metrics.

Post by Stephan Witt

Could you create one, along with some screenshots ?

I made one: http://www.lyx.org/trac/ticket/9130

In the url I posted there, there is some stuff about pixmaps. We may
have to try to understand what they say we shall do :)

JMarc

[Jean-Marc Lasgouttes]

Post by Helge Hafting
This yields two bitmaps that can be combined to a bicolor string with no
changes to kerning or ligatures. Use part of one bitmap before the
cursor position, and the other bitmap after the cursor position.
Hopefully, the windowing system can do this with clipping, so it won't
be necessary to actually handle these bitmaps.

Yes, I plan to do clipping. However, I need to be able to hide a part of
a string too; and I do not know how to do that.

Assume that I want to draw "abcde" and "bcd" is selected. Then I will
draw the string twice:

* first by hiding everything before b and everything after d; this is
plain clipping.

* second, I have to hide everything between b and d. This I am not sure
how to do.

JMarc

[Kornel Benko]

Post by Jean-Marc Lasgouttes

Post by Helge Hafting
This yields two bitmaps that can be combined to a bicolor string with no
changes to kerning or ligatures. Use part of one bitmap before the
cursor position, and the other bitmap after the cursor position.
Hopefully, the windowing system can do this with clipping, so it won't
be necessary to actually handle these bitmaps.

Yes, I plan to do clipping. However, I need to be able to hide a part of
a string too; and I do not know how to do that.
Assume that I want to draw "abcde" and "bcd" is selected. Then I will

Or three times?
a
bcd
e

Post by Jean-Marc Lasgouttes
* first by hiding everything before b and everything after d; this is
plain clipping.
* second, I have to hide everything between b and d. This I am not sure
how to do.
JMarc

Kornel

[Jean-Marc Lasgouttes]

Post by Kornel Benko

Post by Jean-Marc Lasgouttes
Assume that I want to draw "abcde" and "bcd" is selected. Then I will

Or three times?
a
bcd
e

That is what I want to avoid. In Arabic, for example, 'a' will may a
different shape depending on whether it is at the end of a word or
followed by a 'b'.

But the real solution would be to get rid of the Color_selectiontext
thing. Is there somebody who really thinks that we need to handle this
'reverse video' selection?

JMarc

[Vincent van Ravesteijn]

Post by Jean-Marc Lasgouttes

Post by Kornel Benko

Post by Jean-Marc Lasgouttes
Assume that I want to draw "abcde" and "bcd" is selected. Then I will

Or three times?
a
bcd
e

That is what I want to avoid. In Arabic, for example, 'a' will may a
different shape depending on whether it is at the end of a word or
followed by a 'b'.
But the real solution would be to get rid of the Color_selectiontext
thing. Is there somebody who really thinks that we need to handle this
'reverse video' selection?

I'm not sure that is the real solution. The user can always color part
of the string, or have emphasized parts or whatever.

I think the real solution then would be to use glyphs.

The function QTextLayout::glyphRuns can convert a String into glyphs
(the glyphs represent the actual characters that you see on screen, so
all arabic characters have been translated into their correct form).
Then you can draw a subset of the glyphs with QPainter::drawGlyphRun
with the correct color etc.

Vincent

[Jean-Marc Lasgouttes]

Post by Vincent van Ravesteijn
I think the real solution then would be to use glyphs.

But glyphs do not work for ff ligature, right?

I think only clipping can work (see ticket #9116).

But first, we have to get cursor positioning right (ticket #9115).

JMarc

[Vincent van Ravesteijn]

On Tue, May 6, 2014 at 11:06 AM, Jean-Marc Lasgouttes

Post by Vincent van Ravesteijn
1) the connection between characters is lost (in master this is not the
case)
2) the first character actually moves to the right on selection (while
you would expect that the rest of the word moves to the left because the
first character became wider on selection)

OK, I have an idea for having correct selections without loosing the Color_selectiontext enum: we could draw the complete string as selected and non-selected, but use clipping to make sure that only the right part of the selection is visible. It will be a bit tricky, but it is doable.
However, the question is: do we want to keep this Color_selectiontext thing? It makes all selection lose color, which is not very helpful. Therefore, we could just decide to forget about that and just use Color_selection for the background. Actually, I haven't found any wordprocessor-like application that uses a selection background color.
Pavel, Enrico, as users of weird color schemes, I need your input on that.
The bigger problem will be cursor positioning, but I need more information from people who understand Arabic writing to progress on that.
I just noticed that we have no Arabic document (just a farsi splash.lyx) in our doc. This is not very encouraging. Do we even have Arabic writers as users?

I checked this morning. What looks very helpful is:

int QFontMetrics::width ( const QString & text, int len = -1 ) const

This function takes the whole string, like "coffee", and returns the
width up to the nth character. This takes into account the ligatures.
A problem though is that a ligature is taken as a single glyph. This
means for the coffee example, that the ff ligature causes the 3rd and
4rd position to return the same value. In this case it makes sense to
position the cursor in between, but there are of course ligatures
which change the shape of two characters completely, and then putting
the cursor in the middle is strange. See the first example in my
previous mail.

So, for the arabic characters, you'll get the width corresponding to
the actual glyph in the context. So, as an example, a standalone
character might return a width 30, and when the character is connected
to the next it might return 10.

Vincent

[Jean-Marc Lasgouttes]

Post by Vincent van Ravesteijn
int QFontMetrics::width ( const QString & text, int len = -1 ) const

I read a bit the QFontMetrics API and found some interesting things we
do not use for underline and strikeout: underlinePos, strikeOutPos and
lineWidth. It is probably worth trying them out and see whether we get
an output more pleasant to the eye.

More importantly, I found a pointer to QTextLayout. Do you know a bit
about that? I guess we cannot rewrite our complete layout mechanism to
use that, but it may be possible to reuse pieces.

JMarc

[Vincent van Ravesteijn]

Post by Jean-Marc Lasgouttes

Post by Vincent van Ravesteijn
1) the connection between characters is lost (in master this is not the
case)
2) the first character actually moves to the right on selection (while
you would expect that the rest of the word moves to the left because the
first character became wider on selection)

OK, I have an idea for having correct selections without loosing the
Color_selectiontext enum: we could draw the complete string as
selected and non-selected, but use clipping to make sure that only the
right part of the selection is visible. It will be a bit tricky, but
it is doable.

In LyX 2.0.7 coloring parts of arabic strings works ok. So, I'm not sure
why there is a problem here now. Ok, ligatures that should have
different parts colored differently is a bit difficult. My feeling is
that it is ok to split the ligatures in this exceptional case. The
contextual forms in arabic though are not ligatures and can be painted
in different colors without problems.

Post by Jean-Marc Lasgouttes
However, the question is: do we want to keep this Color_selectiontext
thing? It makes all selection lose color, which is not very helpful.
Therefore, we could just decide to forget about that and just use
Color_selection for the background. Actually, I haven't found any
wordprocessor-like application that uses a selection background color.

That wouldn't allow the monochrome style.. green text on black
background, and, when it is selected, black text on green background :(.

Post by Jean-Marc Lasgouttes
Pavel, Enrico, as users of weird color schemes, I need your input on that.
The bigger problem will be cursor positioning, but I need more
information from people who understand Arabic writing to progress on that.

What info do you need ?

Vincent

[Jean-Marc Lasgouttes]

Post by Vincent van Ravesteijn

Post by Jean-Marc Lasgouttes
OK, I have an idea for having correct selections without loosing the
Color_selectiontext enum: we could draw the complete string as
selected and non-selected, but use clipping to make sure that only the
right part of the selection is visible. It will be a bit tricky, but
it is doable.

In LyX 2.0.7 coloring parts of arabic strings works ok. So, I'm not sure
why there is a problem here now. Ok, ligatures that should have
different parts colored differently is a bit difficult. My feeling is
that it is ok to split the ligatures in this exceptional case. The
contextual forms in arabic though are not ligatures and can be painted
in different colors without problems.

Actually, in master, the composition of character is done also by
looking forward and therefore by using characters beyond the ones we are
interested in. However, this is all hardcoded stuff, and I would like
instead to rely on whatever information Qt can give me.

Post by Vincent van Ravesteijn

Post by Jean-Marc Lasgouttes
However, the question is: do we want to keep this Color_selectiontext
thing? It makes all selection lose color, which is not very helpful.
Therefore, we could just decide to forget about that and just use
Color_selection for the background. Actually, I haven't found any
wordprocessor-like application that uses a selection background color.

That wouldn't allow the monochrome style.. green text on black
background, and, when it is selected, black text on green background :(.

OK, if people are fond of 8bit-style monochrome colors, we can make
efforts to keep that. But do you actually like the fact that selected
text loses all color indication and becomes just monochrome?

Therefore I have to make my clipping trick work. Currently on
str-metrics selection of plain Rnglish text when a font has some kerning
is just not usable.

Post by Vincent van Ravesteijn

Post by Jean-Marc Lasgouttes
The bigger problem will be cursor positioning, but I need more
information from people who understand Arabic writing to progress on that.

What info do you need ?

What is the difference between a ligature and a contextual form? Are
there in arabic Compose character that do not really have their own
width (like accents in latin scripts), but decorate another character?

I want to have some feeling of how this works. If you have a web page
for newbies describing these features, this would be perfect. Also, what
program is supposed to have a sound implementation of these languages in
terms of behavior? Word? LibreOffice?

I am not sure when I will have time to continue, but I want to
understand all these things.

And first I will probably try to implement your idea of using Qt to
place cursor.

JMarc

[Vincent van Ravesteijn]

Post by Vincent van Ravesteijn

Post by Jean-Marc Lasgouttes
OK, I have an idea for having correct selections without loosing the
Color_selectiontext enum: we could draw the complete string as
selected and non-selected, but use clipping to make sure that only the
right part of the selection is visible. It will be a bit tricky, but
it is doable.

In LyX 2.0.7 coloring parts of arabic strings works ok. So, I'm not sure
why there is a problem here now. Ok, ligatures that should have
different parts colored differently is a bit difficult. My feeling is
that it is ok to split the ligatures in this exceptional case. The
contextual forms in arabic though are not ligatures and can be painted
in different colors without problems.

Actually, in master, the composition of character is done also by looking

forward and therefore by using characters beyond the ones we are interested
in. However, this is all hardcoded stuff, and I would like instead to rely
on whatever information Qt can give me.

Have you had a look on QFontMetrics::width(QString const & str, int n =
-1). This function interprets the whole string str, and computes the width
of the string up to the nth character. This gives you the correct positions
for the arabic contextual forms.

Post by Vincent van Ravesteijn

Post by Jean-Marc Lasgouttes
The bigger problem will be cursor positioning, but I need more

information from people who understand Arabic writing to progress on

Post by Vincent van Ravesteijn

Post by Jean-Marc Lasgouttes
that.

What info do you need ?

What is the difference between a ligature and a contextual form?

According to: http://en.wikipedia.org/wiki/Arabic_alphabet#Ligatures, there
is only one compulsory ligature (having two forms, see later), and that's
the one I showed in a previous mail.

The contextual forms means that in general, each character has four
different presentation forms. These are the unicode points in the
arabic_table in src/Encoding.cpp. The unicode points are located in the
"Arabic Presentation Forms-B" unicode table.

In the four columns you can see the:
- isolated form
- end form; when the character is only connected to the character in front,
- initial form; when the character is only connected to the character
behind.
- mid form; when the character is connected to both the character in front
as behind,

(i hope you can see the figures)

An example of ha (0x0647):

Ha has four different representation forms.

Here an example of meem (0x0645):

Although the character looks pretty much the same in the first/second and
third/fourth form, they are different forms and have therefore different
unicode points.

The case is different when considering for example waw (0X0648):

This character can only be connected to the character before, but never to
the character behind. This means that the first and third form have the
same unicode point, as well as the second and fourth form. The reader can
confirm this in the arabic_table that this is indeed the case.

See also: QChar::joining().

Are there in arabic Compose character that do not really have their own

width (like accents in latin scripts), but decorate another character?

Most important compose chars are the "accents" that indicate the vowel
sounds and a few more. The range as defined by Encodings::arabicComposeChar
follows exactly what is defined by ISO-8859-6.

I think that the chars are recognized from Qt by QChar::category() ==
Qt::Mark_NonSpacing.

I want to have some feeling of how this works. If you have a web page for

newbies describing these features, this would be perfect. Also, what
program is supposed to have a sound implementation of these languages in
terms of behavior? Word? LibreOffice?
I used Wordpad during my learning, but don't know whether it is sound
enough.

I am not sure when I will have time to continue, but I want to understand

all these things.

And first I will probably try to implement your idea of using Qt to place

cursor.
Ah that answers my first question.

JMarc

Vincent

[Jean-Marc Lasgouttes]

Post by Vincent van Ravesteijn

Post by Jean-Marc Lasgouttes
What is the difference between a ligature and a contextual form?

According to: http://en.wikipedia.org/wiki/Arabic_alphabet#Ligatures,
there is only one compulsory ligature (having two forms, see later), and
that's the one I showed in a previous mail.
The contextual forms means that in general, each character has four
different presentation forms. These are the unicode points in the
arabic_table in src/Encoding.cpp. The unicode points are located in the
"Arabic Presentation Forms-B" unicode table.

Thanks, I understand now.

Post by Vincent van Ravesteijn
(i hope you can see the figures)

Nope. It is strange because in your previous message I had nice
pictures. But I this is the information from the wikipedia page, I read
it now.

Post by Vincent van Ravesteijn
This character can only be connected to the character before, but never
to the character behind. This means that the first and third form have
the same unicode point, as well as the second and fourth form. The
reader can confirm this in the arabic_table that this is indeed the case.
See also: QChar::joining().

Post by Jean-Marc Lasgouttes
And first I will probably try to implement your idea of using Qt to
place cursor.

Ah that answers my first question.

:) Actually, my plan is to find an API somewhere that does the work for
me. I want some library that tells me what to do, without having to care
about the language I am using. Do you think this is too much to ask?

Is there a widget in Qt that is able to edit arabic text, for example?
If I find one, I will look up the sources and try to understand what
they do.

JMarc

[Vincent van Ravesteijn]

Post by Jean-Marc Lasgouttes

Post by Vincent van Ravesteijn

Post by Jean-Marc Lasgouttes
What is the difference between a ligature and a contextual form?

According to: http://en.wikipedia.org/wiki/Arabic_alphabet#Ligatures,
there is only one compulsory ligature (having two forms, see later), and
that's the one I showed in a previous mail.
The contextual forms means that in general, each character has four
different presentation forms. These are the unicode points in the
arabic_table in src/Encoding.cpp. The unicode points are located in the
"Arabic Presentation Forms-B" unicode table.

Thanks, I understand now.

Post by Vincent van Ravesteijn
(i hope you can see the figures)

Nope. It is strange because in your previous message I had nice
pictures. But I this is the information from the wikipedia page, I
read it now.

Yes, I finished and sent the mail on mobile GMail, and I started
worrying about the figures.

Post by Jean-Marc Lasgouttes
:) Actually, my plan is to find an API somewhere that does the work
for me. I want some library that tells me what to do, without having
to care about the language I am using. Do you think this is too much
to ask?

I thought you wanted this: "I want to have some feeling of how this
works. " Now, you tell me you want to remain dumb ;).

No, Qt provides you with all the tools. However, we should use the
library correctly.

Post by Jean-Marc Lasgouttes
Is there a widget in Qt that is able to edit arabic text, for example?
If I find one, I will look up the sources and try to understand what
they do.

All Qt Widgets where you can enter text can edit arabic. The magic is
behind QPainter::drawText.

Post by Jean-Marc Lasgouttes
JMarc

Vincent

[Jean-Marc Lasgouttes]

Post by Vincent van Ravesteijn
I thought you wanted this: "I want to have some feeling of how this
works. " Now, you tell me you want to remain dumb ;).

I am completely happy with the answer you gave me. Now that I know what
type of pitfalls there may be, I see that the best bet is to have a
script-agnostic approach.

JMarc

[Vincent van Ravesteijn]

(re-sent, including figures this time)

Post by Jean-Marc Lasgouttes

Post by Vincent van Ravesteijn

Post by Jean-Marc Lasgouttes
OK, I have an idea for having correct selections without loosing the
Color_selectiontext enum: we could draw the complete string as
selected and non-selected, but use clipping to make sure that only the
right part of the selection is visible. It will be a bit tricky, but
it is doable.

In LyX 2.0.7 coloring parts of arabic strings works ok. So, I'm not sure
why there is a problem here now. Ok, ligatures that should have
different parts colored differently is a bit difficult. My feeling is
that it is ok to split the ligatures in this exceptional case. The
contextual forms in arabic though are not ligatures and can be painted
in different colors without problems.

Actually, in master, the composition of character is done also by
looking forward and therefore by using characters beyond the ones we
are interested in. However, this is all hardcoded stuff, and I would
like instead to rely on whatever information Qt can give me.

Have you had a look on QFontMetrics::width(QString const & str, int n =
-1). This function interprets the whole string str, and computes the
width of the string up to the nth character. This gives you the correct
positions for the arabic contextual forms.

Post by Jean-Marc Lasgouttes

Post by Vincent van Ravesteijn

Post by Jean-Marc Lasgouttes
The bigger problem will be cursor positioning, but I need more
information from people who understand Arabic writing to progress on that.

What info do you need ?

What is the difference between a ligature and a contextual form?

According to: http://en.wikipedia.org/wiki/Arabic_alphabet#Ligatures,
there is only one compulsory ligature (having two forms, see later), and
that's the one I showed in a previous mail.

The contextual forms means that in general, each character has four
different presentation forms. These are the unicode points in the
arabic_table in src/Encoding.cpp. The unicode points are located in the
"Arabic Presentation Forms-B" unicode table.

In the four columns you can see the:
- isolated form
- end form; when the character is only connected to the character in front,
- initial form; when the character is only connected to the character
behind.
- mid form; when the character is connected to both the character in
front as behind,

An example of ha (0x0647):

Ha has four different representation forms.

Here an example of meem (0x0645):


Although the character looks pretty much the same in the first/second
and third/fourth form, they are different forms and have therefore
different unicode points.

The case is different when considering for example waw (0X0648):

This character can only be connected to the character before, but never
to the character behind. This means that the first and third form have
the same unicode point, as well as the second and fourth form. The
reader can confirm this in the arabic_table that this is indeed the case.

Post by Jean-Marc Lasgouttes
Are there in arabic Compose character that do not really have their
own width (like accents in latin scripts), but decorate another
character?

Most important compose chars are the "accents" that indicate the vowel
sounds and a few more. The range as defined by
Encodings::arabicComposeChar follows exactly what is defined by ISO-8859-6.

I think that the chars are recognized from Qt by QChar::category() ==
Qt::Mark_NonSpacing.

Post by Jean-Marc Lasgouttes
I want to have some feeling of how this works. If you have a web page
for newbies describing these features, this would be perfect. Also,
what program is supposed to have a sound implementation of these
languages in terms of behavior? Word? LibreOffice?
I am not sure when I will have time to continue, but I want to
understand all these things.
And first I will probably try to implement your idea of using Qt to
place cursor.

Ah that answers my first question.

Post by Jean-Marc Lasgouttes
JMarc

Vincent

[Pavel Sanda]

Post by Jean-Marc Lasgouttes
However, the question is: do we want to keep this Color_selectiontext
thing? It makes all selection lose color, which is not very helpful.
Therefore, we could just decide to forget about that and just use
Color_selection for the background. Actually, I haven't found any
wordprocessor-like application that uses a selection background color.
Pavel, Enrico, as users of weird color schemes, I need your input on that.

I don't have any opinions about what color should the selection have,
if you just inverse colors that's fine. The only thing is not to hardcode
things so users of black background/white text can't see selection
anymore ;)
I can send my color scheme if it helps with your testing.

Pavel

[Jean-Marc Lasgouttes]

Post by Pavel Sanda
I don't have any opinions about what color should the selection have,
if you just inverse colors that's fine. The only thing is not to hardcode
things so users of black background/white text can't see selection
anymore ;)
I can send my color scheme if it helps with your testing.

The question is whether we want to support the case where the text
changes color when there is a selection.

JMarc

[Abdelrazak Younes]

Post by Jean-Marc Lasgouttes

Post by Vincent van Ravesteijn
1) the connection between characters is lost (in master this is not the
case)
2) the first character actually moves to the right on selection (while
you would expect that the rest of the word moves to the left because the
first character became wider on selection)

OK, I have an idea for having correct selections without loosing the
Color_selectiontext enum: we could draw the complete string as
selected and non-selected, but use clipping to make sure that only the
right part of the selection is visible. It will be a bit tricky, but
it is doable.
However, the question is: do we want to keep this Color_selectiontext
thing? It makes all selection lose color, which is not very helpful.
Therefore, we could just decide to forget about that and just use
Color_selection for the background. Actually, I haven't found any
wordprocessor-like application that uses a selection background color.
Pavel, Enrico, as users of weird color schemes, I need your input on that.
The bigger problem will be cursor positioning, but I need more
information from people who understand Arabic writing to progress on that.
I just noticed that we have no Arabic document (just a farsi
splash.lyx) in our doc. This is not very encouraging. Do we even have
Arabic writers as users?

I don't read Arabic but my wife does. I'll see if I can find time over
the week-end to compile your branch.

Abdel.

[Jean-Marc Lasgouttes]

Post by Abdelrazak Younes
I don't read Arabic but my wife does. I'll see if I can find time over
the week-end to compile your branch.

Thanks, you're a sweetheart. I already know that cursor positioning is
wrong, but I am not sure yet how to make this work. If smething looks
wrong, I'm interested too to know whether it is a regression wrt master
or 2.0.x.

JMarc

[Abdelrazak Younes]

Post by Abdelrazak Younes
I don't read Arabic but my wife does. I'll see if I can find time over

Post by Abdelrazak Younes
the week-end to compile your branch.

Thanks, you're a sweetheart. I already know that cursor positioning is
wrong, but I am not sure yet how to make this work. If smething looks
wrong, I'm interested too to know whether it is a regression wrt master or
2.0.x.

As a first trial cursor positioning seems correct. I don't have much time
to check against lyx2.0 but I copied and paste from the web (Firefox) to
LyX and LibreOffice.
Main visible issue is with letter LAM when combined with ALIF.

LAM is "G" on the english keyboard:
ل

ALIF is "H" on the english keyboard:
ا

LAM-ALIF:
لا

Above should be correctly shown in gmail webmail, it looks like Gamma greek
letter. In LyX if shows this:


Abdel

Post by Abdelrazak Younes
JMarc

[Jean-Marc Lasgouttes]

Post by Abdelrazak Younes
Main visible issue is with letter LAM when combined with ALIF.
ل
ا
لا
Above should be correctly shown in gmail webmail, it looks like Gamma
greek letter.

I think this is OK now.

JMarc

[Jean-Marc Lasgouttes]

Post by Vincent van Ravesteijn
The first thing I notice is that painting of some 'simple' ligatures
seems to be broken in master as well as in LyX 2.0.7. I really think
this worked before. Your branch fixes this issue.

Good.

Post by Vincent van Ravesteijn
Next, the position of the cursor seems to be computed by cutting the
string in two parts.

Yes, definitely.

Post by Vincent van Ravesteijn
Because of this, the width of the character before
the cursor is computed as its width in the unconnected form. This can be

What would be the right solution? I do not really know what to do. Does
unicode give me hints I could use?

Post by Vincent van Ravesteijn
and here, the cursor actually moves right, when moving logically to the
pos:3 pos:4

Could you send to me a short file containing examples?

Post by Vincent van Ravesteijn
1) the connection between characters is lost (in master this is not the
case)
2) the first character actually moves to the right on selection (while
you would expect that the rest of the word moves to the left because the
first character became wider on selection)

Since selection has its own color, the string is split in two. This
breaks all ligatures. I plan (unless there is a strong resistance from
people who love bicolor selections) to remove support for
Color_selectiontext and avoid breaking drawing at cursor. This will
avoid the "dancing text" effect with plain old latin text.

But of course, this will not solve anything wrt cursor position. We need
a strategy, and I do not have one.

JMarc

[Guy Rutenberg]

Hi,

One new feature of the branch is that all the hand-made handling of Hebrew
and Arabic characters is now unused. This is good news for Urdu or other
scripts.
If you do not try it out now, the risk is that you'll discover issues when
it lands on master :)

I did some initial testing and came up with the following wrong behavior:
When in RTL mode (i.e. settng the document language to hebrew) and typing
some narrow characters such as ו (vav) or even the English `i` slightly
moves the right margin to the left. By typing consecutive "narrow"
characters this movement is significant.

Typing wide characters has the opposite effect: The margin moves to the
right.

I've attached a screenshot. All the lines should be aligned the same,
however different character width shift the margin differently (there
should be no shift at all).

Regards,

Guy

[Guy Rutenberg]

Hi,
I actually found a more serious bug. When in RTL mode (`language hebrew`)
if one types a single continuous line without any spaces, LyX simply hangs
when it reaches the end-of-line. If there are spaces it breaks the line
correctly. I've checked it on Linux, can others reproduce the bug as well?

Regards,

Guy

[Jean-Marc Lasgouttes]

Post by Guy Rutenberg
I actually found a more serious bug. When in RTL mode (`language
hebrew`) if one types a single continuous line without any spaces, LyX
simply hangs when it reaches the end-of-line. If there are spaces it
breaks the line correctly. I've checked it on Linux, can others
reproduce the bug as well?

Noted. Could you create a bug too?

Thanks,
JMarc

[Jean-Marc Lasgouttes]

Post by Guy Rutenberg
I actually found a more serious bug. When in RTL mode (`language
hebrew`) if one types a single continuous line without any spaces, LyX
simply hangs when it reaches the end-of-line. If there are spaces it
breaks the line correctly. I've checked it on Linux, can others
reproduce the bug as well?

This is now bug #9120.

JMarc

[Jean-Marc Lasgouttes]

Post by Guy Rutenberg
Hi,
I actually found a more serious bug. When in RTL mode (`language
hebrew`) if one types a single continuous line without any spaces, LyX
simply hangs when it reaches the end-of-line. If there are spaces it
breaks the line correctly. I've checked it on Linux, can others
reproduce the bug as well?

I am not sure how to reproduce. Could I have an example file for this
bug please?

JMarc

[Scott Kostyshak]

On Fri, May 16, 2014 at 11:53 AM, Jean-Marc Lasgouttes

Post by Guy Rutenberg
Hi,
I actually found a more serious bug. When in RTL mode (`language
hebrew`) if one types a single continuous line without any spaces, LyX
simply hangs when it reaches the end-of-line. If there are spaces it
breaks the line correctly. I've checked it on Linux, can others
reproduce the bug as well?

I am not sure how to reproduce. Could I have an example file for this bug
please?

I can reproduce on 37494dfb and 13551a9f. What I did was:

1. open examples/he/splash.lyx
2. click on the bottom right corner (under the 6th point).
3. press return twice (this will give a ".6" above the ".7")
4. hold "f" for a while. On 37494dfb I think it froze right when it
got to the end of the string, but 13551a9f survives a little longer:
hold "f" until it gets to the end of the first line, wraps around to
the end of the next line, and hold it 10 seconds after that. Then hold
"a". I only have to hold "a" for a couple of seconds until it freezes
(infinite loop?).

If you cannot reproduce, might a screen cast help?

Scott

[Jean-Marc Lasgouttes]

I am not sure how to reproduce. Could I have an example file for this bug
please?

Oups! I made this comment under the wrong report from Guy. Actually, I
can reproduce this without problem and solving it is next on my todo list.

Thanks for checking.

JMarc

[Jean-Marc Lasgouttes]

Post by Guy Rutenberg
I did some initial testing and came up with the following wrong
behavior: When in RTL mode (i.e. settng the document language to hebrew)
and typing some narrow characters such as ו (vav) or even the English
`i` slightly moves the right margin to the left. By typing consecutive
"narrow" characters this movement is significant.
Typing wide characters has the opposite effect: The margin moves to the
right.
I've attached a screenshot. All the lines should be aligned the same,
however different character width shift the margin differently (there
should be no shift at all).

Hello Guy,

Thanks a lot for your tests. This effect is very weird. Could you create
a bug, maybe with an example file since I can't type Hebrew characters
:) I have to track correctly all the bugs that got reported.

JMarc

[Jean-Marc Lasgouttes]

Post by Guy Rutenberg
I did some initial testing and came up with the following wrong
behavior: When in RTL mode (i.e. settng the document language to hebrew)
and typing some narrow characters such as ו (vav) or even the English
`i` slightly moves the right margin to the left. By typing consecutive
"narrow" characters this movement is significant.
Typing wide characters has the opposite effect: The margin moves to the
right.
I've attached a screenshot. All the lines should be aligned the same,
however different character width shift the margin differently (there
should be no shift at all).

This is now bug #9119.

JMarc

[Jean-Marc Lasgouttes]

Post by Guy Rutenberg
I did some initial testing and came up with the following wrong
behavior: When in RTL mode (i.e. settng the document language to hebrew)
and typing some narrow characters such as ו (vav) or even the English
`i` slightly moves the right margin to the left. By typing consecutive
"narrow" characters this movement is significant.
Typing wide characters has the opposite effect: The margin moves to the
right.
I've attached a screenshot. All the lines should be aligned the same,
however different character width shift the margin differently (there
should be no shift at all).

Dear Guy,

I tried to reproduce this bug but did not succeed. I would be teker for
an example file.

I wonder too whether you use the "\force_paint_single_char true" lyxrc
setting?

JMarc

[Jean-Marc Lasgouttes]

Post by Jean-Marc Lasgouttes
Hello,
I finally think that the str-metrics branch is ready for a first round
of testing. I would like to know everything that does not work
especially (but not limited to) RTL writing.

I get what I asked for, and quite a lot of work to do :)

The tracking bug for this branch is #9003.
http://www.lyx.org/trac/ticket/9003

I tried to update it so that it is a good entry point to see the
advancement of the feature.

JMarc

[Richard Heck]

Post by Jean-Marc Lasgouttes

Post by Jean-Marc Lasgouttes
Hello,
I finally think that the str-metrics branch is ready for a first round
of testing. I would like to know everything that does not work
especially (but not limited to) RTL writing.

I get what I asked for, and quite a lot of work to do :)
The tracking bug for this branch is #9003.
http://www.lyx.org/trac/ticket/9003
I tried to update it so that it is a good entry point to see the
advancement of the feature.

Do you want to create a str-metrics component for this?

Richard

[Jean-Marc Lasgouttes]

Post by Richard Heck

Post by Jean-Marc Lasgouttes
I tried to update it so that it is a good entry point to see the
advancement of the feature.

Do you want to create a str-metrics component for this?

It seemed a bit too much, especially since this is supposed to be
transient. I have the tracking bug, and all bugs new to the branch have
a title that starts with [str-metrics]. I could also have used a keyword.

JMarc