severe lyx2lyx problem for 2.1 => 2.0 conversion

[Georg Baum]

I am sorry that this comes so late, but I found severe problems in lyx2lyx
which occur for 2.1 => 2.0 conversion while testing my fix for bug #9069.
If you do the roundtrip 2.0 => 2.1 => 2.0 for the 2.0.8 user guide, you get
a document with invalid header settings and data loss where argument and
index insets are mixed. I fixed these two problems at
http://www.lyx.org/trac/changeset/2fe07e1fe1e/lyxgit and
http://www.lyx.org/trac/changeset/5710f68c340/lyxgit.

What to do now? These fixes as well as
http://www.lyx.org/trac/changeset/cacd2a041d2/lyxgit and
http://www.lyx.org/trac/changeset/52d25886131/lyxgit should go to 2.1.1.
Richard, is this OK?

I also think that the final 2.0.x release should have them. I know that
this would create much additional work, since 2.0.8 is already tagged and
packaged (and therefore can't be changed anymore), but I don't think that
it is acceptable to tell people that it can read 2.1 files if in fact
these files will not be converted correctly in most cases.

The question is of course how to avoid such bugs in the future, and I
believe that we need automated lyx2lyx roundtrip tests: Such tests would
have found the problems.


Georg

[Richard Heck]

Post by Georg Baum
I am sorry that this comes so late, but I found severe problems in lyx2lyx
which occur for 2.1 => 2.0 conversion while testing my fix for bug #9069.
If you do the roundtrip 2.0 => 2.1 => 2.0 for the 2.0.8 user guide, you get
a document with invalid header settings and data loss where argument and
index insets are mixed. I fixed these two problems at
http://www.lyx.org/trac/changeset/2fe07e1fe1e/lyxgit and
http://www.lyx.org/trac/changeset/5710f68c340/lyxgit.
What to do now? These fixes as well as
http://www.lyx.org/trac/changeset/cacd2a041d2/lyxgit and
http://www.lyx.org/trac/changeset/52d25886131/lyxgit should go to 2.1.1.
Richard, is this OK?

Yes, those look good.

Post by Georg Baum
I also think that the final 2.0.x release should have them. I know that
this would create much additional work, since 2.0.8 is already tagged and
packaged (and therefore can't be changed anymore), but I don't think that
it is acceptable to tell people that it can read 2.1 files if in fact
these files will not be converted correctly in most cases.

I'm inclined to agree. It would mean re-packaging for everyone, but it
shouldn't be too bad. Other opinions?

Post by Georg Baum
The question is of course how to avoid such bugs in the future, and I
believe that we need automated lyx2lyx roundtrip tests: Such tests would
have found the problems.

That would be a good idea.

Richard

[Pavel Sanda]

Post by Richard Heck
I'm inclined to agree. It would mean re-packaging for everyone, but it
shouldn't be too bad. Other opinions?

We should have 2.0.8.1 (or 2.0.9), 2.0.8 packages are already in the wild.
Pavel

[Richard Heck]

Post by Pavel Sanda

Post by Richard Heck
I'm inclined to agree. It would mean re-packaging for everyone, but it
shouldn't be too bad. Other opinions?

We should have 2.0.8.1 (or 2.0.9), 2.0.8 packages are already in the wild.

Yes, that's true.

rh

[Pavel Sanda]

Post by Richard Heck

Post by Pavel Sanda

Post by Richard Heck
I'm inclined to agree. It would mean re-packaging for everyone, but it
shouldn't be too bad. Other opinions?

We should have 2.0.8.1 (or 2.0.9), 2.0.8 packages are already in the wild.

Yes, that's true.

If the question was whether we should repackage at all then I think yes as well. P

[Pavel Sanda]

Post by Georg Baum
a document with invalid header settings and data loss where argument and
index insets are mixed. I fixed these two problems at

BTW how often this dataloss happens - only if index inset is present?
(I ask whether immediate 2.1.(0.)1 release is needed or we can wait weeks...)

Pavel

[Georg Baum]

Post by Pavel Sanda

Post by Georg Baum
a document with invalid header settings and data loss where argument and
index insets are mixed. I fixed these two problems at

BTW how often this dataloss happens - only if index inset is present?
(I ask whether immediate 2.1.(0.)1 release is needed or we can wait weeks...)

The invalid headers happen for all documents with the standard math package
settings. The data loss happens only if an index or label inset is present
(maybe there are other insets of this kind, but index and label are the ones
which occur several times in our own docs).

I don't think that an immediate action for 2.1.x is needed - I guess the
2.1.1 release will be quite soon anyway. The situation with 2.0.x is
different - we won't produce any further release there, and it is especially
adverstised to be able to read 2.1 files.


Georg

[Richard Heck]

Post by Georg Baum

Post by Pavel Sanda

Post by Georg Baum
a document with invalid header settings and data loss where argument and
index insets are mixed. I fixed these two problems at

BTW how often this dataloss happens - only if index inset is present?
(I ask whether immediate 2.1.(0.)1 release is needed or we can wait weeks...)

The invalid headers happen for all documents with the standard math package
settings. The data loss happens only if an index or label inset is present
(maybe there are other insets of this kind, but index and label are the ones
which occur several times in our own docs).
I don't think that an immediate action for 2.1.x is needed - I guess the
2.1.1 release will be quite soon anyway. The situation with 2.0.x is
different - we won't produce any further release there, and it is especially
adverstised to be able to read 2.1 files.

Since 2.0.8 was never announced, it is easy enough to repackage and
just release 2.0.8.1. I have cherry picked the recent lyx2lyx commits. Is
that all? I.e., can I build the tarballs?

Richard

[Pavel Sanda]

Post by Richard Heck
Since 2.0.8 was never announced, it is easy enough to repackage and
just release 2.0.8.1. I have cherry picked the recent lyx2lyx commits. Is
that all? I.e., can I build the tarballs?

Maybe add small note about the difference between 2.0.8 and 2.0.8.1.
Pavel

[Georg Baum]

Post by Richard Heck
Since 2.0.8 was never announced, it is easy enough to repackage and
just release 2.0.8.1. I have cherry picked the recent lyx2lyx commits. Is
that all? I.e., can I build the tarballs?

In addition to the stuff which is already in 2.1 I'd like to see d6fecb4aa
both in 2.1 and 2.0.8.1. OK?

I did some more testing, and apart from the totate problem I did not found
any further problem other than cosmetic issues such as added empty lines
(which do not change the output).


Georg

[Richard Heck]

Post by Georg Baum

Post by Richard Heck
Since 2.0.8 was never announced, it is easy enough to repackage and
just release 2.0.8.1. I have cherry picked the recent lyx2lyx commits. Is
that all? I.e., can I build the tarballs?

In addition to the stuff which is already in 2.1 I'd like to see d6fecb4aa
both in 2.1 and 2.0.8.1. OK?

Yes, go ahead.

Post by Georg Baum
I did some more testing, and apart from the totate problem I did not found
any further problem other than cosmetic issues such as added empty lines
(which do not change the output).

Excellent.

Richard

[Georg Baum]

Post by Richard Heck

Post by Georg Baum

Post by Richard Heck
Since 2.0.8 was never announced, it is easy enough to repackage and
just release 2.0.8.1. I have cherry picked the recent lyx2lyx commits.
Is that all? I.e., can I build the tarballs?

In addition to the stuff which is already in 2.1 I'd like to see
d6fecb4aa both in 2.1 and 2.0.8.1. OK?

Yes, go ahead.

I put it in 2.1 branch, since I did not see your cherry-picks in 2.0 I'll
leave this one to you as well.


Georg

[Scott Kostyshak]

On Fri, Apr 25, 2014 at 5:05 PM, Georg Baum

Post by Georg Baum
The question is of course how to avoid such bugs in the future, and I
believe that we need automated lyx2lyx roundtrip tests: Such tests would
have found the problems.

Do you have an idea for implementing these tests? It's not hard to
create a bash script if you are on 2.1.x, that checks out the document
files from master branch and replaces lib/{doc,template,examples}.
Then the normal ctest should work. In fact I was planning to do this
for the lyx-tester script I use. But that is a bit messy and relies on
Git and I'm not sure it belongs in LyX's repo.

Scott

[Vincent van Ravesteijn]

Post by Scott Kostyshak
On Fri, Apr 25, 2014 at 5:05 PM, Georg Baum

Post by Georg Baum
The question is of course how to avoid such bugs in the future, and I
believe that we need automated lyx2lyx roundtrip tests: Such tests would
have found the problems.

Do you have an idea for implementing these tests? It's not hard to
create a bash script if you are on 2.1.x, that checks out the document
files from master branch and replaces lib/{doc,template,examples}.
Then the normal ctest should work. In fact I was planning to do this
for the lyx-tester script I use. But that is a bit messy and relies on
Git and I'm not sure it belongs in LyX's repo.
Scott

The first problem is that the lyx2lyx code seems not always be tested by
the writer himself nor critically reviewed by anyone else. Not so long ago
we found a huge error in lyx2lyx that would corrupt any document that was
slightly more real-life than a minimal example. The code was completely
wrong and anyone could have seen that.

Especially the lyx2lyx code could be tested by unit tests. Each
convert/revert function should be accompanied by tests.

Vincent