Can no longer paste PNG's automatically on trunk (intended?)

[Scott Kostyshak]

I remember a lot of work was done on improving pasting (e.g. LaTeX
detection) but I don't remember this change being discussed.

Before (and now on current 2.0git), you could copy a png from the web
(coyping in Chromium on Ubuntu stores it as MIME type "image/bmp" for
me) and paste it directly into LyX. You would be asked where to save
the file. Now, nothing happens. If you go to Edit > Paste Special >
Paste as PNG, then it works.

I liked the automatic detection. Was this removed intentionally?

I remember that there was some debate about whether LaTeX should be
detected and in the end it was decided that it should not. But I think
that was because it's harder to detect because of parsing (Georg noted
that applications don't set the MIME type for text/x-tex
application/x-latex). Here that doesn't seem an issue.

If this is unintended and it would be helpful if I did a bisect, let me know.

Scott

[Vincent van Ravesteijn]

Post by Scott Kostyshak
I remember a lot of work was done on improving pasting (e.g. LaTeX
detection) but I don't remember this change being discussed.
Before (and now on current 2.0git), you could copy a png from the web
(coyping in Chromium on Ubuntu stores it as MIME type "image/bmp" for
me) and paste it directly into LyX. You would be asked where to save
the file. Now, nothing happens. If you go to Edit > Paste Special >
Paste as PNG, then it works.

I don't know of a lot of work in this area and I can still paste
something that I copy from MS paint.

Post by Scott Kostyshak
I liked the automatic detection. Was this removed intentionally?
I remember that there was some debate about whether LaTeX should be
detected and in the end it was decided that it should not. But I think
that was because it's harder to detect because of parsing (Georg noted
that applications don't set the MIME type for text/x-tex
application/x-latex). Here that doesn't seem an issue.
If this is unintended and it would be helpful if I did a bisect, let me know.

I don't see the problem, so I can't say if it is intended.

Post by Scott Kostyshak
Scott

Vincent

[Georg Baum]

Post by Scott Kostyshak
I remember a lot of work was done on improving pasting (e.g. LaTeX
detection) but I don't remember this change being discussed.
Before (and now on current 2.0git), you could copy a png from the web
(coyping in Chromium on Ubuntu stores it as MIME type "image/bmp" for
me) and paste it directly into LyX. You would be asked where to save
the file. Now, nothing happens. If you go to Edit > Paste Special >
Paste as PNG, then it works.
I liked the automatic detection. Was this removed intentionally?

No (and I don't see any difference of 2.1 vs. 2.0 here).

Post by Scott Kostyshak
I remember that there was some debate about whether LaTeX should be
detected and in the end it was decided that it should not. But I think
that was because it's harder to detect because of parsing (Georg noted
that applications don't set the MIME type for text/x-tex
application/x-latex). Here that doesn't seem an issue.
If this is unintended and it would be helpful if I did a bisect, let me know.

For me, it does not work with 2.0 either. In both cases, I get the URL as
text pasted. Paste Special->png works in both versions. The reason is that
my browser puts both text and graphics onto the clipboard, and in this cases
LyX decides to use the text (in the handler of LFUN_PASTE). This could
easily be changed to prefer graphics, but what happens if an application
puts something which is basically text also as a graphic onto the clipboard?

If you see differences between 2.0 and 2.1 on your machine a bisect could be
helpful, but I don't understand where these differences comes from.


Georg

[Scott Kostyshak]

On Wed, Jun 5, 2013 at 3:04 PM, Georg Baum

Post by Georg Baum
For me, it does not work with 2.0 either. In both cases, I get the URL as
text pasted. Paste Special->png works in both versions. The reason is that
my browser puts both text and graphics onto the clipboard, and in this cases
LyX decides to use the text (in the handler of LFUN_PASTE). This could
easily be changed to prefer graphics, but what happens if an application
puts something which is basically text also as a graphic onto the clipboard?

Interesting. Good point. Probably best to be safe here.

Post by Georg Baum
If you see differences between 2.0 and 2.1 on your machine a bisect could be
helpful, but I don't understand where these differences comes from.

Bisect leads me here:
c14b9e67

Before that commit, it works automatically. After, if I paste nothing
at all is written. At some of the commits given in the bisect, LyX
would paste something like the following as text:
%00D%00o%00w%00n%00l%00o%00a%00d

Scott

[Georg Baum]

Post by Scott Kostyshak
c14b9e67

Thanks, I'll have a look.


Georg

[Scott Kostyshak]

On Sun, Jun 9, 2013 at 4:48 AM, Georg Baum

Post by Georg Baum

Post by Scott Kostyshak
c14b9e67

Thanks, I'll have a look.

Did you have a chance to take a look at this Georg? Should I open a trac ticket?

Scott

[Georg Baum]

Post by Scott Kostyshak
On Sun, Jun 9, 2013 at 4:48 AM, Georg Baum

Post by Georg Baum

Post by Scott Kostyshak
c14b9e67

Thanks, I'll have a look.

Did you have a chance to take a look at this Georg? Should I open a trac ticket?

Sorry, it is finally summer here, so I had to spend more time outside:-)

Today I had a look, but could not reproduce your problem. I also changed the
preference from text to graphics for standard paste without arguments, and
pasting of a png file worked fine. Without this, I either get the image (if
e.g. pasted from gimp), or the URL of the image as text (if pasted from
browsers), but as I explained preferring the text over graphics is wanted
and the same as in 2.0.

Which application do you use to store the png on the clipboard? Could you
debug what happens during pasting? The most interesting thing to know would
be the text flavour which is recognized by LyX (e.g. plain text or HTML
etc.), and whether it recognizes text only, or text + graphics. Currently I
believe that the copying application puts something on the clipboard which
is recognized as text by LyX, but in fact is no text.


Georg

[Scott Kostyshak]

On Sun, Jul 7, 2013 at 4:32 AM, Georg Baum

Post by Georg Baum
Sorry, it is finally summer here, so I had to spend more time outside:-)

Same here, except that 37 C means that some days I spend less time outside :)

Post by Georg Baum
Today I had a look, but could not reproduce your problem. I also changed the
preference from text to graphics for standard paste without arguments, and
pasting of a png file worked fine. Without this, I either get the image (if
e.g. pasted from gimp), or the URL of the image as text (if pasted from
browsers), but as I explained preferring the text over graphics is wanted
and the same as in 2.0.
Which application do you use to store the png on the clipboard? Could you
debug what happens during pasting? The most interesting thing to know would
be the text flavour which is recognized by LyX (e.g. plain text or HTML
etc.), and whether it recognizes text only, or text + graphics. Currently I
believe that the copying application puts something on the clipboard which
is recognized as text by LyX, but in fact is no text.

Thanks for taking a look and trying to reproduce. What OS are you
using? What browser? I'm using Ubuntu 13.04 and Chromium and Firefox.

I use the (excellent and cross-platform) clipboard manager CopyQ, but
I can reproduce with or without CopyQ running:

In Chromium or Firefox, when I go to http://www.lyx.org and right
click on intro.png and go to "copy image" and then go to LyX and
paste, nothing is pasted (although "paste" shows up in LyX's message
box). If I then open up Gimp and paste, the image pastes as expected.

CopyQ shows the following after copying the image:

text/html:
<meta http-equiv="content-type" content="text/html;
charset=utf-8"><img src="Loading Image..."
alt="Download"/>

image/bmp:
(shows the picture)

text/uri-list:
http://www.lyx.org/images/intro.png

application/x-copyq-owner-window-title:
LyX | LyX – The Document Processor - Chromium

If I copy from Gimp to LyX, everything works fine and CopyQ shows the
clipboard as having:
image/bmp
image/png
image/jpeg
application/x-copyq-owner-window-title:
*[Untitled]-5.0 (RGB color, 2 layers) 640x400 – GIMP

So perhaps it is the text/html that is causing the behavior I'm
seeing. Maybe LyX recognizes it and pastes it in but when it converts
it it converts it to nothing? That's why I don't see anything pasted?

Scott

[Vincent van Ravesteijn]

Post by Scott Kostyshak
In Chromium or Firefox, when I go to http://www.lyx.org and right
click on intro.png and go to "copy image" and then go to LyX and
paste, nothing is pasted (although "paste" shows up in LyX's message box).

This is indeed a difference between 2.0 and 2.1. Strangely, copying an
image from MS paint just works as before (as Scott also noted).

Vincent

[Georg Baum]

Post by Scott Kostyshak
Thanks for taking a look and trying to reproduce. What OS are you
using? What browser? I'm using Ubuntu 13.04 and Chromium and Firefox.

I tested Iceweasel, Konqueror and Opera on Debian Wheezy. With Chromium I
was finally able to reproduce your problem.

Post by Scott Kostyshak
<meta http-equiv="content-type" content="text/html;
charset=utf-8"><img src="http://www.lyx.org/images/intro.png"
alt="Download"/>

This is converted to an empty string by the HTML import. Once could argue
whether this could be done better, but this is in fact something which can
happen.

Post by Scott Kostyshak
(shows the picture)
http://www.lyx.org/images/intro.png
LyX | LyX – The Document Processor - Chromium

Note that there is no text/plain!

Post by Scott Kostyshak
If I copy from Gimp to LyX, everything works fine and CopyQ shows the
image/bmp
image/png
image/jpeg
*[Untitled]-5.0 (RGB color, 2 layers) 640x400 – GIMP
So perhaps it is the text/html that is causing the behavior I'm
seeing. Maybe LyX recognizes it and pastes it in but when it converts
it it converts it to nothing? That's why I don't see anything pasted?

This is exactly what happens. 2.0 does not accept text/html, and since there
is no text/plain, it chooses to paste the image. 2.1 sees that there is html
available, and then imports that, but the result is empty.

The attached patch fixes two issues:

1) It uses the same type argument of theClipboard().hasTextContents() and
pasteClipboardText(). This eliminates one possible error (which did not
occur in this case)

2) If both text and graphics are available, and pasting text failed, it
pastes graphics instead.

Does this work for you? What about the garbled text which you posted
earlier?


Georg

[Scott Kostyshak]

On Mon, Jul 8, 2013 at 4:10 PM, Georg Baum

Post by Georg Baum
1) It uses the same type argument of theClipboard().hasTextContents() and
pasteClipboardText(). This eliminates one possible error (which did not
occur in this case)
2) If both text and graphics are available, and pasting text failed, it
pastes graphics instead.

How does it know that pasting text failed? Because there is no
text/plain? Or because after text/html is converted it is an empty
string?

Post by Georg Baum
Does this work for you? What about the garbled text which you posted
earlier?

This works well for me.

The garbled text was during some commits when I was bisecting. If you
are curious I can try to find a commit in the past that exhibits this.

Thanks for the patch,

Scott

[Georg Baum]

Post by Scott Kostyshak
How does it know that pasting text failed? Because there is no
text/plain? Or because after text/html is converted it is an empty
string?

The latter.

Post by Scott Kostyshak

Post by Georg Baum
Does this work for you? What about the garbled text which you posted
earlier?

This works well for me.
The garbled text was during some commits when I was bisecting. If you
are curious I can try to find a commit in the past that exhibits this.

No, I am not interested if it does not occur with the latest version.

To all: May this fix go into 2.1-staging?


Georg

[Kornel Benko]

Post by Georg Baum

Post by Scott Kostyshak
How does it know that pasting text failed? Because there is no
text/plain? Or because after text/html is converted it is an empty
string?

The latter.

Post by Scott Kostyshak

Post by Georg Baum
Does this work for you? What about the garbled text which you posted
earlier?

This works well for me.
The garbled text was during some commits when I was bisecting. If you
are curious I can try to find a commit in the past that exhibits this.

No, I am not interested if it does not occur with the latest version.
To all: May this fix go into 2.1-staging?
Georg

+1

Kornel

[Scott Kostyshak]

On Mon, Jul 8, 2013 at 4:10 PM, Georg Baum

Post by Georg Baum
Does this work for you? What about the garbled text which you posted
earlier?

Hi Georg,

This has come back for me with Qt 5 (5.2.1 on Ubuntu 14.04). If I go
to www.google.com in Chromium and right-click on the Google icon and
select "Copy image" and then do a normal paste in LyX, I get the
following:
%00G%00o%00o%00g%00l%00e
If I do the same in Firefox, it works as I expect and tries to paste the image.
'paste png' works after copying in Chromium.
The difference appears to be that after copying in Chromium I have the
following MIME types:

text/html
<meta http-equiv="content-type" content="text/html;
charset=utf-8"><img
src="Loading Image..." alt="Google"/>

text/uri-list
https://www.google.com/images/srpr/logo11w.png

where with Firefox I also have text/html but with a content of:
<img alt="Google" id="hplogo"
src="https://www.google.com/images/srpr/logo11w.png"
style="padding-top:112px" onload="window.lol&amp;&amp;lol()"
height="95" width="269">

If I remove the text/html MIME type, LyX then pastes
https://www.google.com/images/srpr/logo11w.png
which is the contents of text/uri-list
If I remove text/uri-list then the picture is pasted.

Post by Georg Baum
From what I understand, LyX asks Qt to convert HTML to a QString in

tidyHtml(). If that string is empty, then it moves to a different MIME
type. Perhaps the HTML conversion has changed in Qt 5 versus Qt 4?
I'm not sure how the treatment of text/uri-list has changed.

Note that this isn't annoying to me personally. But is this something
we want to try to fix? Or do you think that trying to make this sort
of paste work with all browsers and all the ways that MIME types are
set is an impossible goal?

Scott

[Georg Baum]

Post by Scott Kostyshak
Hi Georg,
This has come back for me with Qt 5 (5.2.1 on Ubuntu 14.04). If I go
to www.google.com in Chromium and right-click on the Google icon and
select "Copy image" and then do a normal paste in LyX, I get the
%00G%00o%00o%00g%00l%00e
If I do the same in Firefox, it works as I expect and tries to paste the
image. 'paste png' works after copying in Chromium.
The difference appears to be that after copying in Chromium I have the
text/html
<meta http-equiv="content-type" content="text/html;
charset=utf-8"><img
src="https://www.google.com/images/srpr/logo11w.png" alt="Google"/>
text/uri-list
https://www.google.com/images/srpr/logo11w.png
<img alt="Google" id="hplogo"
src="https://www.google.com/images/srpr/logo11w.png"
style="padding-top:112px" onload="window.lol&amp;&amp;lol()"
height="95" width="269">
If I remove the text/html MIME type, LyX then pastes
https://www.google.com/images/srpr/logo11w.png
which is the contents of text/uri-list
If I remove text/uri-list then the picture is pasted.

Are these MIME types the only ones? I would think that image/png exists as
well?

Post by Scott Kostyshak
From what I understand, LyX asks Qt to convert HTML to a QString in
tidyHtml(). If that string is empty, then it moves to a different MIME
type. Perhaps the HTML conversion has changed in Qt 5 versus Qt 4?
I'm not sure how the treatment of text/uri-list has changed.

The logic is in Text::dispatch(): It looks if the clipboard has text
contents, and if that is true, pastes text, otherwise it pastes graphics.

Post by Scott Kostyshak
Note that this isn't annoying to me personally. But is this something
we want to try to fix? Or do you think that trying to make this sort
of paste work with all browsers and all the ways that MIME types are
set is an impossible goal?

I think it is sensible to try to detect pure image contents on the
clipboard, however I don't think we should hard code any browser specific
stuff. What I could imagine is to inspect the text/HTML further: If (after
cleaning) the only tag in the body is <img>, and if the clipboard contains
image contents as well, then ignore the HTML for pasting without argument
and prefer the image.


Georg

[Scott Kostyshak]

On Mon, Jun 2, 2014 at 2:52 PM, Georg Baum

Post by Georg Baum

Post by Scott Kostyshak
Hi Georg,
This has come back for me with Qt 5 (5.2.1 on Ubuntu 14.04). If I go
to www.google.com in Chromium and right-click on the Google icon and
select "Copy image" and then do a normal paste in LyX, I get the
%00G%00o%00o%00g%00l%00e
If I do the same in Firefox, it works as I expect and tries to paste the
image. 'paste png' works after copying in Chromium.
The difference appears to be that after copying in Chromium I have the
text/html
<meta http-equiv="content-type" content="text/html;
charset=utf-8"><img
src="https://www.google.com/images/srpr/logo11w.png" alt="Google"/>
text/uri-list
https://www.google.com/images/srpr/logo11w.png
<img alt="Google" id="hplogo"
src="https://www.google.com/images/srpr/logo11w.png"
style="padding-top:112px" onload="window.lol&amp;&amp;lol()"
height="95" width="269">
If I remove the text/html MIME type, LyX then pastes
https://www.google.com/images/srpr/logo11w.png
which is the contents of text/uri-list
If I remove text/uri-list then the picture is pasted.

Are these MIME types the only ones? I would think that image/png exists as
well?

You are correct. What I listed was the difference between Chromium
(which does not work) and Firefox (which does work). If you want me to
list all the MIME types, let me know.

Post by Georg Baum

Post by Scott Kostyshak
From what I understand, LyX asks Qt to convert HTML to a QString in
tidyHtml(). If that string is empty, then it moves to a different MIME
type. Perhaps the HTML conversion has changed in Qt 5 versus Qt 4?
I'm not sure how the treatment of text/uri-list has changed.

The logic is in Text::dispatch(): It looks if the clipboard has text
contents, and if that is true, pastes text, otherwise it pastes graphics.

Post by Scott Kostyshak
Note that this isn't annoying to me personally. But is this something
we want to try to fix? Or do you think that trying to make this sort
of paste work with all browsers and all the ways that MIME types are
set is an impossible goal?

I think it is sensible to try to detect pure image contents on the
clipboard, however I don't think we should hard code any browser specific
stuff. What I could imagine is to inspect the text/HTML further: If (after
cleaning) the only tag in the body is <img>, and if the clipboard contains
image contents as well, then ignore the HTML for pasting without argument
and prefer the image.

I like this idea. Where should this be done? The place where Qt is
used to get the text is inGuiClipboard::getAsText at

case PlainTextType:
str = qApp->clipboard()->text(QClipboard::Clipboard)
.normalized(QString::NormalizationForm_C);

I guess the idea is we should never get to this point?

Scott

[Georg Baum]

Post by Scott Kostyshak
On Mon, Jun 2, 2014 at 2:52 PM, Georg Baum
You are correct. What I listed was the difference between Chromium
(which does not work) and Firefox (which does work). If you want me to
list all the MIME types, let me know.

No, I don't need that, it is enough to know that image/png is also on the
clipboard.

Post by Scott Kostyshak

Post by Georg Baum
I think it is sensible to try to detect pure image contents on the
clipboard, however I don't think we should hard code any browser specific
stuff. What I could imagine is to inspect the text/HTML further: If
(after cleaning) the only tag in the body is <img>, and if the clipboard
contains image contents as well, then ignore the HTML for pasting without
argument and prefer the image.

I like this idea. Where should this be done? The place where Qt is
used to get the text is inGuiClipboard::getAsText at
str = qApp->clipboard()->text(QClipboard::Clipboard)
.normalized(QString::NormalizationForm_C);
I guess the idea is we should never get to this point?

I think it needs to be done in two places: In GuiClipboard and in
Text::dispatch() (or a new service routine called from there). The reason
for this is that I don't want to change the behaviour of the special pasting
methods: If somebody explicitly says he wants to paste text, then he should
get the image link.
Attached is a sketch of the idea, without the actual HTML parsing part to
find out whether the whole body conatins only an <img> tag, and without any
documentation.


Georg

[Scott Kostyshak]

On Tue, Jun 3, 2014 at 3:49 PM, Georg Baum

Post by Georg Baum

Post by Scott Kostyshak
On Mon, Jun 2, 2014 at 2:52 PM, Georg Baum
You are correct. What I listed was the difference between Chromium
(which does not work) and Firefox (which does work). If you want me to
list all the MIME types, let me know.

No, I don't need that, it is enough to know that image/png is also on the
clipboard.

Post by Scott Kostyshak

Post by Georg Baum
I think it is sensible to try to detect pure image contents on the
clipboard, however I don't think we should hard code any browser specific
stuff. What I could imagine is to inspect the text/HTML further: If
(after cleaning) the only tag in the body is <img>, and if the clipboard
contains image contents as well, then ignore the HTML for pasting without
argument and prefer the image.

I like this idea. Where should this be done? The place where Qt is
used to get the text is inGuiClipboard::getAsText at
str = qApp->clipboard()->text(QClipboard::Clipboard)
.normalized(QString::NormalizationForm_C);
I guess the idea is we should never get to this point?

I think it needs to be done in two places: In GuiClipboard and in
Text::dispatch() (or a new service routine called from there). The reason
for this is that I don't want to change the behaviour of the special pasting
methods: If somebody explicitly says he wants to paste text, then he should
get the image link.
Attached is a sketch of the idea, without the actual HTML parsing part to
find out whether the whole body conatins only an <img> tag, and without any
documentation.

Thanks for taking a look, Georg. From what I understand, the only
missing piece to the patch is the XML parser. But I added a line
lyxerr << "str is: " << str << std::endl;
right before the "Now use some XML parser" comment, and it is only
triggered when pasting from Firefox, not from Chromium.

Scott

[Georg Baum]

Post by Scott Kostyshak
Thanks for taking a look, Georg. From what I understand, the only
missing piece to the patch is the XML parser.

Yes.

Post by Scott Kostyshak
But I added a line
lyxerr << "str is: " << str << std::endl;
right before the "Now use some XML parser" comment, and it is only
triggered when pasting from Firefox, not from Chromium.

This indicates either a bug (maybe GuiClipboard::on_dataChanged() is not
called although it should), or I misunderstood how
GuiClipboard::on_dataChanged() works.
However, I think you got the general idea.


Georg

[Scott Kostyshak]

On Wed, Jun 4, 2014 at 4:25 PM, Georg Baum

Post by Georg Baum

Post by Scott Kostyshak
Thanks for taking a look, Georg. From what I understand, the only
missing piece to the patch is the XML parser.

Yes.

Post by Scott Kostyshak
But I added a line
lyxerr << "str is: " << str << std::endl;
right before the "Now use some XML parser" comment, and it is only
triggered when pasting from Firefox, not from Chromium.

This indicates either a bug (maybe GuiClipboard::on_dataChanged() is not
called although it should), or I misunderstood how
GuiClipboard::on_dataChanged() works.
However, I think you got the general idea.

I investigated further. The reason is that cache_.hasText() is true so
hasTextContents is true so has_text_contents_not_only_image_link_ is
true, so the if is not triggered.

Does that help?

Scott