|
Hi all,
The attached is a proposal for a new external template for gnuplot, a graph plotting program. I have been using it for my personal use for a while, but I thought it may be better included in the main body as it is convenient. With this template, users can directly edit the gnuplot source code from lyx, whereas its consequent graphics is automatically included in outputs. Although the attached script can generate other graphics formats as well, in the purpose of external template, it is only used for generation of an EPS graphics file from a gnuplot source code. If necessary, users can add additional converters for each graphics format. I think it should be better checked by professionals, but I hope it is useful. How do you think? Best regards, Koji --- lib/configure.py.orig 2008-05-06 22:20:27.000000000 +0900 +++ lib/configure.py 2008-05-15 01:37:48.000000000 +0900 @@ -244,6 +244,10 @@ checkViewer('a FEN viewer and editor', ['xboard -lpf $$i -mode EditPosition'], rc_entry = [r'\Format fen fen FEN "" "%%" "%%" ""']) # + checkViewer('a Gnuplot viewer and editor', ['emacs', 'emcws', 'xemacs', 'gvim', 'kedit', 'kwrite', \ + 'kate', 'nedit', 'gedit', 'notepad'], + rc_entry = [r'\Format gnuplot gnuplot Gnuplot "" "%%" "%%" "vector"']) + # path, iv = checkViewer('a raster image viewer', ['xv', 'kview', 'gimp-remote', 'gimp']) path, ie = checkViewer('a raster image editor', ['gimp-remote', 'gimp']) addToRC(r'''\Format bmp bmp BMP "" "%s" "%s" "" @@ -499,6 +503,10 @@ checkProg('a Noteedit -> LilyPond converter', ['noteedit --export-lilypond $$i'], rc_entry = [ r'\converter noteedit lilypond "%%" ""', '']) # + checkProg('a Gnuplot -> EPS converter', ['gnuplot'], + rc_entry = [ r'\converter gnuplot eps "python -tt $$s/scripts/gnuplot.py $$i $$o" ""', '']) + # + # # FIXME: no rc_entry? comment it out # checkProg('Image converter', ['convert $$i $$o']) # --- lib/external_templates.orig 2008-05-15 01:43:46.000000000 +0900 +++ lib/external_templates 2008-05-15 01:44:46.000000000 +0900 @@ -314,3 +314,58 @@ UpdateResult "$$Tempname" FormatEnd TemplateEnd + +Template gnuplot + GuiName "gnuplot: $$AbsOrRelPathParent$$Basename" + HelpText + Create plots and graphics with Gnuplot + + 'gnuplot' binary must be found in path for it + to work correctly. + If you use emacs as an editor, you may want + to use it in the gnuplot mode. + HelpTextEnd + InputFormat gnuplot + FileFilter "*.{gnuplot,plt,plot,gnu}" + AutomaticProduction true + Transform Rotate + Transform Resize + Transform Clip + Transform Extra + Format LaTeX + Requirement "graphicx" + TransformOption Rotate RotationLatexOption + TransformOption Resize ResizeLatexOption + TransformOption Clip ClipLatexOption + TransformOption Extra ExtraOption + Option Arg "[$$Extra,$$Rotate,$$Resize,$$Clip]" + Product "\\includegraphics$$Arg{$$AbsOrRelPathMaster$$Basename.e +ps}" + UpdateFormat eps + UpdateResult "$$AbsPath$$Basename.eps" + ReferencedFile latex "$$AbsOrRelPathMaster$$Basename.eps" + ReferencedFile dvi "$$AbsPath$$Basename.eps" + FormatEnd + Format PDFLaTeX + Requirement "graphicx" + TransformOption Rotate RotationLatexOption + TransformOption Resize ResizeLatexOption + TransformOption Clip ClipLatexOption + TransformOption Extra ExtraOption + Option Arg "[$$Extra,$$Rotate,$$Resize,$$Clip]" + Product "\\includegraphics$$Arg{$$AbsOrRelPathMaster$$Basename.e +ps}" + UpdateFormat eps + UpdateResult "$$AbsPath$$Basename.eps" + ReferencedFile pdflatex "$$AbsOrRelPathMaster$$Basename.eps" + FormatEnd + Format Ascii + Product "[gnuplot: $$FName]" + FormatEnd + Format DocBook + Product "[gnuplot: $$FName]" + FormatEnd + Format LinuxDoc + Product "[gnuplot: $$FName]" + FormatEnd +TemplateEnd --- lib/scripts/gnuplot.py.orig 2008-05-15 01:46:32.000000000 +0900 +++ lib/scripts/gnuplot.py 2008-05-11 03:11:09.000000000 +0900 @@ -0,0 +1,123 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- + +# file gnuplot.py +# This file is part of LyX, the document processor. +# Licence details can be found in the file COPYING. + +# \author Koji Yokota + +# Full author contact details are available in file CREDITS. + +# This script creates a graphic file from a GnuPlot source file. +# It is aimed to be called from external templates. +# For it to work correctly, gnuplot must be found in path. +# +# Usage: python gnuplot.py [-t type [-o option]] \ +# <GnuPlot source file> <EPS output file> +# type: terminal type +# option: comma-separated options for a given terminal type +# Default values are: +# type: "postscript" +# option: "eps enhanced color" +# (It will be reset to null once type is specified) + +import re, sys, os, getopt, tempfile + +class Error: pass + +# Get options and arguments +opts, args = getopt.getopt(sys.argv[1:], "t:o:") +try: + if (len(args) != 2): + raise Error() +except Error: + print '\n' + sys.argv[0] + print '\nThis script outputs a graphics file from a Gnuplot source file.\n' + print 'Usage:\n python ' + sys.argv[0] + ' [-t type [-o option]] \\' + print ' <gnuplot source file> <output graphics file>\n' + print ' type: terminal type (see "help terminal" in gnuplot for' + print ' available values)' + print ' option: comma-separated options for a given terminal type' + print ' (see "help terminal <terminal type>" in gnuplot for' + print ' available values)' + print '\n Default values are:' + print ' type: "postscript"' + print ' option: "eps enhanced color" (It will be reset to null' + print ' once type is specified)\n' + sys.exit() + +# Set output graphic type and options +# It's very likely that terminal type and its options are set for review +# purpose (e.g. x11) or else. This script automates specification of +# output format. +terminal_type = 'postscript' +terminal_option = ['eps', 'enhanced', 'color'] +type_specified = 0 +option_specified = 0 +for o,v in opts: + if o == '-t': + terminal_type = v + type_specified = 1 + if o == '-o': + terminal_option = v.split(',') + option_specified = 1 +if type_specified and not option_specified: + terminal_option = [] + +# Set "set terminal" and "set output" lines in the gnuplot source file +terminal_line = 'set terminal ' + terminal_type +if terminal_option != []: + for o in terminal_option: + terminal_line = terminal_line + ' ' + o +output_line = 'set output "' + args[1] + '"' + +# Definition of strings to be replaced +find_terminal = re.compile(r"(^\s*set\s+terminal.*$)",re.IGNORECASE) +find_output = re.compile(r"(^\s*set\s+output.*$)",re.IGNORECASE) +find_comment = re.compile(r"(^#|^\s*$)",re.IGNORECASE) + +# comment_out function comments out 'set terminal...' and 'set output...' +# lines and outputs other lines as is +def comment_out(line): + if find_terminal.search(line): + print find_terminal.sub(r'# \1', line[:-1]) + count_terminal = 1 + elif find_output.search(line): + print find_output.sub(r'# \1', line[:-1]) + count_output = 1 + else: + print line[:-1] + +# Flag to show whether head comment part is finished +comment_ended = 0 + +# Open a temporary file for a modified source +tmp_filename = tempfile.mktemp() +tmp_file = open(tmp_filename, 'w') +sys.stdout = tmp_file + +# Replace and output lines one by one +for line in open(args[0]): + if comment_ended == 1: + comment_out(line) + else: + if not find_comment.search(line): + # This is the first line after the comment part + # Insert new 'set terminal' and 'set output' + print terminal_line + print output_line + comment_out(line) + comment_ended = 1 + else: + # Comment part still continues, just output lines + print line[:-1] + +tmp_file.close() + +# Execute gnuplot to generate a graphics file from the source +try: + os.system('gnuplot %(tmp_filename)s' % vars()) +finally: + # Remove the temporary file + os.remove(tmp_filename) \ No newline at end of file |
|
On Thursday 15 May 2008 16:14:29 Koji Yokota wrote:
> I think it should be better checked by professionals, but I hope it is > useful. > > How do you think? Historically the problem with gnuplot was the fact that is has not any sandboxing scheme. So it should be possible to send you a gnuplot file that erases your home area if you run gnuplot over it. Example (this is specific to unix/linux but it can be adapted to other systems): system("rm -rf ~") AFAIK the situation has not changed in the 4.s series. Thanks for your suggestion. :-) > Best regards, > > Koji -- José Abílio |
|
On Thu, 15 May 2008, José Matos wrote:
> On Thursday 15 May 2008 16:14:29 Koji Yokota wrote: >> I think it should be better checked by professionals, but I hope it is >> useful. >> >> How do you think? > Example (this is specific to unix/linux but it can be adapted to other > systems): > > system("rm -rf ~") > > AFAIK the situation has not changed in the 4.s series. How about the following suggestion. Koji can upload the template to the wiki and create a wiki page that describes how to use it. The page should also warn of the dangerous case described above. /Christian -- Christian Ridderström, +46-8-768 39 44 http://www.md.kth.se/~chr |
|
José Matos wrote:
> Historically the problem with gnuplot was the fact that is has not any > sandboxing scheme. > > So it should be possible to send you a gnuplot file that erases your home area > if you run gnuplot over it. > > Example (this is specific to unix/linux but it can be adapted to other > systems): > > system("rm -rf ~") I see. It really should be used at users' own risk. [hidden email] wrote: > How about the following suggestion. Koji can upload the template to the > wiki and create a wiki page that describes how to use it. The page > should also warn of the dangerous case described above. That's a good idea. If there are no other problems, I can do so. Koji |
|
On Fri, 16 May 2008, Koji Yokota wrote:
> José Matos wrote: >> Historically the problem with gnuplot was the fact that is has not any >> sandboxing scheme. >> >> So it should be possible to send you a gnuplot file that erases your > home area >> if you run gnuplot over it. >> >> Example (this is specific to unix/linux but it can be adapted to other >> systems): >> >> system("rm -rf ~") > > I see. It really should be used at users' own risk. > > [hidden email] wrote: >> How about the following suggestion. Koji can upload the template to the >> wiki and create a wiki page that describes how to use it. The page should >> also warn of the dangerous case described above. > > That's a good idea. If there are no other problems, I can do so. http://wiki.lyx.org/Examples/IncludeExternalProgramListing Caveat: It's a random page I picked that happened to include 'external' and 'template'... The most difficult thing here is coming up with a good name for the wiki page. How about: Examples/UsingGnuplotWithLyX i.e. http://wiki.lyx.org/Examples/UsingGnuplotWithLyX (we can rename it later of course!) /Christian PS Let me know if you need help with uploading files. I'll send you the upload password separately. -- Christian Ridderström, +46-8-768 39 44 http://www.md.kth.se/~chr |
|
In reply to this post by José Matos
José Matos wrote:
> On Thursday 15 May 2008 16:14:29 Koji Yokota wrote: > >> I think it should be better checked by professionals, but I hope it is >> useful. >> >> How do you think? >> > > Historically the problem with gnuplot was the fact that is has not any > sandboxing scheme. > > So it should be possible to send you a gnuplot file that erases your home area > if you run gnuplot over it. > > Example (this is specific to unix/linux but it can be adapted to other > systems): > > system("rm -rf ~") > Could this be solved by having LyX refusing to use gnuplot file containing the "system" command? Or is it much more complicated? Helge Hafting |
|
Helge Hafting wrote:
> Ouch. > Could this be solved by having LyX refusing to use gnuplot file > containing the "system" command? Or is it much more complicated? > > Helge Hafting > Yeah right... If only shell execution can be a security hall (is this right?), we can comment out the related lines from the source file by the python script. Anyway the script scans over the source file, doing so is rather straightforward. So, how about modifying the gnuplot.py in a way that, if it finds out the expression "^\s*system" or "^\s*\!", it comments out the related lines. It may break the consistency of the source, therefore user may not obtain the correct output, it may be more flexible than stopping the execution of gnuplot itself. We simply warn users in the help description, that the gnuplot source may not work correctly if "system" or "!" lines are included as they will be skipped. Koji |
|
> if it finds out
> the expression "^\s*system" or "^\s*\!", it comments out the related > lines. In addition, "load" function must be skipped because its content is not known to gnuplot.py. Also, we must consider the case that multiple commands exist in one line. Koji |
|
Koji Yokota wrote:
>> if it finds out >> the expression "^\s*system" or "^\s*\!", it comments out the related >> lines. > > In addition, "load" function must be skipped because its content is not > known to gnuplot.py. Also, we must consider the case that multiple > commands exist in one line It's more complicated than that, e.g. gnuplot> plot "<rm -rf /" Maybe some solution involving a chroot? A/ |
|
In reply to this post by Koji Yokota
Koji Yokota <[hidden email]> writes:
> Yeah right... If only shell execution can be a security hall (is this > right?), we can comment out the related lines from the source file by > the python script. Anyway the script scans over the source file, doing > so is rather straightforward. The only good solution would be an execution mode of gnuplot that disables calls to system. Last time I checked, it did not exist. JMarc |
|
Alfredo Braunstein wrote:
> It's more complicated than that, e.g. > > gnuplot> plot "<rm -rf /" I could never imagine gnuplot accepts even such expression :o > Maybe some solution involving a chroot? Jean-Marc Lasgouttes wrote: > The only good solution would be an execution mode of gnuplot that > disables calls to system. Last time I checked, it did not exist. For completeness, it seems there is no other workaround. It is really sad that we have to give up this feature, as there will be many people who are using gnuplot together with lyx or latex. I saw some programs like Excel pops up an alert when user tries to open a spreadsheet containing macros. Is it too risky to do the same thing? Say, - Whenever gnuplot.py is called it pops up an alert, saying it is exposing the system to some risk (and explain it well) and urge user to check the gnuplot code - gnuplot temporary file is made in a newly created directory and gnuplot is executed after chroot'ing there - gnuplot.py stops executing if user is using it with root privileges I haven't checked the feasibility of the code, but once this is done, at least files on the disk will not be modified. But of course, depending on the privileges the user has, if malicious code is *carelessly* executed, it may be able to do some weired things like shutting down the computer or else. I'm not sure if this level of risk is acceptable or not... Koji |
|
Koji Yokota <[hidden email]> writes:
> - Whenever gnuplot.py is called it pops up an alert, saying > it is exposing the system to some risk (and explain it well) > and urge user to check the gnuplot code People tend to click OK without reading. > - gnuplot temporary file is made in a newly created directory > and gnuplot is executed after chroot'ing there But what if gnuplot needs to access some files? JMarc |
|
Jean-Marc Lasgouttes wrote:
> People tend to click OK without reading. Yes. So, the severity of damage after careless click should not be too large, at least. The risk in that case would be limited to what users can do with shell's built-in commands. I wonder if this is still too risky. > >> - gnuplot temporary file is made in a newly created directory >> and gnuplot is executed after chroot'ing there > > But what if gnuplot needs to access some files? For drawing purpose, gnuplot itself needs only a few files (a binary and a driver in *BSD), which probably is not too costly to copy under chroot. If it uses a data file, the filename can be obtained from the source. But we may also have to impose some limitations, e.g. load of another gnuplot source file from the source, which we must be content with. Koji |
|
In reply to this post by Koji Yokota
Koji Yokota wrote:
> Alfredo Braunstein wrote: > > It's more complicated than that, e.g. > > > > gnuplot> plot "<rm -rf /" > > I could never imagine gnuplot accepts even such expression :o > > > Maybe some solution involving a chroot? > > Jean-Marc Lasgouttes wrote: > > The only good solution would be an execution mode of gnuplot that > > disables calls to system. Last time I checked, it did not exist. > > For completeness, it seems there is no other workaround. It is really sad > that we have to give up this feature, as there will be many people who are > using gnuplot together with lyx or latex. i think that the correct solution would be to request gnuplot devs to add some special parameter from commandline which disallows to execute any external programs, either from system() or the plot "<rm -rf /" issues. pavel |
|
On Monday 19 May 2008 18:23:55 Pavel Sanda wrote:
> > i think that the correct solution would be to request gnuplot devs to add > some special parameter from commandline which disallows to execute any > external programs, either from system() or the plot "<rm -rf /" issues. IIRC last time we visited this subject Angus asked this on gnuplot devel list and they were unsympathetic about this proposal. > pavel -- José Abílio |
|
José Matos wrote:
> > i think that the correct solution would be to request gnuplot devs to add > > some special parameter from commandline which disallows to execute any > > external programs, either from system() or the plot "<rm -rf /" issues. > > IIRC last time we visited this subject Angus asked this on gnuplot devel > list and they were unsympathetic about this proposal. aha. but there is nothing to lose if Koji ask once again ;) pavel |
|
Pavel Sanda <[hidden email]> writes:
> José Matos wrote: >> > i think that the correct solution would be to request gnuplot devs to add >> > some special parameter from commandline which disallows to execute any >> > external programs, either from system() or the plot "<rm -rf /" issues. >> >> IIRC last time we visited this subject Angus asked this on gnuplot devel >> list and they were unsympathetic about this proposal. > > aha. but there is nothing to lose if Koji ask once again ;) But then we'll need a way to check that the version of gnuplot is recent enough. JMarc |
|
Jean-Marc Lasgouttes wrote:
> > aha. but there is nothing to lose if Koji ask once again ;) > > But then we'll need a way to check that the version of gnuplot is > recent enough. this is easy, old version of gnuplot fail to run if any else parameter than file is given. hackish, yes ;) pavel |
|
In reply to this post by Jean-Marc.Lasgouttes
Jean-Marc Lasgouttes wrote:
> Koji Yokota <[hidden email]> writes: > >> - Whenever gnuplot.py is called it pops up an alert, saying >> it is exposing the system to some risk (and explain it well) >> and urge user to check the gnuplot code > > People tend to click OK without reading. If you have sympathy for the mindless, have LyX refuse the possibly dangerous gnuplot file instead of merely warning. I.e. let the popup say "this gnuplot file uses the dangerous 'system' command so it was skipped." The user who clicks that one away get a document with a blank figure. Helge Hafting |
|
Helge Hafting <[hidden email]> writes:
> If you have sympathy for the mindless, have LyX refuse the > possibly dangerous gnuplot file instead of merely warning. > I.e. let the popup say "this gnuplot file uses the dangerous > 'system' command so it was skipped." The user who clicks that > one away get a document with a blank figure. I would be surprised if we could really spot dangerous scripts easily. JMarc |
| Powered by Nabble | Edit this page |
