[Pywps-devel] Problems with PyWPS/GRASS
Jachym Cepicky
jachym.cepicky at gmail.com
Mon Jan 31 15:13:03 CET 2011
hi,
Stephan Breisig píše v Po 31. 01. 2011 v 14:37 +0100:
> Hi Jachym,
>
> thanks a lot! Now it really works!
>
> For a linux/GRASS/python beginner it isn´t so easy to handle all these
> environment variables and location/mapset/path settings. There are still
> many things I have to learn and I hope I can ask you again should
> further questions arise.
yes, it is complicated .. and did you already think about the
client-side?
>
> Regarding the GRASS python scripting library, I think I will focus on
> the self.cmd() approachfor the moment.
not so nice, but should work (me personally will would have like to have
a look at the new grass-python stuff).
>
> In any case, PyWPS is really a very nice and helpfull tool!
Nice to hear this - good luck
jachym
>
> Best regards
>
> Stephan
>
>
>
>
> Am 31.01.2011 12:53, schrieb Jachym Cepicky:
> > Hi,
> >
> > I setuped the environemnt (ubuntu, grass 6.4 from packages) and tested
> > your script.
> >
> > First problem is, when you are working with g.copy, you are copying
> > vector file 'testmap' within CURRENT mapset, which is 'newmapset' in
> > that moment. g.copy gives you just warning (yes, should be displayed
> > somewhere), but does not fail. so process is successfully calculated,
> > but nothing happend.
> >
> > all you need to do is call
> >
> > self.cmd(["g.copy", "vect=testmap@%(mapset)s,testout" %
> > ({'mapset':self.grassMapset})])
> >
> >
> > than it should work
> >
> > for the second part:
> >
> > GRASS environment is initialized second before execute is called of the
> > process, which is GRASS demanding. If you try import grass.script at the
> > beginning of the file, it will fail from two reasons:
> >
> > 1) "/usr/lib/grass64/etc/python/" is not in sys.path
> > 2) os.getenv("GISBASE") returns null, because gisbase is not
> > initialized yet
> >
> > so, all you have to do is, put this as first steps in execute():
> >
> > def execute(self):
> > sys.path.append("/usr/lib/grass64/etc/python/")
> > import grass.script as grass
> >
> >
> > than it should work.
> >
> > PyWPS is much older, than the GRASS-python interface. We did not
> > prepared PyWPS for this new stuff yet - would be nice of course. This
> > way, it should be at least usable
> >
> > Jachym
> >
> > Stephan Breisig píše v Ne 30. 01. 2011 v 22:40 +0100:
> >> Hi Jachym, hi Jorge,
> >>
> >> thank you for your response!
> >>
> >> Meanwhile I have made many "experiments" especially with respect to
> >> creating a new mapset and copying the calculated map there. I also
> >> finally commented out most of the environment variable settings in the
> >> pywps.cgi file; interestingly, at the beginning I had only introduced
> >> them in the wrapper script because the pywps.cfg GRASS-settings alone
> >> were not accepted - now everythings works fine with only the pywps.cfg
> >> settings.
> >>
> >> My test process class looks like this now:
> >>
> >> #!usr/bin/env python
> >>
> >>
> >> from pywps.Process.Process import WPSProcess
> >> from types import *
> >>
> >> #import grass.script as grass
> >> import sys
> >>
> >> class Process(WPSProcess):
> >>
> >> def __init__(self):
> >> WPSProcess.__init__(self,
> >> identifier = "test",
> >> title = "",
> >> storeSupported = True,
> >> statusSupported = True,
> >> abstract = "",
> >> grassLocation = "xy_unreferenziert"
> >> )
> >>
> >> def execute(self):
> >> self.cmd(["v.in.ascii", "-zt", "input=/var/www/testdaten.csv",
> >> "output=testmap", "fs=','", "x=1", "y=2", "z=3", "cat=3"])
> >> self.cmd(["g.mapset", "-c", "mapset=newmapset",
> >> "location=xy_unreferenziert",
> >> "gisdbase=/home/stephanbreisig/grassdata/"])
> >> #location and gisdbase not necessary; entries stem from a first
> >> approach using "grassLoacation=True" in the initialisation part to
> >> which I may go back later
> >> #self.cmd(["g.region", "-d"]) --> had to be included in a first
> >> step due to the error message "default region not set"
> >> self.cmd(["g.copy", "vect=testmap,testout"])
> >> return
> >>
> >>
> >> When I run the version shown above there are at least no error
> >> messages ("PyWPS Process test successfully calculated") and the new
> >> mapset "newmapset" is indeed created within the xy_unreferenziert
> >> location. But it is empty except for the WIND file. Do you have any
> >> idea what went wrong?
> >>
> >> I have already tried to list all vectormaps (by g.list -f type=vect)
> >> after the first line of code (of course with self.textOut() set) in
> >> order to find out if the v.in.ascii command actually worked. The
> >> result was another error message:
> >>
> >> XML-Verarbeitungsfehler: Nicht übereinstimmendes Tag. Erwartet:
> >> </pywpstu_2At>.
> >> (english translation: "XML-processing error: Non-corresponding tag.
> >> Expected:</pywps...>")
> >> Adresse:
> >> http://localhost/cgi-bin/pywps.cgi?Service=WPS&Version=1.0.0&Request=Execute&Identifier=test
> >> Zeile Nr. 21, Spalte 3:</wps:LiteralData>
> >> --^
> >>
> >> A completely different but nonetheless big question is, why I can not
> >> use the GRASS python scriting library. When I try to import the
> >> grass.script module (commented out above), an error message states
> >> that there is no such module. Maybe this has something to do with the
> >> GISRC variable (which now is commented out) or some other
> >> python-related settings. In an earlier stage of my tests I
> >> systematically reported the results of using/commenting out the export
> >> of GISRC in the pywps.cgi wrapper script and the use of
> >> grass.script/scritping library commands respectively. I send the
> >> report as a pdf-file in a seperate mail to your gmail-adress, Jachym!
> >> Maybe it is of general interest for you or you can at least easily
> >> identify the mistake(s) I made. :-)
> >>
> >> Best regards,
> >>
> >> Stephan
> >>
> >>
> >> Am 29.01.2011 22:21, schrieb Jachym Cepicky:
> >>> Hi,
> >>> PyWPS was actually written with direct support for GRASS GIS, to make
> >>> the things as easy as possible for the user (at least from mine point of
> >>> view)
> >>>
> >>> in general, in your script, you should not set all the env. variables,
> >>> you did - PyWPS should do the job for you.
> >>>
> >>> 1) First you have to setup the [grass] section in pywps.cfg file (as
> >>> described in
> >>> http://pywps.wald.intevation.org/documentation/pywps-3.2/configuration/index.html#grass
> >>>
> >>> 2) While initializing the process, you can set grassLocation
> >>> configuration option to:
> >>>
> >>> False (default) - GRASS environment will not be initialized
> >>> True - Temporary GRASS Location&& Mapset will be created and GRASS
> >>> Environment will be initialized
> >>> String - existing GRASS Location will be used, in which *temporary*
> >>> Mapset will be created
> >>> (as described in
> >>> http://pywps.wald.intevation.org/documentation/pywps-3.2/special/grass.html#allowing-process-to-be-executed-in-grass-environment)
> >>>
> >>> Then, in the execute() method, you can call
> >>>
> >>> out = self.cmd(["g.list","rast"]) and in variable "out", result of
> >>> "g.list rast" will be stored.
> >>>
> >>> NOTE: PyWPS creates *temporary* location and/or mapset, which is
> >>> deleted, after the calculation is done. If you want to store any map,
> >>> you have switch to target location (g.mapset) and copy the map there
> >>> (g.copy).
> >>>
> >>> Is that clear? Please ask, in case you need to clarify something else.
> >>>
> >>> Jachym
> >>>
> >>> Jorge de Jesus píše v Pá 28. 01. 2011 v 08:42 +0000:
> >>>> Hi
> >>>>
> >>>> Don't worry about verbose emails, the more info the better :-)
> >>>>
> >>>> Anyway I don't work that much with GRASS, maybe Jachym/Luca/Soeren
> >>>> should know better how to deal with your problem
> >>>>
> >>>> I am not so certain about GISRC, but I think, that it contains
> >>>> variables used in the GRASS session, maybe it overwrites some of your
> >>>> parameters ?! But I am not certain since we normally don't set in the
> >>>> GRASS-PyWPS configuration
> >>>>
> >>>> I look at the wiki http://wiki.rsg.pml.ac.uk/pywps/index.php/GRASS and
> >>>> there is a note:
> >>>>
> >>>> PyWPS will try to use the following path syntax, to use/create a GRASS
> >>>> location:
> >>>>
> >>>> $GISDBASE/$LOCATION_NAME/$MAPSET
> >>>>
> >>>> Looking at your variables I think that you will end up with a location
> >>>> like this:
> >>>>
> >>>> /home/stephanbreisig/grassdata/home/stephanbreisig/grassdata/xy_unreferenziert
> >>>>
> >>>> I think that $LOCATION_NAME should just be: xy_unreferenziert (without
> >>>> $GISDBASE)
> >>>>
> >>>> And the final warning:
> >>>>
> >>>> pywps.cfg will have PRECEDENCE. Any GRASS configuration in the
> >>>> pywps.cfg will overwrite the variables in your script
> >>>>
> >>>> I kindly ask you to inform the mailing list when you find the solution
> >>>> to your problem GRASS-PyWPS is a bit complicated and I am trying to
> >>>> compile documentation to help all the "newbies"
> >>>>
> >>>> All the best
> >>>>
> >>>> Jorge
> >>>>
> >>>> On 27/01/11 23:51, Stephan Breisig wrote:
> >>>>> Hello,
> >>>>>
> >>>>> for my master thesis I want to use GRASS via PyWPS. Unfortunatelly,
> >>>>> this seems to be as problematic as one might expect for a
> >>>>> Linux/GRASS/Python (almost) beginner like me. I hope you will not be
> >>>>> too much annoyed with the quite long mail, but I think the more
> >>>>> information you get the better! Here are the details:
> >>>>>
> >>>>> What I use: Ubuntu 10.04.1, GRASS 6.4.0-2, Python 2.6.5, PyWPS 3.1.0
> >>>>> and Apache 2.2.14; WPS is run on localhost
> >>>>>
> >>>>> My pywps.cgi wrapper script looks like this:
> >>>>>
> >>>>> #! /bin/sh
> >>>>>
> >>>>> export
> >>>>> PYWPS_CFG=/usr/local/lib/python2.6/dist-packages/pywps/pywps.cfg
> >>>>> export
> >>>>> PYWPS_PROCESSES=/usr/local/lib/python2.6/dist-packages/pywps/processes
> >>>>>
> >>>>> export GISBASE=/usr/lib/grass64/
> >>>>> export GISDBASE=/home/stephanbreisig/grassdata/
> >>>>>
> >>>>> export PATH=
> >>>>> $PATH:/$GISBASE/bin:/$GISBASE/scripts:/$GISBASE/etc:/$GISBASE/lib:/usr/lib/python2.6:/$GISBASE/etc/python/grass:/$GISBASE/etc/python/grass/lib:/$GISBASE/etc/python/grass/script
> >>>>> #export GISRC=$HOME/.grassrc6
> >>>>>
> >>>>> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$GISBASE/lib
> >>>>> export PYTHONPATH=$GISBASE/etc/python
> >>>>> export PYTHONLIB=/usr/lib/python2.6
> >>>>>
> >>>>> export LOCATION_NAME=$GISDBASE/xy_unreferenziert
> >>>>> export GIS_LOCK=$$
> >>>>>
> >>>>> /usr/local/bin/wps.py
> >>>>>
> >>>>> And this is a simple test process causing problems:
> >>>>>
> >>>>> #!usr/bin/env python
> >>>>>
> >>>>> from pywps.Process.Process import WPSProcess
> >>>>> from types import *
> >>>>> #import grass.script as grass
> >>>>> import sys
> >>>>>
> >>>>> class Process(WPSProcess):
> >>>>>
> >>>>> def __init__(self):
> >>>>> WPSProcess.__init__(self, identifier = "test", title = "csv to
> >>>>> vectormap", storeSupported = "true", statusSupported = "true",
> >>>>> grassLocation = "xy_unreferenziert")
> >>>>>
> >>>>> def execute(self):
> >>>>> testmap = self.cmd(["v.in.ascii", "-zt",
> >>>>> "input=/var/www/testdaten.csv", "output=testmap", "fs=','", "x=1",
> >>>>> "y=2", "z=3", "cat=3"])
> >>>>> return
> >>>>>
> >>>>> I can only get any reasonable results (for example just a correct
> >>>>> output for g.version) when I use self.cmd() instead of GRASSpython
> >>>>> scripting library commands in the process and when GISRC is
> >>>>> commented out in pywps.cgi.
> >>>>> The output for the process above on firefox actually looks fine (no
> >>>>> error messsages, process successful), nevertheless in the GRASS
> >>>>> location specified in the process
> >>>>> (/home/stephanbreisig/grassdata/xy_unreferenziert/) there is just
> >>>>> the PERMANENT mapset as before and nothing else has happened.
> >>>>> Browsing the documentations and wikis etc. for GRASS and PyWPS
> >>>>> (especially regarding environment variables) turned out to be of
> >>>>> little help.
> >>>>>
> >>>>>
> >>>>> Many thanks in advance for your help!
> >>>>>
> >>>>>
> >>>>> Best regards,
> >>>>>
> >>>>> Stephan Breisig
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> Überprüft mit AntiVir MailGuard v10.0.1.38 AVE 8.2.4.150 VDF 7.11.2.16
> >>>>>
> >>>>> _______________________________________________
> >>>>> Pywps-devel mailing list
> >>>>> Pywps-devel at wald.intevation.org
> >>>>> http://lists.wald.intevation.org/mailman/listinfo/pywps-devel
> >>>>
> >>>> ______________________________________________________________________
> >>>>
> >>>> Plymouth Marine Laboratory
> >>>> Registered Office:
> >>>> Prospect Place
> >>>> The Hoe
> >>>> Plymouth PL1 3DH
> >>>>
> >>>> Website: www.pml.ac.uk
> >>>> Registered Charity No. 1091222
> >>>> PML is a company limited by guarantee
> >>>> registered in England& Wales
> >>>> company number 4178503
> >>>>
> >>>> Please think before you print.
> >>>>
> >>>>
> >>>> ______________________________________________________________________
> >>>>
> >>>> This e-mail, its content and any file attachments are confidential.
> >>>>
> >>>> If you have received this e-mail in error please do not copy, disclose
> >>>> it to any third party or use the contents or attachments in any way.
> >>>> Please notify the sender by replying to this e-mail or e-mail
> >>>> forinfo at pml.ac.uk and then delete the email without making any copies
> >>>> or using it in any other way.
> >>>>
> >>>> The content of this message may contain personal views which are not
> >>>> the views of Plymouth Marine Laboratory unless specifically stated.
> >>>>
> >>>> You are reminded that e-mail communications are not secure and may
> >>>> contain viruses. Plymouth Marine Laboratory accepts no liability for
> >>>> any loss or damage which may be caused by viruses.
> >>>>
> >>>>
> >>>> ______________________________________________________________________
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> Pywps-devel mailing list
> >>>> Pywps-devel at wald.intevation.org
> >>>> http://lists.wald.intevation.org/mailman/listinfo/pywps-devel
> >>
> >> --
> >>
> >> Überprüft mit AntiVir MailGuard v10.0.1.38 AVE 8.2.4.158 VDF 7.11.2.32
>
> Überprüft mit AntiVir MailGuard v10.0.1.38 AVE 8.2.4.158 VDF 7.11.2.37
--
Jachym Cepicky
e-mail: jachym.cepicky gmail com
URL: http://les-ejk.cz
PGP Public key: http://les-ejk.cz/pgp/JachymCepicky.pgp
More information about the Pywps-devel
mailing list