[Osaas-commits] r40 - in trunk: . server server/osaas server/test
scm-commit@wald.intevation.org
scm-commit at wald.intevation.org
Thu Nov 20 15:26:56 CET 2008
Author: bh
Date: 2008-11-20 15:26:54 +0100 (Thu, 20 Nov 2008)
New Revision: 40
Modified:
trunk/ChangeLog
trunk/server/README.txt
trunk/server/demo-config.xml
trunk/server/osaas/dbbackend.py
trunk/server/osaas/dbinit.py
trunk/server/osaas/run.py
trunk/server/test/test_dbbackend.py
Log:
Add support for PostGreSQL databases in addition to the Oracle
database support.
* server/README.txt: Add note about psycopg2
* server/demo-config.xml: Add DBConnection settings Type and
Parameters and corresponding explanations
* server/osaas/dbbackend.py (oracle_sequence_nextval)
(postgresql_sequence_nextval): Two new functions to help with the
oracle/postgresql abstraction of getting the next value from a
sequence
(load_db_api): New function to load a DB API on demand
(DBBackend.__init__): Pass the db_api to the DBBackend as a
parameter instead of getting it from a global variable. Also,
connection_parameters may include a string with the all connection
parameters as used by e.g. psycopg2.
(DBBackend.connect): Use the intance's db_api, not a global dbapi
module
(DBBackend.named_parameter): New method to help building SQL
statements depending on the parameter style of the db api used by
the instance
* server/osaas/run.py (OSAASProgramMixin.add_db_options): Add db
connection parameter Parameters
(OSAASServerProgram.instantiate_server): Adapt to dbbackend
changes: load db api and pass opts.db_parameters to the backend
* server/osaas/dbinit.py (DBInitProgram.main): Adapt to dbbackend
changes: load db api and pass opts.db_parameters to the backend
* server/test/test_dbbackend.py (MockDbApi, TestBackend): Instead
of TestBackend, a special test version of the DBBackend class, use
a mock DB-API module that's passed to a vanilla DBBackend.
(TestDBBackend.new_backend): New method to make instantiating the
DBBackend a little easier in the test cases
(TestDBBackend.test_create_tables)
(TestDBBackend.test_create_tables_error)
(TestDBBackend.test_handle_record)
(TestDBBackend.test_handle_record_error): Adapt test cases to the
dbbackend and test infrastructure changes.
Modified: trunk/ChangeLog
===================================================================
--- trunk/ChangeLog 2008-11-19 11:53:52 UTC (rev 39)
+++ trunk/ChangeLog 2008-11-20 14:26:54 UTC (rev 40)
@@ -1,3 +1,47 @@
+2008-11-20 Bernhard Herzog <bh at intevation.de>
+
+ Add support for PostGreSQL databases in addition to the Oracle
+ database support.
+
+ * server/README.txt: Add note about psycopg2
+
+ * server/demo-config.xml: Add DBConnection settings Type and
+ Parameters and corresponding explanations
+
+ * server/osaas/dbbackend.py (oracle_sequence_nextval)
+ (postgresql_sequence_nextval): Two new functions to help with the
+ oracle/postgresql abstraction of getting the next value from a
+ sequence
+ (load_db_api): New function to load a DB API on demand
+ (DBBackend.__init__): Pass the db_api to the DBBackend as a
+ parameter instead of getting it from a global variable. Also,
+ connection_parameters may include a string with the all connection
+ parameters as used by e.g. psycopg2.
+ (DBBackend.connect): Use the intance's db_api, not a global dbapi
+ module
+ (DBBackend.named_parameter): New method to help building SQL
+ statements depending on the parameter style of the db api used by
+ the instance
+
+ * server/osaas/run.py (OSAASProgramMixin.add_db_options): Add db
+ connection parameter Parameters
+ (OSAASServerProgram.instantiate_server): Adapt to dbbackend
+ changes: load db api and pass opts.db_parameters to the backend
+
+ * server/osaas/dbinit.py (DBInitProgram.main): Adapt to dbbackend
+ changes: load db api and pass opts.db_parameters to the backend
+
+ * server/test/test_dbbackend.py (MockDbApi, TestBackend): Instead
+ of TestBackend, a special test version of the DBBackend class, use
+ a mock DB-API module that's passed to a vanilla DBBackend.
+ (TestDBBackend.new_backend): New method to make instantiating the
+ DBBackend a little easier in the test cases
+ (TestDBBackend.test_create_tables)
+ (TestDBBackend.test_create_tables_error)
+ (TestDBBackend.test_handle_record)
+ (TestDBBackend.test_handle_record_error): Adapt test cases to the
+ dbbackend and test infrastructure changes.
+
2008-11-19 Stephan Holl <stephan.holl at intevation.de>
* contrib/osaas, contrib/README.txt: New. Added SuSE-specific
Modified: trunk/server/README.txt
===================================================================
--- trunk/server/README.txt 2008-11-19 11:53:52 UTC (rev 39)
+++ trunk/server/README.txt 2008-11-20 14:26:54 UTC (rev 40)
@@ -14,9 +14,14 @@
Python >= 2.3 http://python.org/, comes with most Linux Distributions
+And one of the following database API modules, depending on which
+database you use:
+
cx_Oracle >= 4.3.1 http://www.python.net/crew/atuining/cx_Oracle/
+ psycopg2 >= 2.0.5.1 http://www.initd.org/pub/software/psycopg/
+
Configuration
-------------
Modified: trunk/server/demo-config.xml
===================================================================
--- trunk/server/demo-config.xml 2008-11-19 11:53:52 UTC (rev 39)
+++ trunk/server/demo-config.xml 2008-11-20 14:26:54 UTC (rev 40)
@@ -15,6 +15,15 @@
<!-- The database connection parameters -->
<DBConnection>
+ <!-- Type of the database. Possible values: oracle, postgresql -->
+ <Type>postgresql</Type>
+
+ <!-- Connection parameters as a single string. Use this for postgresql. -->
+ <Parameters>user=mydbuser password=mydbpassword host=localhost</Parameters>
+
+ <!-- Connection parameters for Oracle. Make sure Parameters is
+ empty or not present at all when using oracle. The Parameters
+ entry takes precedence over the following parameters. -->
<User>mydbuser</User>
<Password>mydbpassword</Password>
<DSN>myDSN</DSN>
Modified: trunk/server/osaas/dbbackend.py
===================================================================
--- trunk/server/osaas/dbbackend.py 2008-11-19 11:53:52 UTC (rev 39)
+++ trunk/server/osaas/dbbackend.py 2008-11-20 14:26:54 UTC (rev 40)
@@ -1,4 +1,4 @@
-# Copyright (C) 2007 by Intevation GmbH
+# Copyright (C) 2007, 2008 by Intevation GmbH
# Authors:
# Bernhard Herzog <bh at intevation.de>
#
@@ -7,16 +7,39 @@
"""Database backend of OSAAS"""
-try:
- import cx_Oracle as dbapi
-except ImportError:
- class dbapi(object):
- def connect(*args):
- raise RuntimeError("Trying to call DBAPI methods"
- " without a DBAPI module")
- connect = staticmethod(connect)
+def oracle_sequence_nextval(name):
+ """Returns the Oracle SQL expression fetching the next value of a sequence.
+ """
+ return "%s.nextval" % name
+def postgresql_sequence_nextval(name):
+ """Returns the POstGreSQL expression fetching the next value of a sequence.
+ """
+ return "nextval('%s')" % name
+
+db_api_names = dict(
+ postgresql=("psycopg2", dict(osaas_nextval=postgresql_sequence_nextval)),
+ oracle=("cx_Oracle", dict(osaas_nextval=oracle_sequence_nextval))
+)
+
+def load_db_api(dbtype):
+ """Loads the DB API module indicated by the dbtype parameter. The
+ dbtype is looked up in the db_api_names dict, which matches the
+ dbtype to a concrete python module name, among other things. The
+ function returns the python module with the DB API.
+ """
+ modulename, extra_api = db_api_names.get(dbtype, (None, None))
+ if modulename is None:
+ raise ValueError("Unknown dbtype: %r; must be one of %s"
+ % (dbtype, ", ".join(db_api_names.keys())))
+ module = __import__(modulename)
+ for key, value in extra_api.items():
+ setattr(module, key, value)
+
+ return module
+
+
class DBField(object):
def __init__(self, parameter, column, column_type):
@@ -34,8 +57,14 @@
class DBBackend(object):
- def __init__(self, connection_parameters, rules, logger=None):
+ def __init__(self, db_api, connection_parameters, rules, logger=None):
+ self.db_api = db_api
self.connection = None
+ params, user, password, dsn = connection_parameters
+ if params:
+ connection_parameters = (params,)
+ else:
+ connection_parameters = (user, password, dsn)
self.connection_parameters = connection_parameters
self.rules = rules
self.logger = logger
@@ -54,15 +83,29 @@
self.connection = self.connect(*self.connection_parameters)
def connect(self, *args):
- return dbapi.connect(*args)
+ return self.db_api.connect(*args)
+ def named_parameter(self, name):
+ """Return a named parameter reference according to the db_api used"""
+ paramstyle = self.db_api.paramstyle
+ if paramstyle == "named":
+ template = ":%s"
+ elif paramstyle == "pyformat":
+ template = "%%(%s)s"
+ else:
+ raise RuntimeError("Unsupported DB-API paramstyle: %r", paramstyle)
+
+ return template % name
+
def handle_record(self, record):
self.ensure_connection()
rule = self.rules[0]
names = ", ".join([f.column for f in rule.fields])
- statement = ("INSERT INTO %s (ID, %s) VALUES (%s.nextval, %s)"
- % (rule.table, names, rule.table + "_seq",
- ", ".join([":%s" % f.column for f in rule.fields])))
+ statement = ("INSERT INTO %s (ID, %s) VALUES (%s, %s)"
+ % (rule.table, names,
+ self.db_api.osaas_nextval(rule.table + "_seq"),
+ ", ".join([self.named_parameter(f.column)
+ for f in rule.fields])))
values = {}
for f in rule.fields:
values[f.column] = getattr(record, f.parameter)
Modified: trunk/server/osaas/dbinit.py
===================================================================
--- trunk/server/osaas/dbinit.py 2008-11-19 11:53:52 UTC (rev 39)
+++ trunk/server/osaas/dbinit.py 2008-11-20 14:26:54 UTC (rev 40)
@@ -1,4 +1,4 @@
-# Copyright (C) 2007 by Intevation GmbH
+# Copyright (C) 2007, 2008 by Intevation GmbH
# Authors:
# Bernhard Herzog <bh at intevation.de>
#
@@ -9,7 +9,7 @@
from http.run import ProgramWithOptions
from run import OSAASProgramMixin
-from dbbackend import DBBackend
+from dbbackend import DBBackend, load_db_api
class DBInitProgram(OSAASProgramMixin, ProgramWithOptions):
@@ -20,8 +20,9 @@
def main(self):
opts, rest = self.parse_options(sys.argv[1:])
-
- backend = DBBackend((opts.db_user, opts.db_password, opts.db_dsn),
+ backend = DBBackend(load_db_api(opts.db_type),
+ (opts.db_parameters,
+ opts.db_user, opts.db_password, opts.db_dsn),
opts.db_rules)
backend.create_tables()
Modified: trunk/server/osaas/run.py
===================================================================
--- trunk/server/osaas/run.py 2008-11-19 11:53:52 UTC (rev 39)
+++ trunk/server/osaas/run.py 2008-11-20 14:26:54 UTC (rev 40)
@@ -1,4 +1,4 @@
-# Copyright (C) 2007 by Intevation GmbH
+# Copyright (C) 2007, 2008 by Intevation GmbH
# Authors:
# Bernhard Herzog <bh at intevation.de>
#
@@ -10,14 +10,14 @@
from osaas.http.run import ServerProgram
from osaas.server import OSAASServer
from osaas.config import OSAASOption, read_config_file
-from osaas.dbbackend import DBBackend
+from osaas.dbbackend import DBBackend, load_db_api
class OSAASProgramMixin:
optparse_option_class = OSAASOption
def add_db_options(self, parser):
- for name in ["User", "Password", "DSN"]:
+ for name in ["Type", "Parameters", "User", "Password", "DSN"]:
option = parser.add_option("--db-" + name.lower())
option.xml_path = "OSAASConfig/DBConnection/" + name
@@ -39,7 +39,9 @@
return parser
def instantiate_server(self, server_class, server_address, opts, **kw):
- dbbackend = DBBackend((opts.db_user, opts.db_password, opts.db_dsn),
+ dbbackend = DBBackend(load_db_api(opts.db_type),
+ (opts.db_parameters,
+ opts.db_user, opts.db_password, opts.db_dsn),
opts.db_rules,
logging.getLogger("httpserver.error.dbbackend"))
return server_class(server_address, userdbfile=opts.userdb_file,
Modified: trunk/server/test/test_dbbackend.py
===================================================================
--- trunk/server/test/test_dbbackend.py 2008-11-19 11:53:52 UTC (rev 39)
+++ trunk/server/test/test_dbbackend.py 2008-11-20 14:26:54 UTC (rev 40)
@@ -1,4 +1,4 @@
-# Copyright (C) 2007 by Intevation GmbH
+# Copyright (C) 2007, 2008 by Intevation GmbH
# Authors:
# Bernhard Herzog <bh at intevation.de>
#
@@ -36,16 +36,26 @@
self.testcase.method_called("Cursor.execute", statement,
*args, **kw)
-class TestBackend(DBBackend):
+class MockDbApi(object):
- def __init__(self, testcase, *args):
+ def __init__(self, testcase, connection_parameters):
self.testcase = testcase
- DBBackend.__init__(self, *args)
+ if connection_parameters[0]:
+ connection_parameters = (connection_parameters[0],)
+ else:
+ connection_parameters = connection_parameters[1:]
+ self.connection_parameters = connection_parameters
def connect(self, *args):
self.testcase.assertEquals(args, self.connection_parameters)
return MockConnection(self.testcase)
+ def osaas_nextval(self, name):
+ return "%s.nextval" % name
+
+ paramstyle = "named"
+
+
class MockLogger(object):
def __init__(self):
@@ -76,13 +86,16 @@
"""Called by method_called so that tests can simulate errors"""
pass
+ def new_backend(self, connection_parameters, *args):
+ return DBBackend(MockDbApi(self, connection_parameters),
+ connection_parameters, *args)
+
def test_create_tables(self):
- backend = TestBackend(self,
- ("someuser", "aPassword", "mydns"),
- [Insertion("mytable",
- [DBField("REQUEST", "req",
- "character varying(11)")])],
- self.logger)
+ backend = self.new_backend((None, "someuser", "aPassword", "mydns"),
+ [Insertion("mytable",
+ [DBField("REQUEST", "req",
+ "character varying(11)")])],
+ self.logger)
backend.create_tables()
self.assertEquals(self.method_log,
@@ -102,11 +115,10 @@
raise RuntimeError("Simulated database error")
self.check_method_called = check_method_called
- backend = TestBackend(self,
- ("someuser", "aPassword", "mynds"),
- [Insertion("mytable",
- [DBField("REQUEST", "req",
- "character varying(11)")])])
+ backend = self.new_backend((None, "someuser", "aPassword", "mynds"),
+ [Insertion("mytable",
+ [DBField("REQUEST", "req",
+ "character varying(11)")])])
try:
backend.create_tables()
except RuntimeError:
@@ -117,12 +129,11 @@
("Connection.rollback",)])
def test_handle_record(self):
- backend = TestBackend(self,
- ("someuser", "aPassword", "mydns"),
- [Insertion("mytable",
- [DBField("REQUEST", "req",
- "character varying(11)")])],
- self.logger)
+ backend = self.new_backend((None, "someuser", "aPassword", "mydns"),
+ [Insertion("mytable",
+ [DBField("REQUEST", "req",
+ "character varying(11)")])],
+ self.logger)
record = Record()
record.REQUEST = "SERVICE=WMS&REQUEST=GetMap"
backend.handle_record(record)
@@ -141,12 +152,11 @@
if name == "Cursor.execute" and args[0].startswith("INSERT"):
raise RuntimeError("Simulated database error")
self.check_method_called = check_method_called
- backend = TestBackend(self,
- ("someuser", "aPassword", "mydns"),
- [Insertion("mytable",
- [DBField("REQUEST", "req",
- "character varying(11)")])],
- self.logger)
+ backend = self.new_backend((None, "someuser", "aPassword", "mydns"),
+ [Insertion("mytable",
+ [DBField("REQUEST", "req",
+ "character varying(11)")])],
+ self.logger)
record = Record()
record.REQUEST = "SERVICE=WMS&REQUEST=GetMap"
record.raw_record = "REQUEST=SERVICE=WMS%26REQUEST=GetMap"
More information about the Osaas-commits
mailing list