[Openvas-commits] r11546 - in trunk/openvas-manager: . tools

scm-commit@wald.intevation.org scm-commit at wald.intevation.org
Thu Sep 1 16:52:04 CEST 2011


Author: hdoreau
Date: 2011-09-01 16:52:03 +0200 (Thu, 01 Sep 2011)
New Revision: 11546

Added:
   trunk/openvas-manager/tools/
   trunk/openvas-manager/tools/cpe_update.xsl
   trunk/openvas-manager/tools/cpe_youngerthan.xsl
   trunk/openvas-manager/tools/cve_update.xsl
   trunk/openvas-manager/tools/cve_youngerthan.xsl
   trunk/openvas-manager/tools/db_init.sql
   trunk/openvas-manager/tools/greenbone-scapdata-sync
   trunk/openvas-manager/tools/openvas-scapdata-sync
Modified:
   trunk/openvas-manager/ChangeLog
Log:
Created a `tools' directory with files required for SCAP data
management and synchronization.

* tools/cpe_update.xsl (new): Helper script which generates SQL
statements to update the SCAP database from CPE entries.

* tools/cpe_youngerthan.xsl (new): Helper script which filters CPE
entries younger than a given date.

* tools/cve_update.xsl (new): Helper script which generates SQL
statements to update the SCAP database from CVE entries.

* tools/cve_youngerthan.xsl (new): Helper script which filters CVE
entries younger than a given date.

* tools/db_init.sql (new): SCAP database initialization script.

* tools/greenbone-scapdata-sync (new): SCAP data synchronization
script for GSF.

* tools/openvas-scapdata-sync (new): SCAP data synchronization script
for the OpenVAS feed.


Modified: trunk/openvas-manager/ChangeLog
===================================================================
--- trunk/openvas-manager/ChangeLog	2011-09-01 14:51:47 UTC (rev 11545)
+++ trunk/openvas-manager/ChangeLog	2011-09-01 14:52:03 UTC (rev 11546)
@@ -1,3 +1,28 @@
+2011-09-01  Henri Doreau <henri.doreau at greenbone.net>
+
+	Created a `tools' directory with files required for SCAP data
+	management and synchronization.
+
+	* tools/cpe_update.xsl (new): Helper script which generates SQL
+	statements to update the SCAP database from CPE entries.
+
+	* tools/cpe_youngerthan.xsl (new): Helper script which filters CPE
+	entries younger than a given date.
+
+	* tools/cve_update.xsl (new): Helper script which generates SQL
+	statements to update the SCAP database from CVE entries.
+
+	* tools/cve_youngerthan.xsl (new): Helper script which filters CVE
+	entries younger than a given date.
+
+	* tools/db_init.sql (new): SCAP database initialization script.
+
+	* tools/greenbone-scapdata-sync (new): SCAP data synchronization
+	script for GSF.
+
+	* tools/openvas-scapdata-sync (new): SCAP data synchronization script
+	for the OpenVAS feed.
+
 2011-08-18  Matthew Mundell <matthew.mundell at greenbone.net>
 
 	Drop the UNIQUE constraint in task_preferences.  The task name pair is

Added: trunk/openvas-manager/tools/cpe_update.xsl
===================================================================
--- trunk/openvas-manager/tools/cpe_update.xsl	2011-09-01 14:51:47 UTC (rev 11545)
+++ trunk/openvas-manager/tools/cpe_update.xsl	2011-09-01 14:52:03 UTC (rev 11546)
@@ -0,0 +1,43 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+OpenVAS
+$Id$
+Description: Generate SQL (SQLite compatible) queries to update the CPE database.
+
+Authors:
+Henri Doreau <henri.doreau at greenbone.net>
+
+Copyright:
+Copyright (C) 2011 Greenbone Networks GmbH
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 2,
+or, at your option, any later version as published by the Free
+Software Foundation
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+-->
+<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:scap-core="http://scap.nist.gov/schema/scap-core/0.3" xmlns:meta="http://scap.nist.gov/schema/cpe-dictionary-metadata/0.2" xmlns:ns6="http://scap.nist.gov/schema/scap-core/0.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:config="http://scap.nist.gov/schema/configuration/0.1" xmlns:cpe="http://cpe.mitre.org/dictionary/2.0" xsi:schemaLocation="http://scap.nist.gov/schema/configuration/0.1 http://nvd.nist.gov/schema/configuration_0.1.xsd http://scap.nist.gov/schema/scap-core/0.3 http://nvd.nist.gov/schema/scap-core_0.3.xsd http://cpe.mitre.org/dictionary/2.0 http://cpe.mitre.org/files/cpe-dictionary_2.2.xsd http://scap.nist.gov/schema/scap-core/0.1 http://nvd.nist.gov/schema/scap-core_0.1.xsd http://scap.nist.gov/schema/cpe-dictionary-metadata/0.2 http://nvd.nist.gov/schema/cpe-dictionary-metadata_0.2.xsd">
+
+  <xsl:output method="text"/>
+
+<xsl:template match="cpe:cpe-item">INSERT OR REPLACE INTO cpes (name) VALUES ("<xsl:value-of select="@name"/>");
+</xsl:template>
+
+<xsl:template match="cpe:generator"/>
+
+<xsl:template match="/">
+BEGIN TRANSACTION;
+  <xsl:apply-templates/>
+COMMIT;
+</xsl:template>
+
+</xsl:stylesheet>
+

Added: trunk/openvas-manager/tools/cpe_youngerthan.xsl
===================================================================
--- trunk/openvas-manager/tools/cpe_youngerthan.xsl	2011-09-01 14:51:47 UTC (rev 11545)
+++ trunk/openvas-manager/tools/cpe_youngerthan.xsl	2011-09-01 14:52:03 UTC (rev 11546)
@@ -0,0 +1,39 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+OpenVAS
+$Id$
+Description: Select CPEs which have been updated after a certain date.
+
+Authors:
+Henri Doreau <henri.doreau at greenbone.net>
+
+Copyright:
+Copyright (C) 2011 Greenbone Networks GmbH
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 2,
+or, at your option, any later version as published by the Free
+Software Foundation
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+-->
+<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:scap-core="http://scap.nist.gov/schema/scap-core/0.3" xmlns:meta="http://scap.nist.gov/schema/cpe-dictionary-metadata/0.2" xmlns:ns6="http://scap.nist.gov/schema/scap-core/0.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:config="http://scap.nist.gov/schema/configuration/0.1" xmlns:cpe="http://cpe.mitre.org/dictionary/2.0" xsi:schemaLocation="http://scap.nist.gov/schema/configuration/0.1 http://nvd.nist.gov/schema/configuration_0.1.xsd http://scap.nist.gov/schema/scap-core/0.3 http://nvd.nist.gov/schema/scap-core_0.3.xsd http://cpe.mitre.org/dictionary/2.0 http://cpe.mitre.org/files/cpe-dictionary_2.2.xsd http://scap.nist.gov/schema/scap-core/0.1 http://nvd.nist.gov/schema/scap-core_0.1.xsd http://scap.nist.gov/schema/cpe-dictionary-metadata/0.2 http://nvd.nist.gov/schema/cpe-dictionary-metadata_0.2.xsd">
+
+  
+<xsl:template match="cpe:cpe-list">
+  <xsl:copy>
+    <xsl:for-each select="cpe:cpe-item[number(translate(substring(meta:item-metadata/@modification-date,1,10),'-','')) &gt; number($refdate)]">
+      <xsl:copy-of select="."/>
+    </xsl:for-each>
+  </xsl:copy>
+</xsl:template>
+
+</xsl:stylesheet>
+

Added: trunk/openvas-manager/tools/cve_update.xsl
===================================================================
--- trunk/openvas-manager/tools/cve_update.xsl	2011-09-01 14:51:47 UTC (rev 11545)
+++ trunk/openvas-manager/tools/cve_update.xsl	2011-09-01 14:52:03 UTC (rev 11546)
@@ -0,0 +1,59 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+OpenVAS
+$Id$
+Description: Generate SQL (SQLite compatible) queries to update the CVE
+database.
+
+Authors:
+Henri Doreau <henri.doreau at greenbone.net>
+
+Copyright:
+Copyright (C) 2011 Greenbone Networks GmbH
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 2,
+or, at your option, any later version as published by the Free
+Software Foundation
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+-->
+<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:vuln="http://scap.nist.gov/schema/vulnerability/0.4" xmlns:cpe-lang="http://cpe.mitre.org/language/2.0" xmlns:scap-core="http://scap.nist.gov/schema/scap-core/0.1" xmlns:cve="http://scap.nist.gov/schema/feed/vulnerability/2.0" xmlns:cvss="http://scap.nist.gov/schema/cvss-v2/0.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:patch="http://scap.nist.gov/schema/patch/0.1">
+
+<xsl:output method="text"/>
+
+<xsl:template match="cve:entry">
+  <xsl:variable name="cvss">
+    <xsl:choose>
+      <xsl:when test="vuln:cvss/cvss:base_metrics/cvss:score/text()">
+        <xsl:value-of select="number(vuln:cvss/cvss:base_metrics/cvss:score/text())"/>
+      </xsl:when>
+      <xsl:otherwise>
+        <xsl:value-of select="0"/>
+      </xsl:otherwise>
+    </xsl:choose>
+  </xsl:variable>
+  <xsl:variable name="cveid" select="@id"/>
+
+INSERT OR REPLACE INTO cves (cve,last_mod,cvss,description) VALUES ("<xsl:value-of select="$cveid"/>","<xsl:value-of select="vuln:last-modified-datetime/text()"/>",<xsl:value-of select="$cvss"/>,"<xsl:value-of select="translate(vuln:summary/text(), '&quot;', '')"/>");
+  <xsl:for-each select="vuln:vulnerable-software-list/vuln:product">
+INSERT OR IGNORE INTO cpes (name) VALUES ("<xsl:value-of select="text()"/>");
+INSERT OR REPLACE INTO affected_products (cve,cpe) VALUES ((SELECT id FROM cves WHERE cve="<xsl:value-of select="$cveid"/>"),(SELECT id FROM cpes WHERE name="<xsl:value-of select="text()"/>"));
+  </xsl:for-each>
+</xsl:template>
+
+<xsl:template match="/">
+BEGIN TRANSACTION;
+  <xsl:apply-templates/>
+COMMIT;
+</xsl:template>
+
+</xsl:stylesheet>
+

Added: trunk/openvas-manager/tools/cve_youngerthan.xsl
===================================================================
--- trunk/openvas-manager/tools/cve_youngerthan.xsl	2011-09-01 14:51:47 UTC (rev 11545)
+++ trunk/openvas-manager/tools/cve_youngerthan.xsl	2011-09-01 14:52:03 UTC (rev 11546)
@@ -0,0 +1,38 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+OpenVAS
+$Id$
+Description: Select CVEs which have been updated after a certain date.
+
+Authors:
+Henri Doreau <henri.doreau at greenbone.net>
+
+Copyright:
+Copyright (C) 2011 Greenbone Networks GmbH
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 2,
+or, at your option, any later version as published by the Free
+Software Foundation
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+-->
+<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:vuln="http://scap.nist.gov/schema/vulnerability/0.4" xmlns:cpe-lang="http://cpe.mitre.org/language/2.0" xmlns:scap-core="http://scap.nist.gov/schema/scap-core/0.1" xmlns:cve="http://scap.nist.gov/schema/feed/vulnerability/2.0" xmlns:cvss="http://scap.nist.gov/schema/cvss-v2/0.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:patch="http://scap.nist.gov/schema/patch/0.1">
+
+<xsl:template match="cve:nvd">
+  <xsl:copy>
+    <xsl:for-each select="cve:entry[number(translate(substring(vuln:last-modified-datetime/text(),1,10),'-','')) &gt; number($refdate)]">
+      <xsl:copy-of select="."/>
+    </xsl:for-each>
+  </xsl:copy>
+</xsl:template>
+
+</xsl:stylesheet>
+

Added: trunk/openvas-manager/tools/db_init.sql
===================================================================
--- trunk/openvas-manager/tools/db_init.sql	2011-09-01 14:51:47 UTC (rev 11545)
+++ trunk/openvas-manager/tools/db_init.sql	2011-09-01 14:52:03 UTC (rev 11546)
@@ -0,0 +1,56 @@
+/* 
+ * OpenVAS
+ * Description: SCAP database initialization script
+ *
+ * Authors:
+ * Henri Doreau <henri.doreau at greenbone.net>
+ *
+ * Copyright:
+ * Copyright (C) 2011 Greenbone Networks GmbH
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2,
+ * or, at your option, any later version as published by the Free
+ * Software Foundation
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+ */
+
+/* --- TABLES CREATION --- */
+DROP TABLE IF EXISTS cves;
+DROP TABLE IF EXISTS cpes;
+DROP TABLE IF EXISTS affected_products;
+
+
+CREATE TABLE  cves (
+  id INTEGER PRIMARY KEY AUTOINCREMENT,
+  cve VARCHAR(10) UNIQUE NOT NULL,
+  last_mod DATE,
+  cvss FLOAT,
+  description TEXT
+);
+CREATE UNIQUE INDEX cve_idx ON cves (cve);
+
+
+CREATE TABLE cpes (
+  id INTEGER PRIMARY KEY AUTOINCREMENT,
+  name VARCHAR(80) UNIQUE NOT NULL
+);
+CREATE UNIQUE INDEX cpe_idx ON cpes (name);
+
+
+CREATE TABLE affected_products (
+  cve INTEGER NOT NULL,
+  cpe INTEGER NOT NULL,
+  FOREIGN KEY(cve) REFERENCES cves(id),
+  FOREIGN KEY(cpe) REFERENCES cpes(id)
+);
+CREATE INDEX afp_idx ON affected_products (cve,cpe);
+

Added: trunk/openvas-manager/tools/greenbone-scapdata-sync
===================================================================
--- trunk/openvas-manager/tools/greenbone-scapdata-sync	2011-09-01 14:51:47 UTC (rev 11545)
+++ trunk/openvas-manager/tools/greenbone-scapdata-sync	2011-09-01 14:52:03 UTC (rev 11546)
@@ -0,0 +1,269 @@
+#!/bin/bash
+#
+# greenbone-scap-sync
+# This script synchronizes an OpenVAS installation with the Greenbone SCAP
+# data directory.
+#
+# Authors:
+# Henri Doreau <henri.doreau at greenbone.net>
+#
+# Copyright:
+# Copyright (C) 2011 Greenbone Networks GmbH
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2,
+# or, at your option, any later version as published by the Free
+# Software Foundation
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+
+# If you modify this script, please prefix the version
+# with some characters in order to make it newer than
+# any follow-up version to prevent that your version
+# will be overwritten.
+VERSION=20110901
+
+# SETTINGS
+# ========
+# RSYNC_DELETE controls whether files which are not part of the repository will
+# be removed from the local directory after synchronization. The default value
+# for this setting is "--delete", which means that files which are not part of
+# feed will be deleted.
+RSYNC_DELETE="--delete"
+
+# PORT controls the outgoing TCP port for updates. If PAT/Port-Translation is
+# not used, this should be "24". For some application layer firewalls or gates
+# the value 22 (Standard SSH) is useful. Only change if you know what you are
+# doing.
+
+PORT=24
+
+# LOGDIR and LOGFILE specify the location of the greenbone-scapdata-sync logfile.
+# The default value for LOGDIR is  "/var/log/", the default value for
+# LOGFILE is "greenbone-scapdata-sync.log". Please make sure this script has
+# sufficient right to access the logfile.
+LOGDIR="/var/log/"
+LOGFILE="greenbone-scapdata-sync.log"
+
+[ -r /etc/openvas/greenbone-scapdata-sync.conf ] && . /etc/openvas/greenbone-scapdata-sync.conf
+
+BASE_SYNC_DIR="/var/lib/openvas"
+SCAP_DIR="$BASE_SYNC_DIR/scap-data"
+
+SCAP_RES="/usr/share/openvas/scap"
+
+SCAP_DB="$SCAP_DIR/scap.db"
+
+ACCESSKEY="/etc/openvas/gsf-access-key"
+
+while test $# -gt 0; do
+ case "$1" in
+        --version)
+                echo $VERSION
+                exit 0
+                ;;
+ esac
+ shift
+done
+
+if [[ ! -w $LOGDIR ]]
+then
+  NOLOG=1
+  echo
+  echo "== greenbone-scapdata-sync $VERSION ================================================"
+  echo "Warning: The logfile directory"
+  echo "  ($LOGDIR)"
+  echo "does not exist or is not writeable. Please make sure this directory exists and"
+  echo "is writable."
+  echo "Please be aware that logging is not possible during this script run!"
+  echo "==============================================================================="
+  echo
+fi
+
+log_write(){
+  if (( $NOLOG ))
+  then
+    echo "LOG: [`date -R`] $1"
+  else
+    echo "[`date -R`] $1" >> $LOGDIR$LOGFILE
+  fi
+}
+
+RSYNC=`command -v rsync`
+if [[ -z $RSYNC ]]
+then
+  echo
+  echo "== greenbone-scapdata-sync $VERSION ================================================"
+  echo "Could not find tools necessary for synchronization."
+  echo "Please make sure that the tool"
+  echo "  rsync"
+  echo "is installed and available in your PATH variable."
+  echo "If you are still not able to continue, please contact Greenbone Support at"
+  echo "support at greenbone.net and include the error messages displayed above (if any)"
+  echo "and your customer ID."
+  echo "==============================================================================="
+  echo
+  log_write "rsync not found, aborting synchronization."
+  logger "Software Update: rsync not found, aborting synchronization."
+  exit 1
+fi
+
+SQLITE=`command -v sqlite3`
+if [[ -z $SQLITE ]]
+then
+  echo
+  echo "== greenbone-scapdata-sync $VERSION ================================================"
+  echo "Could not find tools necessary for synchronization."
+  echo "Please make sure that the tool"
+  echo "  sqlite3"
+  echo "is installed and available in your PATH variable."
+  echo "If you are still not able to continue, please contact Greenbone Support at"
+  echo "support at greenbone.net and include the error messages displayed above (if any)"
+  echo "and your customer ID."
+  echo "==============================================================================="
+  echo
+  log_write "sqlite3 not found, aborting synchronization."
+  logger "Software Update: sqlite3 not found, aborting synchronization."
+  exit 1
+fi
+
+if [ -z $BASE_SYNC_DIR ]
+then
+  echo
+  echo "BASE_SYNC_DIR ($BASE_SYNC_DIR) not found!"
+  echo
+  exit 1
+fi
+
+update_scap_db() {
+  if [ ! -f $SCAP_DB ]
+  then
+    log_write "(Re-)initializing SCAP database."
+    sqlite3 $SCAP_DB < $SCAP_RES/db_init.sql
+    touch $SCAP_DIR/*.xml
+    REFDATE=0
+  else
+    REFDATE=`stat -c "%y" $SCAP_DB | cut -d " " -f 1 | tr -d "-"`
+  fi
+
+  DB_LASTMOD=`stat -c "%Y" $SCAP_DB`
+
+  CPE_BASE="$SCAP_DIR/official-cpe-dictionary_v2.2.xml"
+  if [ -e $CPE_BASE ]
+  then
+    if [ `stat -c "%Y" $CPE_BASE` -ge $DB_LASTMOD ]
+    then
+      log_write "Updating CPEs"
+      xsltproc --stringparam refdate $REFDATE $SCAP_RES/cpe_youngerthan.xsl $CPE_BASE | xsltproc $SCAP_RES/cpe_update.xsl - | sqlite3 $SCAP_DB
+    else
+      log_write "Skipping CPEs, file is older than last revision (this is not an error)."
+    fi
+  fi
+
+  for cvefile in `ls $SCAP_DIR/nvdcve-2.0-*.xml`
+  do
+    if [ `stat -c "%Y" $cvefile` -ge $DB_LASTMOD ]
+    then
+      log_write "Updating $cvefile"
+      xsltproc --stringparam refdate $REFDATE $SCAP_RES/cve_youngerthan.xsl $cvefile | xsltproc $SCAP_RES/cve_update.xsl - | sqlite3 $SCAP_DB
+    else
+      log_write "Skipping $cvefile, file is older than last revision (this is not an error)."
+    fi
+  done
+}
+
+sync_scapdata(){
+  if [[ -e $ACCESSKEY ]]
+  then
+    echo "Found Greenbone Security Feed subscription file, trying to synchronize with Greenbone SCAP data Repository ..."
+    notsynced=1
+    retried=0
+
+    mkdir -p "$BASE_SYNC_DIR"
+    read feeduser < $ACCESSKEY
+    read -d "@" custid < $ACCESSKEY
+    if [[ -z $feeduser || -z $custid ]]
+    then
+      echo "== greenbone-scapdata-sync $VERSION ================================================"
+      echo "Synchronization was not possible because credential information could not"
+      echo "be read from your access key."
+      echo "Please make sure that the key located at"
+      echo "  $sysconfdir/openvas/gsf-access-key"
+      echo "is intact and in a valid format."
+      echo "If you are still not able to continue, please contact Greenbone Support at"
+      echo "support at greenbone.net and include the error messages displayed above (if any)"
+      echo "and your customer ID."
+      echo "==============================================================================="
+      log_write "Could not determine credentials, aborting synchronization."
+      logger "Software Update: Could not determine credentials, aborting synchronization."
+      exit 1
+    fi
+    while (($notsynced))
+    do
+      # --protocol=29 is a workaround for a known bug in rsync 3.0.3
+      if [[ -e /admin/ezcli.state ]]
+      then
+        gsmproxy=`grep proxy_feed /admin/ezcli.state | sed -e 's/^.*\/\///' -e 's/:/ /' -e 's/[\t ]*$//'`
+        PORT=`grep ^syncport /admin/ezcli.state | sed -e "s/^syncport\t//g"`
+      fi
+      if [[ $gsmproxy == "proxy_feed" || -z $gsmproxy ]]
+      then
+        rsync -e "ssh -p $PORT -i $ACCESSKEY" -ltvrP --protocol=29 $RSYNC_DELETE $custid at feed.greenbone.net:/scap-data $BASE_SYNC_DIR
+      else
+        rsync -e "ssh -o \"ProxyCommand corkscrew $gsmproxy %h %p\" -p $PORT -i $ACCESSKEY" -ltvrP --protocol=29 $RSYNC_DELETE $custid at feed.greenbone.net:/scap-data $BASE_SYNC_DIR
+      fi
+      if (( $? )) ; then
+        echo
+        echo "== greenbone-scapdata-sync $VERSION ================================================"
+        echo "The synchronization with the repository failed. This may indicate a serious issue"
+        echo "with either your infrastructure or the repository itself."
+        echo "Your local SCAP data repository may be damaged now. Please resolve any connection"
+        echo "issues and try again."
+        echo "If you suspect an issue with the Greenbone SCAP data Repository, please contact"
+        echo "Greenbone support at support at greenbone.net and include the error messages"
+        echo "displayed above (if any) and your customer ID ($custid)."
+        echo "==============================================================================="
+        echo
+        log_write "rsync failed, aborting synchronization."
+	logger "Software Update:  failed, aborting synchronization."
+        exit 1
+      fi
+      notsynced=0
+    done
+    echo "Synchronization with the Greenbone SCAP data Repository successful."
+    log_write "Synchronization with the Greenbone SCAP data Repository successful."
+    logger "Software Update: Synchronization with the Greenbone SCAP data Repository successful."
+    echo
+
+    update_scap_db
+  else
+    echo
+    echo "== greenbone-scapdata-sync $VERSION ================================================"
+    echo "Could not find gsf-access-key."
+    echo
+    echo "This access key can be obtained from Greenbone Networks GmbH,"
+    echo "see http://greenbone.net/solutions/gbn_feed.html for details."
+    echo
+    echo "Please make sure the personal access key you obtained from"
+    echo "Greenbone is placed in the following directory:"
+    echo $ACCESSKEY
+    echo "Please make also sure that the filename is gsf-access-key."
+    echo "If you are still not able to synchronize, please contact Greenbone Support at"
+    echo "support at greenbone.net and include the error messages displayed above (if any)"
+    echo "and your customer ID."
+    echo "==============================================================================="
+    echo
+    log_write "gsf-access-key not found, aborting synchronization."
+    logger "Software Update: gsf-access-key not found, aborting synchronization."
+    exit 1
+  fi
+}
+
+sync_scapdata


Property changes on: trunk/openvas-manager/tools/greenbone-scapdata-sync
___________________________________________________________________
Name: svn:executable
   + *

Added: trunk/openvas-manager/tools/openvas-scapdata-sync
===================================================================
--- trunk/openvas-manager/tools/openvas-scapdata-sync	2011-09-01 14:51:47 UTC (rev 11545)
+++ trunk/openvas-manager/tools/openvas-scapdata-sync	2011-09-01 14:52:03 UTC (rev 11546)
@@ -0,0 +1,292 @@
+#!/bin/sh
+#
+# OpenVAS
+# $Id$
+# Description: Synchronize with SCAP data feed.
+#
+# Authors:
+# Henri Doreau <henri.doreau at greenbone.net>
+#
+# Copyright:
+# Copyright (C) 2011 Greenbone Networks GmbH
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2,
+# or, at your option, any later version as published by the Free
+# Software Foundation
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+
+# configure SCAP_DIR where we will sync SCAP data
+if [ -z "$SCAP_DIR" ]; then
+  OPENVASSD=`which openvassd`
+  if [ -z "$OPENVASSD" ] ; then
+    echo "[e] Error: openvassd is not in the path, could not determine SCAP directory."
+    exit 1
+  else
+    # get the parent directory of the plugins
+    SCAP_DIR=`openvassd -s | awk -F" = " '/^plugins_folder/ { print $2 }' | sed -s 's/\(^.*\)plugins/\1/'`
+    # suffix it with "scap-data" which is our target (destination) directory
+    SCAP_DIR="$SCAP_DIR/scap-data"
+  fi
+fi
+
+# Script and feed information which will be made available to user through
+# command line options and automated tools.
+SCRIPT_NAME="openvas-scapdata-sync"
+VERSION=@OPENVASSD_VERSION@
+RESTRICTED=0
+
+
+# The URL of the plugin feed
+if [ -z "$OV_RSYNC_FEED" ]; then
+  OV_RSYNC_FEED=rsync://feed.openvas.org:/scap-data
+  # An alternative syntax which might work if the above doesn't:
+  # OV_RSYNC_FEED=rsync at feed.openvas.org::scap-data
+fi
+
+if [ -z "$OV_HTTP_FEED" ]; then
+  OV_HTTP_FEED=http://www.openvas.org/openvas-scap-data-current.tar.bz2
+fi
+
+if [ -z "$TMPDIR" ]; then
+  SYNC_TMP_DIR=/tmp
+  # If we have mktemp, create a temporary dir (safer)
+  if [ -n "`which mktemp`" ]; then
+    SYNC_TMP_DIR=`mktemp -t -d openvas-scap-data-sync.XXXXXXXXXX` || { echo "ERROR: Cannot create temporary directory for file download" >&2; exit 1 ; }
+    trap "rm -rf $SYNC_TMP_DIR" EXIT HUP INT TRAP TERM
+  fi
+else
+  SYNC_TMP_DIR="$TMPDIR"
+fi
+
+do_help () {
+  echo "$0: Sync SCAP data using different protocols"
+  echo " --rsync		sync with rsync (default)"
+  echo " --wget		sync with wget"
+  echo " --curl		sync with curl"
+  echo " --check		just checksum check"
+  echo "OpenVAS administrator functions:"
+  echo " --selftest	perform self-test"
+  echo " --identify	display information"
+  echo " --version	display version"
+  echo " --dst-dir <dir>	SCAP destination directory"
+  echo ""
+  echo "Environment variables:"
+  echo "SCAP_DIR		where to extract plugins (absolute path)"
+  echo "OV_RSYNC_FEED	URL of rsync feed"
+  echo "OV_HTTP_FEED	URL of http feed"
+  echo "TMPDIR		temporary directory used to download the files"
+  echo "Note that you can use standard ones as well (e.g. http_proxy) for wget/curl"
+  echo ""
+  exit 0
+}
+
+CMD_RSYNC=`which rsync`
+CMD_WGET=`which wget`
+CMD_CURL=`which curl`
+CMD_SQLITE=`which sqlite3`
+TMP_SCAP="$SYNC_TMP_DIR/openvas-feed-`date +%F`-$$.tar.bz2"
+
+chk_system_tools () {
+  echo "[i] Searching for required system tools (look for warnings/errors)..."
+
+  if [ -z "$CMD_RSYNC" ]; then
+    echo "[w] Warning: RSYNC not found";
+  fi
+
+  if [ -z "$CMD_WGET" ]; then
+    echo "[w] Warning: wget not found";
+  fi
+
+  if [ -z "$CMD_CURL" ]; then
+    echo "[w] Warning: curl not found";
+  fi
+
+  if [ -z "$CMD_SQLITE" ]; then
+    echo "[e] Error: sqlite3 not found (required)";
+    exit 1
+  fi
+
+  if [ -z "$CMD_RSYNC" -a -z "$CMD_WGET" -a -z "$CMD_CURL" ]; then
+    SELFTEST_FAIL=1
+  fi
+
+  echo "[i] If you did not get any warnings, that means you have all tools required"
+}
+
+do_rsync () {
+  if [ -z "$CMD_RSYNC" ]; then
+    echo "[w] rsync not found!"
+  else
+    echo "[i] Using rsync: $CMD_RSYNC"
+    echo "[i] Configured SCAP data rsync feed: $OV_RSYNC_FEED"
+    mkdir -p "$SCAP_DIR"
+    eval "$CMD_RSYNC -ltvrP \"$OV_RSYNC_FEED\" \"$SCAP_DIR\""
+    if [ $? -ne 0 ] ; then
+      echo "Error: rsync failed. Your SCAP data might be broken now."
+      exit 1
+    fi
+  fi
+}
+
+do_wget () {
+  if [ -z "$CMD_WGET" ]; then
+    echo "[w] GNU wget not found!"
+  else
+    echo "[i] Using GNU wget: $CMD_WGET"
+    echo "[i] Configured SCAP http feed: $OV_HTTP_FEED"
+    echo "[i] Downloading to: $TMP_SCAP"
+    mkdir -p "$SCAP_DIR" \
+    && wget "$OV_HTTP_FEED" -O $TMP_SCAP \
+    && cd "$SCAP_DIR" \
+    && tar xvjf $TMP_SCAP \
+    && rm -f $TMP_SCAP \
+    && echo "[i] Download complete"
+  fi
+}
+
+do_curl () {
+  if [ -z "$CMD_CURL" ]; then
+    echo "[w] curl not found!"
+  else
+    echo "[i] Using curl: $CMD_CURL"
+    echo "[i] Configured SCAP http feed: $OV_HTTP_FEED"
+    echo "[i] Downloading to: $TMP_SCAP"
+    mkdir -p "$SCAP_DIR" \
+    && curl "$OV_HTTP_FEED" -o $TMP_SCAP \
+    && cd "$SCAP_DIR" \
+    && tar xvjf $TMP_SCAP \
+    && rm -f $TMP_SCAP \
+    && echo "[i] Download complete"
+  fi
+}
+
+do_self_test () {
+  chk_system_tools
+}
+
+show_intro () {
+  echo "[i] This script synchronizes a SCAP data directory with the OpenVAS one."
+  echo "[i] SCAP dir: $SCAP_DIR"
+}
+
+do_sync () {
+  if [ -z "$CMD_RSYNC" ] ; then
+    if [ -z "$CMD_WGET" ]; then
+      echo "[w] GNU wget not found!"
+      if [ -z "$CMD_CURL" ]; then
+        echo "[w] curl not found!"
+        echo -n "[e] no utility available in PATH environment variable to download plugins"
+        exit 1
+      else
+        echo "[i] Will use curl"
+        do_curl
+      fi
+    else
+      echo "[i] Will use wget"
+      do_wget
+    fi
+  else
+    echo "[i] Will use rsync"
+    do_rsync
+  fi
+}
+
+update_sec_db () {
+
+  SEC_DB="$SCAP_DIR/scap.db"
+  CPEBASE="$SCAP_DIR/official-cpe-dictionary_v2.2.xml"
+
+  if [ ! -f $SEC_DB ]
+  then
+    echo "[i] (Re-)initializing database"
+    sqlite3 $SEC_DB < db_init.sql
+    touch $SCAP_DIR/*.xml
+    REFDATE=0
+  else
+    REFDATE=`stat -c "%y" $SEC_DB | cut -d " " -f 1 | tr -d "-"`
+  fi
+
+  DB_LASTMOD=`stat -c "%Y" $SEC_DB`
+
+  if [ `stat -c "%Y" $CPEBASE` -ge $DB_LASTMOD ]
+  then
+    echo "[i] Updating CPEs"
+    xsltproc --stringparam refdate $REFDATE cpe_youngerthan.xsl $CPEBASE | xsltproc cpe_update.xsl - | sqlite3 $SEC_DB
+  else
+    echo "[i] Skipping CPEs, file is older than last revision"
+  fi
+
+  for cvefile in `ls $SCAP_DIR/nvdcve-2.0-*.xml`
+  do
+    if [ `stat -c "%Y" $cvefile` -ge $DB_LASTMOD ]
+    then
+      echo "[i] Updating $cvefile"
+      xsltproc --stringparam refdate $REFDATE cve_youngerthan.xsl $cvefile | xsltproc cve_update.xsl - | sqlite3 $SEC_DB
+    else
+      echo "[i] Skipping $cvefile, file is older than last revision"
+    fi
+  done
+}
+
+if [ -n "$1" ]; then
+  while test $# -gt 0; do
+    case "$1" in
+      --help)
+        do_help
+        exit 0
+        ;;
+      --rsync)
+        do_rsync
+        exit 0
+        ;;
+      --wget)
+        do_wget
+        exit 0
+        ;;
+      --curl)
+        do_curl
+        exit 0
+        ;;
+      --check)
+        exit 0
+        ;;
+      --version)
+        echo $VERSION
+        exit 0
+        ;;
+      --identify)
+        echo "SCAPSYNC|$SCRIPT_NAME|$VERSION|$FEED_NAME|$RESTRICTED|SCAPSYNC"
+        exit 0
+        ;;
+      --selftest)
+        SELFTEST_FAIL=0
+        do_self_test
+        exit $SELFTEST_FAIL
+        ;;
+      --feedversion)
+        do_feedversion
+        exit 0
+        ;;
+      --nvt-dir)
+        SCAP_DIR="$2"
+        shift
+        ;;
+    esac
+    shift
+  done
+fi
+
+show_intro
+do_sync
+update_sec_db
+
+exit 0


Property changes on: trunk/openvas-manager/tools/openvas-scapdata-sync
___________________________________________________________________
Name: svn:executable
   + *



More information about the Openvas-commits mailing list