[Openvas-devel] Applying redis integration approach to lowering OpenVAS scanner memory footprint
lists at securityspace.com
Tue Jul 8 01:26:41 CEST 2014
Couple questions (sorry for one being simple as a result of being
offlist for quite some time):
1) Which version of OpenVAS is redis integration required in?
2) Has anyone done a memory analysis of what the memory profile looks
like of the scanners before and after redis integration?
3) Given previous work we had done in the area, IIRC (it's been many
years), the memory footprint of OpenVAS ended up being huge as a
result of legacy design decisions that resulted in each forked
scanner launched as a result of a client connect duplicating all
active memory. Any thoughts to taking the heavy memory load
of script information and distributing this information into
a redis like DB?
If anyone has any insight here, would be much appreciated. I'll
provide some quick context and background here:
1) Scanner memory has, in the past, been a resource bottleneck.
2) Investigation showed that due to how forked clients updated
previous memory blocks, all linux memory blocks, normally
shared until "copy on write", were effectively copied,
resulting in no shared memory savings.
3) The bulk of memory usage was in the description fields of
all the tests.
4) The worst case memory usage scenario (unfortunately, something
we suffer from), was a case where each client connect is allowed
to scan only ONE host. A platform capable of running, say, of
supporting the ability to scan 50 IPs concurrently, with each
IP supporting 2 NASL scripts concurrently, would have a huge
50x memory load of the original openvas scanner process.
(Actual script execution forks were minimal in their memory
usage, as they leveraged the shared memory structure of linux)
I could see a huge, immediate impact/savings if we were able to take
a class of information deemed non-sensitive (static descriptions would
fit that bill), and move these off to a common key/value server....
The savings on the memory profile I'm betting would be huge here...
On 24/04/14 04:35 PM, Henri Doreau wrote:
> 2014-04-24 14:09 GMT+02:00 Jan-Oliver Wagner <Jan-Oliver.Wagner at greenbone.net>:
>> On Donnerstag, 24. April 2014, Henri Doreau wrote:
>>> 2014-04-24 8:26 GMT+02:00 Jan-Oliver Wagner <Jan-Oliver.Wagner at greenbone.net>:
>>>> I noted that you seem to have missed committing the file doc/redis_config.txt
>>>> which was present in your earlier patches...
>>> Woops, sorry, fixed.
>>>> What I think could help people a lot is to have a doc/example_redis_2_4.conf
>>>> and a doc/example_redis_2_6.conf which ware installed as
>>>> PREFIX/etc/openvas/redis_2_4.conf (or 2_6 depending on what was detected).
>>>> The INSTALL could then provide a simple example on how to run redis with
>>>> the prepared redis conf.
>>> Good idea, I'll prepare something like this.
>> wonderful. Perhaps a location like usr/share/doc/openvas is better than /etc
> Please find sample configuration files attached. I hope I didn't
> forget anything. I'm not sure how to tell cmake to detect (at install
> time) which version of redis is installed... What about simply
> installing both files? If someone well-versed into cmake knows it,
> feel free to commit the files.
> Openvas-devel mailing list
> Openvas-devel at wald.intevation.org
More information about the Openvas-devel