[Openvas-devel] Applying redis integration approach to lowering OpenVAS scanner memory footprint

Thomas Reinke lists at securityspace.com
Tue Jul 8 01:26:41 CEST 2014


Hi all,

Couple questions (sorry for one being simple as a result of being
offlist for quite some time):

1) Which version of OpenVAS is redis integration required in?
2) Has anyone done a memory analysis of what the memory profile looks
    like of the scanners before and after redis integration?
3) Given previous work we had done in the area, IIRC (it's been many
    years), the memory footprint of OpenVAS ended up being huge as a
    result of legacy design decisions that resulted in each forked
    scanner launched as a result of a client connect duplicating all
    active memory.   Any thoughts to taking the heavy memory load
    of script information and distributing this information into
    a redis like DB?

If anyone has any insight here, would be much appreciated.  I'll
provide some quick context and background here:

1) Scanner memory has, in the past, been a resource bottleneck.
2) Investigation showed that due to how forked clients updated
    previous memory blocks, all linux memory blocks, normally
    shared until "copy on write", were effectively copied,
    resulting in no shared memory savings.
3) The bulk of memory usage was in the description fields of
    all the tests.
4) The worst case memory usage scenario (unfortunately, something
    we suffer from), was a case where each client connect is allowed
    to scan only ONE host. A platform capable of running, say, of
    supporting the ability to scan 50 IPs concurrently, with each
    IP supporting 2 NASL scripts concurrently, would have a huge
    50x memory load of the original openvas scanner process.
    (Actual script execution forks were minimal in their memory
    usage, as they leveraged   the shared memory structure of linux)

I could see a huge, immediate impact/savings if we were able to take
a class of information deemed non-sensitive (static descriptions would
fit that bill), and move these off to a common key/value server....
The savings on the memory profile I'm betting would be huge here...

Thomas


On 24/04/14 04:35 PM, Henri Doreau wrote:
> 2014-04-24 14:09 GMT+02:00 Jan-Oliver Wagner <Jan-Oliver.Wagner at greenbone.net>:
>> On Donnerstag, 24. April 2014, Henri Doreau wrote:
>>> 2014-04-24 8:26 GMT+02:00 Jan-Oliver Wagner <Jan-Oliver.Wagner at greenbone.net>:
>>>> I noted that you seem to have missed committing the file doc/redis_config.txt
>>>> which was present in your earlier patches...
>>>>
>>> Woops, sorry, fixed.
>>
>> thanks.
>>
>>
>>>> What I think could help people a lot is to have a doc/example_redis_2_4.conf
>>>> and a doc/example_redis_2_6.conf which ware installed as
>>>> PREFIX/etc/openvas/redis_2_4.conf (or 2_6 depending on what was detected).
>>>>
>>>> The INSTALL could then provide a simple example on how to run redis with
>>>> the prepared redis conf.
>>>>
>>> Good idea, I'll prepare something like this.
>>
>> wonderful. Perhaps a location like usr/share/doc/openvas is better than /etc
>>
>
> Please find sample configuration files attached. I hope I didn't
> forget anything. I'm not sure how to tell cmake to detect (at install
> time) which version of redis is installed... What about simply
> installing both files? If someone well-versed into cmake knows it,
> feel free to commit the files.
>
> Regards
>
>
>
> _______________________________________________
> Openvas-devel mailing list
> Openvas-devel at wald.intevation.org
> https://lists.wald.intevation.org/cgi-bin/mailman/listinfo/openvas-devel
>




More information about the Openvas-devel mailing list