This patch speeds up startup and adds the --unique-map option #181
Conversation
Barnyard2 takes a long time to start due to poorly optimized list code. For every SID added to the list, the entire list is iterated to get to the tail of the list. Adding and maintaining a tail pointer prevents this, since we can just add entries directly at the end. The --unique-map option is also added, this prevents each SID from the files being checked against what's in memory. Most tools generate unique files, making this check unnecessary and wasteful.
|
The code should not assume certain tools usage if input is corrupted output On Mon, Feb 8, 2016 at 9:01 AM, Rob Mosher notifications@github.com wrote:
|
|
Well, it's added as an option if you have known good input. The linked list optimization is unrelated, you can merge whichever parts you want. |
|
I agree that using tail for that can speed up startup but its not the in On Tue, Mar 22, 2016 at 8:57 PM, Rob Mosher notifications@github.com
|
|
It still shaved a few minutes of time off during startup using a long list. |
|
#define long_list On Tue, Mar 22, 2016 at 9:13 PM, Rob Mosher notifications@github.com
|
|
10-20k |
|
And on what type of hardware? Minutes you see are from here: On Tue, Mar 22, 2016 at 9:19 PM, Rob Mosher notifications@github.com
|
|
hardware is AMD GX-412TC SOC, quad core 1ghz With the --unique-map option, BcUniqueMap will return 1, and prevent cacheSignatureLookup from running on the line you referenced. It still takes quite some time to load the rules, even with that in place. |
|
Sorry i replied quickly with something that should be read as follow: Initialization of 10-20k if it takes minutes to read the sid-msg.map is Where is agree is tail adding should be good enough, on the other On Tue, Mar 22, 2016 at 9:22 PM, beenph beenph@gmail.com wrote:
|
|
If you use a database, do a select count(*) from signature; On Tue, Mar 22, 2016 at 9:35 PM, beenph beenph@gmail.com wrote:
|
|
50k in there at the moment. it's not really bypassing integrity, worst case is the same rule can be added multiple times. If other tools are known to produce unique output, it helps somewhat. |
Barnyard2 takes a long time to start due to poorly optimized list code. For every SID added to the list, the entire list is iterated to get to the tail of the list. Adding and maintaining a tail pointer prevents this,
since we can just add entries directly at the end.
The --unique-map option is also added, this prevents each SID from the files being checked against what's in memory. Most tools generate unique files, making this check unnecessary and wasteful.