The following are resources I have created to help improve Security Operations Centers (SOC) environments. Please let me know if you have any suggestions on what else I can add @ForgottenSec
Panel Discussion of Problems in the SOC
BSidesAugusta Presentation comparing IDS options
Detection Engineering
Snort Analysis Guide
Snort rules need to be as precise as possible to avoid false positives yet general enough to not need an obscene number of rules. A significant amount of filtering is done by a special first analysis called “Fast Pattern Matcher” to reduce the number of rules each packet is tested for.
Rule Header alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS
| 1 | 2 | 3 | 4 | 5 | 6 | 7 |
Seven Parts make up the Rule Header:
1. Action – When the rule is triggered, what should snort do. Snort can only alert if it is not inline active (see explanation of the three deployment configurations)
* alert – Log
* drop – Drop & Log
* replace – Overwrite with data of the same size
* sdrop – Drop without logging
* reject – Drop, Log, and reply rejected (TCP reset or ICMP unavailable)
2. Protocol – What Protocol should the alert analyze (tcp, udp, icmp, ip or any)
3. Source IP – IP Address of the sending computer. This can contain “any” or a variable (starts with a $)
4. Source Port – Port of the sending computer. This can contain “any” or a variable (starts with a $)
5. Direction – Rules can be unidirectional or bidirectional (-> or <- or <>)
6. Destination IP – IP Address of the receiving computer. This can contain “any” or a variable (starts with a $)
7. Destination Port – Port of the receiving computer. This can contain “any” or a variable (starts with a $)
Rule Body(msg:"SERVER-WEBAPP Mutiny editdocument servlet arbitrary file upload attempt"; flow:to_server,established; content:"/interface/editdocument"; fast_pattern:only; http_uri; content:"uploadFile"; nocase; http_client_body; content:"uploadPath"; nocase; http_client_body; pcre:"/uploadPath[^-]+?(%2e|\x2e){2}(%2f|\x2f)/miP"; metadata:policy security-ips drop, service http; reference:cve,2013-0136; reference:url,osvdb.org/show/osvdb/93444; classtype:attempted-admin; sid:26798; rev:2;)
1. msg – the displayed name of the alert (starts with the uppercase rule file name followed by a description of the reason for alerting)
2. flow – Established refers to requiring a tcp connection been setup with a handshake while the “to_server” and “to_client” refer to the direction(flow:to_server, established or flow:to_client, established or flow:established)
3. content – Character string match (can define actual bites by enclosing bytes in “|”), regularly multiple occurrences within each rule
4. fast_pattern – modifier to specify match should only be done on the initial pass
5. http_uri – modifier to specify testing this after HTTP Uniform Resource Identifier (URI) has been normalized by the HTTP preprocessor to previous content match
6. nocase – modifier to ignore upper or lower case in content to previous content match
7. http_client_body – Only looks in the HTTP request body for previous content match, can’t be used with rawbytes modifier
8. pcre – Perl Compatible Regular Expression, the processor expensive dynamic matching language
9. metadata – tells metadata about the rule rather then being a true part of the rule
10. policy security-ips drop – three pre-made policies exist within the SourceFire ruleset (Security over Connectivity, Connectivity over Security, and balanced). This metadata says this rule will only be turned on automatically if your using the “Security over Connectivity” policy or manually turned on.
11. service: http – defines that this rule should be applied to any http service
12. reference:cve,2013-0136 – Details of the exploit this rule is based on can be found in the National Vulnerability Database (NVD) Common Vulnerabilities and Exposures (CVE) database under “2013-0136” – https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-0136
13. reference:url,osvdb.org/show/osvdb/93444 – Details of the exploit this rule is based on can be found at the url https://osvdb.org/show/osvdb/93444
14. classtype:attempted-admin – Details the type of exploit the rule is based on
15. sid:26798 – Identifies the Signature ID, Typically prepended along with the gid at the beginning of the alert
16. rev:2 – this rule has been revised 1 since the beginning, similar to a version number
gid – is not defined in the rule, but the generator id (gid) refers to the type of rule. All plain text rules, like the one above, come from gid 1. gid 3 rules are from shared object (compiled) rules.gid’s between 105 and 145 are decoder and preprocessor rule based alerts.
[**] [1:26798:2] SERVER-WEBAPP Mutiny editdocument servlet arbitrary file upload attempt [**]
[Classification: Attempted Administrator Privilege Gain] [Priority: 1]
05/29-19:44:02.238185 249.94.153.251 -> 249.94.153.77
An alert starts off with the gid,sid,rev inside brackets then the display name of the alert
1. “[1:26798:2]” – GID (Generator ID) 1, SID (Signature ID) 26798, REV (Revision Number) 2. See below for explanation
2. “SERVER-WEBAPP” – filename of the rules file where the alert should be stored unless they have been combined into a single rule file
3. “Mutiny editdocument servlet arbitrary file upload attempt” – exact type of traffic discribed in the alert
4. “[Classification: Attempted Administrator Privilege Gain]” – type of attack goal for this particular exploit
5. “[Priority: 1]” – if snort gets multiple alerts for the same packet, it uses priority to determine which it will display (usually will trigger on the 3 alerts with the lowest priority and will stop inspecting traffic that triggers 8 alerts – as configured by the line starting with “config event_queue:” in snort.conf)
6. “05/29-19:44:02.238185” – Date & Time the alert was generated in system time (keeping track of the timezone of your sensors is important for correlating data)
7. “249.94.153.251” – Source IP
8. “249.94.153.77” – Destination IP
Signature ID (SID) is the rule number.
Revision Number (REV) is the number of times the rule has been changed starting with 1 being rule creation.
Engine Rules
- 1 – Standard Rule Engine – files can be found in the “.rules” file(s)
- 3 – Shared Object Rules – Compiled rules for complex threats and/or threats with details TALOS can not publicly share
Preprocessor Rules
- 105 – Back Orifice Preprocessor
- 106 – RPC Decode Preprocessor
- 112
- Will be adding the rest of the list: http://manual-snort-org.s3-website-us-east-1.amazonaws.com/node18.html
To find more info about a rule, Search for the rule SID and title in a search engine!
Return to top of page
Detection Engineering
One of the biggest concerns in the SOC is that we see far more alerts then we can look at. One of the more interesting specialties I’ve seen being more visible lately is focused on what I call Detection Engineering. This encompasses two primary things:
- Investigating alerts that make the most noise and how they can be resolved without losing visibility
- Researching new detection capabilities within existing tools by writing correlation and/or IDS/IPS rules
The above two goals typically seem to fall somewhere between senior analysts, engineers and/or just fall through the cracks. Many organizations turn off noisy rules thereby losing visibility. This is a significantly faster solution, but creates additional holes in inspection capability. Often fixing the alert will require a correction to networking and/or application configuration which can take resources. Additionally, known good exceptions to alerts can be resolved with pass rules specifying the differences between the known good traffic and the traffic the rule is meant to look for.