Stop climbing through the haystack to find the needle: Use a magnet

Categories: Uncategorized

by Barrett Mononen – Sr. Product Marketing Manager, RSA

A few weekends back I had the pleasure of going to the local children’s museum with a young nephew of mine. One of the attractions was a magnet from an old air craft carrier’s radar system –it was huge and really powerful. The sign explained what it was and joked, “Finding a needle in the haystack isn’t hard with this.”

This got me thinking. As security professionals we are constantly thinking about finding the needle (security incident) in the data haystack. But what if we just used a really powerful magnet? The needle in this case is the tiniest piece of evidence that an adversary is traversing your network or attempting to inflict digital damage and the haystack represents the mountain of innocuous data that the needle is hidden within. And in the era of big data that haystack isn’t getting any smaller.

The data a typical security analyst has to look at is growing by the second: Logs, packets, critical IT assets, threat intelligence, event data, and data classification feeds, to name some key ones. And on top of that, attackers are getting better at disguising their needles. Potential threats are more targeted, stealthy and dynamic than they have ever been. Which means you won’t find the needle if you aren’t collecting the hay in which the needle may be hiding. So, it’s more than just collecting a lot of data, it’s about collecting the right data. This means log collection AND full packet capture, it means external threat intelligence applied to this data to help identify previously unknown attack sequences and it means enabling analysis on all of this data to help detect threats without signatures.

“It’s more than just collecting a lot of data, it’s about collecting the right data.”

The haystack is growing, the needles are getting smaller, yet more damaging and we’re collecting lots of (the right) data. Now what?

Must be time for the really big magnet, right? Well, not exactly. A lot of organizations have started down that path, but it’s more than just buying the right magnet. It’s about pointing that magnet at the right sized haystack. To put it more realistically, how about we use tactics to remove a lot of the hay and make our existing “magnets” more powerful? This could make the haystacks more manageable.

Tactics can be applied like removing items within your data set that you know are “good” – or not threatening – to reveal items that have a higher probability of being ”bad”. This method, sometimes called data or traffic carving, can be an incredibly valuable tool. Start a new investigation where you aren’t looking for anything in particular – just looking to remove things you know are good, normal or OK activity. I’ll bet you’ll be surprised at what is left behind – at the very least some activity that is hard to explain.

Now I’m sure we all wish we had an aircraft carrier-sized magnet to find the needle in a haystack, but using the right tactics in combination with stronger tools can actually improve your results.

Barrett is a member of the product marketing team focused on the evolution of RSA’s SIEM and security analysis portfolio and is always looking to bring fresh “insights” to the security management landscape.  Outside of work you can find Barrett at the top of the closest mountain or running his legs off in the nearest road race.


The RSA Security Management team is made up of RSA’s managers of the Archer, enVision, DLP, and NetWitness product lines. We interact with customers on a daily basis and will use this blog to share insights about our customers’ security strategies and challenges. We hope the blogs will stimulate your thoughts and that you will want to share your insights with us by commenting on the blogs!