Logs And Backups

Logs and BackupsLogs and Backups
Charles Southerland (a.k.a. proidiot)Charles Southerland (a.k.a. proidiot)
Stuph LabsStuph Labs
Information Warfare Summit 2013Information Warfare Summit 2013

Imagine an outsider trying to deface yourImagine an outsider trying to deface your
organization's website.organization's website.
We'll say they're using SQL injection to do this.We'll say they're using SQL injection to do this.

The logs will likely give you a trove of informationThe logs will likely give you a trove of information
about how the attack occurred, and the backupabout how the attack occurred, and the backup
will allow you to revert the changes quickly.will allow you to revert the changes quickly.

Now imagine getting attacked by an insider.Now imagine getting attacked by an insider.

Those backups suddenly become a liability.Those backups suddenly become a liability.

And those logs, which can usually be modifiedAnd those logs, which can usually be modified
with ease by an insider, will probably not help youwith ease by an insider, will probably not help you
figure out who attacked, how they attacked, orfigure out who attacked, how they attacked, or
perhaps even that they attacked at all.perhaps even that they attacked at all.

In fact, the logs might be almost as bad to leak asIn fact, the logs might be almost as bad to leak as
your backups.your backups.

Unfortunately, there doesn't appear to be a goodUnfortunately, there doesn't appear to be a good
one-size-fits-all way to deal with backups and logsone-size-fits-all way to deal with backups and logs
with respect to insider threats at this time.with respect to insider threats at this time.

I have no doubt that there is some vendor outI have no doubt that there is some vendor out
there that will sell you a “security in a box”there that will sell you a “security in a box”
solution to this problem, but I seriously doubt suchsolution to this problem, but I seriously doubt such
a solution would be a good choice for manya solution would be a good choice for many
organizations.organizations.

...but I have some ideas that might work for some...but I have some ideas that might work for some
organizations.organizations.

First: rethink backupsFirst: rethink backups

Why you might need to recover from backups:Why you might need to recover from backups:
●
Something went wrongSomething went wrong
and you can recover quicklyand you can recover quickly
●
Something catastrophic happenedSomething catastrophic happened
and you must recover carefullyand you must recover carefully

The best solution to non-breach recoveryThe best solution to non-breach recovery
is failover.is failover.
After all, the time it takes to restore from aAfter all, the time it takes to restore from a
backup is still downtime.backup is still downtime.

For the actual backups, separately backupFor the actual backups, separately backup
sensitive user data, other business data, etc.sensitive user data, other business data, etc.

Use a configuration management system (e.g.Use a configuration management system (e.g.
Chef, Puppet, CFEngine) and back up those filesChef, Puppet, CFEngine) and back up those files
in a form that necessary personnel can quicklyin a form that necessary personnel can quickly
decrypt and use as needed.decrypt and use as needed.

Encrypt all backups using a cryptosystem that isEncrypt all backups using a cryptosystem that is
appropriate for the sensitivity of the particularappropriate for the sensitivity of the particular
data, and be sure to always verify the authenticitydata, and be sure to always verify the authenticity
of the data (e.g. md5sum).of the data (e.g. md5sum).

Limit access to the onsite backups to a handful ofLimit access to the onsite backups to a handful of
people, and choose different people to grantpeople, and choose different people to grant
access to the crypto keys for those onsiteaccess to the crypto keys for those onsite
backups.backups.

Very closely monitor and log all access to theVery closely monitor and log all access to the
onsite backups. These onsite backups shouldonsite backups. These onsite backups should
preferably be kept somewhere that would be verypreferably be kept somewhere that would be very
difficult to extract information unnoticed from (i.e.difficult to extract information unnoticed from (i.e.
a computer with an air gap to the network).a computer with an air gap to the network).

Keep lots of backups in an offsite facility yourKeep lots of backups in an offsite facility your
employees don't have access to (e.g. Amazonemployees don't have access to (e.g. Amazon
Web Services, Rackspace).Web Services, Rackspace).

Amazon's Glacier would probably be a goodAmazon's Glacier would probably be a good
choice.choice.

Again, profusely log all access to the offsiteAgain, profusely log all access to the offsite
backups.backups.

Treat access to offsite backups like you do theTreat access to offsite backups like you do the
onsite one: encrypt all data, assure differentonsite one: encrypt all data, assure different
people have access to the data vs. the keys, etc.people have access to the data vs. the keys, etc.

Every 6 months and every time someone leavesEvery 6 months and every time someone leaves
who had access to the key or data for the onsitewho had access to the key or data for the onsite
backups, immediately destroy the key and data,backups, immediately destroy the key and data,
create new keys for the new backups, and thencreate new keys for the new backups, and then
randomly assign who will have access to whichrandomly assign who will have access to which
keys and data.keys and data.

It would be best to have similar practices withIt would be best to have similar practices with
regard to the keys and data for the offsiteregard to the keys and data for the offsite
backups, but care must be taken not to handlebackups, but care must be taken not to handle
these actions in an insecure way.these actions in an insecure way.

Next: rethink logsNext: rethink logs

It is vital to assure that none of the sensitive dataIt is vital to assure that none of the sensitive data
leaks into the logs.leaks into the logs.

However, all other data, no matter how menial,However, all other data, no matter how menial,
should be recorded into the logs.should be recorded into the logs.

Hard drive space is very cheap and big data canHard drive space is very cheap and big data can
be extremely useful...be extremely useful...

...so open the floodgates (e.g. this user requested...so open the floodgates (e.g. this user requested
this page by following this link from this ip addressthis page by following this link from this ip address
at this time from a browser with this agent string)at this time from a browser with this agent string)

You can use Apache Hadoop to analyze this dataYou can use Apache Hadoop to analyze this data
and do cool things like...and do cool things like...

...or determine user preferences...or determine user preferences

...or better protect your network...or better protect your network

You will accumulate an incredible amount of logYou will accumulate an incredible amount of log
data, but the sheer size could prove to be adata, but the sheer size could prove to be a
deterrant to would-be attackersdeterrant to would-be attackers

Not to mention that all access to the onsite andNot to mention that all access to the onsite and
offsite logs will also be heavily loggedoffsite logs will also be heavily logged

Access to the verbose offsite logs will rarely beAccess to the verbose offsite logs will rarely be
time sensitive, so access to those keys could betime sensitive, so access to those keys could be
much more heavily restricted apart from thosemuch more heavily restricted apart from those
timestimes

Not all of these suggestions will not work for everyNot all of these suggestions will not work for every
organizationorganization

The logs you keep on site do not need to be allThe logs you keep on site do not need to be all
that verbosethat verbose

And you don't really need to keep the onsite logsAnd you don't really need to keep the onsite logs
for very long (they're only needed to documentfor very long (they're only needed to document
the things that the IT dept can fix in a short time)the things that the IT dept can fix in a short time)

Also, as these approaches would likely require aAlso, as these approaches would likely require a
significant amount of resources to set up andsignificant amount of resources to set up and
maintain, it would likely not be cost effective formaintain, it would likely not be cost effective for
some organizationssome organizations

However, the kinds of organizations that InfragardHowever, the kinds of organizations that Infragard
focuses on would have such a high potential costfocuses on would have such a high potential cost
to an insider threat that alternate approaches toto an insider threat that alternate approaches to
this problem must at least be consideredthis problem must at least be considered

While there is currently no best solution to theWhile there is currently no best solution to the
problems that insider threats pose to logs andproblems that insider threats pose to logs and
backups, I feel it would be negligent not tobackups, I feel it would be negligent not to
continue looking for one.continue looking for one.

Logs And Backups

Recommended

Recommended

More Related Content

Similar to Logs And Backups

Similar to Logs And Backups (20)

More from Charles Southerland

More from Charles Southerland (6)

Recently uploaded

Recently uploaded (20)

Logs And Backups