Article

Efficient data structures for tamper-evident logging

Authors:

Scott A. Crosby,

Dan S. WallachAuthors Info & Claims

SSYM'09: Proceedings of the 18th conference on USENIX security symposium

Pages 317 - 334

Published: 10 August 2009 Publication History

Abstract

Many real-world applications wish to collect tamperevident logs for forensic purposes. This paper considers the case of an untrusted logger, serving a number of clients who wish to store their events in the log, and kept honest by a number of auditors who will challenge the logger to prove its correct behavior. We propose semantics of tamper-evident logs in terms of this auditing process. The logger must be able to prove that individual logged events are still present, and that the log, as seen now, is consistent with how it was seen in the past. To accomplish this efficiently, we describe a tree-based data structure that can generate such proofs with logarithmic size and space, improving over previous linear constructions. Where a classic hash chain might require an 800 MB trace to prove that a randomly chosen event is in a log with 80 million events, our prototype returns a 3 KB proof with the same semantics. We also present a flexible mechanism for the log server to present authenticated and tamper-evident search results for all events matching a predicate. This can allow large-scale log servers to selectively delete old events, in an agreed-upon fashion, while generating efficient proofs that no inappropriate events were deleted. We describe a prototype implementation and measure its performance on an 80 million event syslog trace at 1,750 events per second using a single CPU core. Performance improves to 10,500 events per second if cryptographic signatures are offloaded, corresponding to 1.1 TB of logging throughput per week.

References

[1]

ACCORSI, R., AND HOHL, A. Delegating secure logging in pervasive computing systems. In Security in Pervasive Computing (York, UK, Apr. 2006), pp. 58-72.

Digital Library

[2]

ANAGNOSTOPOULOS, A., GOODRICH, M. T., AND TAMASSIA, R. Persistent authenticated dictionaries and their applications. In International Conference on Information Security (ISC) (Seoul, Korea, Dec. 2001), pp. 379-393.

Digital Library

[3]

BELLARE, M., AND MINER, S. K. A forward-secure digital signature scheme. In CRYPTO '99 (Santa Barbara, CA, Aug. 1999), pp. 431-448.

Digital Library

[4]

BELLARE, M., AND YEE, B. S. Forward integrity for secure audit logs. Tech. rep., University of California at San Diego, Nov. 1997.

[5]

BENALOH, J., AND DE MARE, M. One-way accumulators: a decentralized alternative to digital signatures. In Workshop on the Theory and Application of Cryptographic Techniques on Advances in Cryptology (EuroCrypt '93) (Lofthus, Norway, May 1993), pp. 274-285.

Digital Library

[6]

BETHENCOURT, J., BONEH, D., AND WATERS, B. Cryptographic methods for storing ballots on a voting machine. In Network and Distributed System Security Symposium (NDSS) (San Diego, CA, Feb. 2007).

[7]

BLIBECH, K., AND GABILLON, A. CHRONOS: An authenticated dictionary based on skip lists for timestamping systems. In Workshop on Secure Web Services (Fairfax, VA, Nov. 2005), pp. 84-90.

Digital Library

[8]

BLOOM, B. H. Space/time trade-offs in hash coding with allowable errors. Communications of the ACM 13, 7 (1970), 422-426.

Digital Library

[9]

BULDAS, A., LAUD, P., LIPMAA, H., AND WILLEMSON, J. Time-stamping with binary linking schemes. In CRYPTO '98 (Santa Barbara, CA, Aug. 1998), pp. 486-501.

Digital Library

[10]

BULDAS, A., LIPMAA, H., AND SCHOENMAKERS, B. Optimally efficient accountable time-stamping. In International Workshop on Practice and Theory in Public Key Cryptography (PKC) (Melbourne, Victoria, Australia, Jan. 2000), pp. 293-305.

Digital Library

[11]

CAMENISCH, J., AND LYSYANSKAYA, A. Dynamic accumulators and application to efficient revocation of anonymous credentials. In CRYPTO '02 (Santa Barbara, CA, Aug. 2002), pp. 61-76.

Digital Library

[12]

CHAN, H., PERRIG, A., PRZYDATEK, B., AND SONG, D. SIA: Secure information aggregation in sensor networks. Journal Computer Security 15, 1 (2007), 69-102.

Digital Library

[13]

CHAN, H., PERRIG, A., AND SONG, D. Secure hierarchical in-network aggregation in sensor networks. In ACM Conference on Computer and Communications Security (CCS '06) (Alexandria, VA, Oct. 2006), pp. 278-287.

Digital Library

[14]

CHOR, B., GOLDREICH, O., KUSHILEVITZ, E., AND SUDAN, M. Private information retrieval. In Annual Symposium on Foundations of Computer Science (Milwaukee, WI, Oct. 1995), pp. 41-50.

Digital Library

[15]

CHUN, B.-G., MANIATIS, P., SHENKER, S., AND KUBIATOWICZ, J. Attested append-only memory: Making adversaries stick to their word. In SOSP '07 (Stevenson, WA, Oct. 2007), pp. 189-204.

Digital Library

[16]

D. S. PARKER, J., POPEK, G. J., RUDISIN, G., STOUGHTON, A., WALKER, B. J., WALTON, E., CHOW, J. M., EDWARDS, D., KISER, S., AND KLINE, C. Detection of mutual inconsistency in distributed systems. IEEE Transactions on Software Engineering 9, 3 (1983), 240-247.

Digital Library

[17]

DAVIS, D., MONROSE, F., AND REITER, M. K. Time-scoped searching of encrypted audit logs. In Information and Communications Security Conference (Malaga, Spain, Oct. 2004), pp. 532-545.

[18]

DEUTSCH, P. Gzip file format specification version 4.3. RFC 1952, May 1996. http://www.ietf.org/rfc/rfc1952.txt.

Digital Library

[19]

DEVANBU, P., GERTZ, M., KWONG, A., MARTEL, C., NUCKOLLS, G., AND STUBBLEBINE, S. G. Flexible authentication of XML documents. Journal of Computer Security 12, 6 (2004), 841-864.

Digital Library

[20]

DEVANBU, P., GERTZ, M., MARTEL, C., AND STUBBLEBINE, S. G. Authentic data publication over the internet. Journal Computer Security 11, 3 (2003), 291-314.

Digital Library

[21]

GENNARO, R., AND ROHATGI, P. How to sign digital streams. In CRYPTO '97 (Santa Barbara, CA, Aug. 1997), pp. 180-197.

Digital Library

[22]

GERR, P. A., BABINEAU, B., AND GORDON, P. C. Compliance: The effect on information management and the storage industry. The Enterprise Storage Group, May 2003. http://searchstorage.techtarget.com/tip/0,289483, sid5 gci906152,00.html.

[23]

GOH, E.-J. Secure indexes. Cryptology ePrint Archive, Report 2003/216, 2003. http://eprint.iacr.org/2003/216/ See also http://eujingoh.com/papers/secureindex/.

[24]

GOODRICH, M., TAMASSIA, R., AND SCHWERIN, A. Implementation of an authenticated dictionary with skip lists and commutative hashing. In DARPA Information Survivability Conference & Exposition II (DISCEX II) (Anaheim, CA, June 2001), pp. 68-82.

[25]

GOODRICH, M. T., TAMASSIA, R., TRIANDOPOULOS, N., AND COHEN, R. F. Authenticated data structures for graph and geometric searching. In Topics in Cryptology, The Cryptographers' Track at the RSA Conference (CT-RSA) (San Francisco, CA, Apr. 2003), pp. 295-313.

Digital Library

[26]

GOYAL, V., PANDEY, O., SAHAI, A., AND WATERS, B. Attribute-based encryption for fine-grained access control of encrypted data. In ACM Conference on Computer and Communications Security (CCS '06) (Alexandria, Virginia, Oct. 2006), pp. 89-98.

Digital Library

[27]

HABER, S., AND STORNETTA, W. S. How to time-stamp a digital document. In CRYPTO '98 (Santa Barbara, CA, 1990), pp. 437-455.

Digital Library

[28]

HAEBERLEN, A., KOUZNETSOV, P., AND DRUSCHEL, P. PeerReview: Practical accountability for distributed systems. In SOSP '07 (Stevenson, WA, Oct. 2007).

Digital Library

[29]

HOLT, J. E. Logcrypt: Forward security and public verification for secure audit logs. In Australasian Workshops on Grid Computing and E-research (Hobart, Tasmania, Australia, 2006).

Digital Library

[30]

HU, L., AND EVANS, D. Secure aggregation for wireless networks. In Symposium on Applications and the Internet Workshops (SAINT) (Orlando, FL, July 2003), p. 384.

Digital Library

[31]

ITKIS, G. Cryptographic tamper evidence. In ACM Conference on Computer and Communications Security (CCS '03) (Washington D.C., Oct. 2003), pp. 355-364.

Digital Library

[32]

KELSEY, J., CALLAS, J., AND CLEMM, A. Signed Syslog messages. http://tools.ietf.org/id/draft-ietf-syslog-sign-23.txt (work in progress), Sept. 2007.

[33]

KILTZ, E., MITYAGIN, A., PANJWANI, S., AND RAGHAVAN, B. Append-only signatures. In International Colloquium on Automata, Languages and Programming (Lisboa, Portugal, July 2005).

Digital Library

[34]

KOCHER, P. C. On certificate revocation and validation. In International Conference on Financial Cryptography (FC '98) (Anguilla, British West Indies, Feb. 1998), pp. 172-177.

Digital Library

[35]

KOTLA, R., ALVISI, L., DAHLIN, M., CLEMENT, A., AND WONG, E. Zyzzyva: Speculative byzantine fault tolerance. In SOSP '07 (Stevenson, WA, Oct. 2007), pp. 45-58.

Digital Library

[36]

LI, J., KROHN, M., MAZIÈ RES, D., AND SHASHA, D. Secure untrusted data repository (SUNDR). In Operating Systems Design & Implementation (OSDI) (San Francisco, CA, Dec. 2004).

Digital Library

[37]

LIPMAA, H. On optimal hash tree traversal for interval time-stamping. In Proceedings of the 5th International Conference on Information Security (ISC02) (Seoul, Korea, Nov. 2002), pp. 357-371.

Digital Library

[38]

LONVICK, C. The BSD Syslog protocol. RFC 3164, Aug. 2001. http://www.ietf.org/rfc/rfc3164.txt.

[39]

MA, D. Practical forward secure sequential aggregate signatures. In Proceedings of the 2008 ACM symposium on Information, computer and communications security (ASIACCS'08) (Tokyo, Japan, Mar. 2008), pp. 341-352.

Digital Library

[40]

MA, D., AND TSUDIK, G. Forward-secure sequential aggregate authentication. In Proceedings of the 2007 IEEE Symposium on Security and Privacy (Oakland, CA, May 2007), IEEE Computer Society, pp. 86-91.

Digital Library

[41]

MA, D., AND TSUDIK, G. A new approach to secure logging. Transactions on Storage 5, 1 (2009), 1-21.

Digital Library

[42]

MANIATIS, P., AND BAKER, M. Enabling the archival storage of signed documents. In FAST '02: Proceedings of the 1st USENIX Conference on File and Storage Technologies (Monterey, CA, 2002).

Digital Library

[43]

MANIATIS, P., AND BAKER, M. Secure history preservation through timeline entanglement. In USENIX Security Symposium (San Francisco, CA, Aug. 2002).

Digital Library

[44]

MANIATIS, P., ROUSSOPOULOS, M., GIULI, T. J., ROSENTHAL, D. S. H., AND BAKER, M. The LOCKSS peer-to-peer digital preservation system. ACM Transactions on Computer Systems 23, 1 (2005), 2-50.

Digital Library

[45]

MANULIS, M., AND SCHWENK, J. Provably secure framework for information aggregation in sensor networks. In Computational Science and Its Applications (ICCSA) (Kuala Lumpur, Malaysia, Aug. 2007), pp. 603-621.

Digital Library

[46]

MERKLE, R. C. A digital signature based on a conventional encryption function. In CRYPTO '88 (1988), pp. 369-378.

Digital Library

[47]

MITRA, S., HSU, W. W., AND WINSLETT, M. Trustworthy keyword search for regulatory-compliant records retention. In International Conference on Very Large Databases (VLDB) (Seoul, Korea, Sept. 2006), pp. 1001-1012.

Digital Library

[48]

MONTEIRO, S. D. S., AND ERBACHER, R. F. Exemplifying attack identification and analysis in a novel forensically viable Syslog model. In Workshop on Systematic Approaches to Digital Forensic Engineering (Oakland, CA, May 2008), pp. 57-68.

Digital Library

[49]

NAOR, M., AND NISSIM, K. Certificate revocation and certificate update. In USENIX Security Symposium (San Antonio, TX, Jan. 1998).

Digital Library

[50]

OSTROVSKY, R., SAHAI, A., AND WATERS, B. Attribute-based encryption with non-monotonic access structures. In ACM Conference on Computer and Communications Security (CCS '07) (Alexandria, VA, Oct. 2007), pp. 195-203.

Digital Library

[51]

PAVLOU, K., AND SNODGRASS, R. T. Forensic analysis of database tampering. In ACM SIGMOD International Conference on Management of Data (Chicago, IL, June 2006), pp. 109-120.

Digital Library

[52]

PETERSON, Z. N. J., BURNS, R., ATENIESE, G., AND BONO, S. Design and implementation of verifiable audit trails for a versioning file system. In USENIX Conference on File and Storage Technologies (San Jose, CA, Feb. 2007).

Digital Library

[53]

PUGH, W. Skip lists: A probabilistic alternative to balanced trees. In Workshop on Algorithms and Data Structures (1989), pp. 437-449.

Digital Library

[54]

SAHAI, A., AND WATERS, B. Fuzzy identity based encryption. In Workshop on the Theory and Application of Cryptographic Techniques on Advances in Cryptology (EuroCrypt '05) (May 2005), vol. 3494, pp. 457 - 473.

Digital Library

[55]

SANDLER, D., AND WALLACH, D. S. Casting votes in the Auditorium. In USENIX/ACCURATE Electronic Voting Technology Workshop (EVT'07) (Boston, MA, Aug. 2007).

Digital Library

[56]

SCHNEIER, B., AND KELSEY, J. Automatic event-stream notarization using digital signatures. In Security Protocols Workshop (Cambridge, UK, Apr. 1996), pp. 155-169.

Digital Library

[57]

SCHNEIER, B., AND KELSEY, J. Secure audit logs to support computer forensics. ACM Transactions on Information and System Security 1, 3 (1999).

Digital Library

[58]

SION, R. Strong WORM. In International Conference on Distributed Computing Systems (Beijing, China, May 2008), pp. 69-76.

Digital Library

[59]

SNODGRASS, R. T., YAO, S. S., AND COLLBERG, C. Tamper detection in audit logs. In Conference on Very Large Data Bases (VLDB) (Toronto, Canada, Aug. 2004), pp. 504-515.

Digital Library

[60]

SONG, D. X., WAGNER, D., AND PERRIG, A. Practical techniques for searches on encrypted data. In IEEE Symposium on Security and Privacy (Berkeley, CA, May 2000), pp. 44-55.

Digital Library

[61]

WATERS, B. R., BALFANZ, D., DURFEE, G., AND SMETTERS, D. K. Building an encrypted and searchable audit log. In Network and Distributed System Security Symposium (NDSS) (San Diego, CA, Feb. 2004).

[62]

WEATHERSPOON, H., WELLS, C., AND KUBIATOWICZ, J. Naming and integrity: Self-verifying data in peer-to-peer systems. In Future Directions in Distributed Computing (2003), vol. 2584 of Lecture Notes in Computer Science, pp. 142-147.

Digital Library

[63]

YUMEREFENDI, A. R., AND CHASE, J. S. Strong accountability for network storage. ACM Transactions on Storage 3, 3 (2007).

Digital Library

[64]

ZHU, Q., AND HSU, W. W. Fossilized index: The linchpin of trustworthy non-alterable electronic records. In ACM SIGMOD International Conference on Management of Data (Baltimore, MD, June 2005), pp. 395-406.

Digital Library

Cited By

Cox RGriesemer RPike RTaylor IThompson K(2022)The Go programming language and environmentCommunications of the ACM10.1145/348871665:5(70-78)Online publication date: 25-Apr-2022
https://dl.acm.org/doi/10.1145/3488716
Panwar NSharma SWang GMehrotra SVenkatasubramanian NDiallo MSani A(2021)IoT Notary: Attestable Sensor Data Capture in IoT EnvironmentsACM Transactions on Internet of Things10.1145/34782903:1(1-30)Online publication date: 27-Oct-2021
https://dl.acm.org/doi/10.1145/3478290
Kroll J(2021)Outlining TraceabilityProceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency10.1145/3442188.3445937(758-771)Online publication date: 3-Mar-2021
https://dl.acm.org/doi/10.1145/3442188.3445937
Show More Cited By

Recommendations

Efficient tamper-evident data structures for untrusted servers
Finding the Evidence in Tamper-Evident Logs
SADFE '08: Proceedings of the 2008 Third International Workshop on Systematic Approaches to Digital Forensic Engineering

Secure logs are powerful tools for building systems that must resist forgery, prove temporal relationships, and stand up to forensic scrutiny. The proofs of order and integrity encoded in these tamper-evident chronological records, typically built using ...
Squeezing succinct data structures into entropy bounds
SODA '06: Proceedings of the seventeenth annual ACM-SIAM symposium on Discrete algorithm

Consider a sequence S of n symbols drawn from an alphabet A = {1, 2,. . .,σ}, stored as a binary string of nlog σ bits. A succinct data structure on S supports a given set of primitive operations on S using just f (n) = o(n log σ) extra bits. We present ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

SSYM'09: Proceedings of the 18th conference on USENIX security symposium

August 2009

432 pages

Publisher

USENIX Association

United States

Publication History

Published: 10 August 2009

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

26
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cox RGriesemer RPike RTaylor IThompson K(2022)The Go programming language and environmentCommunications of the ACM10.1145/348871665:5(70-78)Online publication date: 25-Apr-2022
https://dl.acm.org/doi/10.1145/3488716
Panwar NSharma SWang GMehrotra SVenkatasubramanian NDiallo MSani A(2021)IoT Notary: Attestable Sensor Data Capture in IoT EnvironmentsACM Transactions on Internet of Things10.1145/34782903:1(1-30)Online publication date: 27-Oct-2021
https://dl.acm.org/doi/10.1145/3478290
Kroll J(2021)Outlining TraceabilityProceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency10.1145/3442188.3445937(758-771)Online publication date: 3-Mar-2021
https://dl.acm.org/doi/10.1145/3442188.3445937
de Vos MPouwelse J(2020)ConTribProceedings of the 1st International Workshop on Distributed Infrastructure for Common Good10.1145/3428662.3428789(13-18)Online publication date: 7-Dec-2020
https://dl.acm.org/doi/10.1145/3428662.3428789
Chiasserini CGiaccone PMalnati GMacagno MSviridov G(2020)Blockchain-based Mobility Verification of Connected Cars2020 IEEE 17th Annual Consumer Communications & Networking Conference (CCNC)10.1109/CCNC46108.2020.9045104(1-6)Online publication date: 10-Jan-2020
https://dl.acm.org/doi/10.1109/CCNC46108.2020.9045104
De Coninck QMichel FPiraux MRochet FGiven-Wilson TLegay APereira OBonaventure OWu JHall W(2019)Pluginizing QUICProceedings of the ACM Special Interest Group on Data Communication10.1145/3341302.3342078(59-74)Online publication date: 19-Aug-2019
https://dl.acm.org/doi/10.1145/3341302.3342078
Wang GShi ZNixon MHan S(2019)SoKProceedings of the 1st ACM Conference on Advances in Financial Technologies10.1145/3318041.3355457(41-61)Online publication date: 21-Oct-2019
https://dl.acm.org/doi/10.1145/3318041.3355457
Wang HYang GChinprutthiwong PXu LZhang YGu GLie DMannan MBackes MWang X(2018)Towards Fine-grained Network Security Forensics and Diagnosis in the SDN EraProceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security10.1145/3243734.3243749(3-16)Online publication date: 15-Oct-2018
https://dl.acm.org/doi/10.1145/3243734.3243749
Sutton ASamavi R(2018)Timestamp-based Integrity Proofs for Linked DataProceedings of the International Workshop on Semantic Big Data10.1145/3208352.3208353(1-6)Online publication date: 10-Jun-2018
https://dl.acm.org/doi/10.1145/3208352.3208353
Yu YAfanasyev ASeedorf JZhang ZZhang LSchmidt TSeedorf J(2017)NDN DeLoreanProceedings of the 4th ACM Conference on Information-Centric Networking10.1145/3125719.3125724(11-21)Online publication date: 26-Sep-2017
https://dl.acm.org/doi/10.1145/3125719.3125724
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Table of Contents