research-article

An exponential speedup in parallel running time for submodular maximization without loss in approximation

Authors:

Eric Balkanski,

Aviad Rubinstein,

Yaron SingerAuthors Info & Claims

SODA '19: Proceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms

Pages 283 - 302

Published: 06 January 2019 Publication History

Abstract

In this paper we study the adaptivity of submodular maximization. Adaptivity quantifies the number of sequential rounds that an algorithm makes when function evaluations can be executed in parallel. Adaptivity is a fundamental concept that is heavily studied across a variety of areas in computer science, largely due to the need for parallelizing computation. For the canonical problem of maximizing a monotone submodular function under a cardinality constraint, it is well known that a simple greedy algorithm achieves a 1 - 1/e approximation [NWF78] and that this approximation is optimal for polynomial-time algorithms [NW78]. Somewhat surprisingly, despite extensive efforts on submodular optimization for large-scale datasets, until very recently there was no known algorithm that achieves a constant factor approximation for this problem whose adaptivity is sublinear in the size of the ground set n.

Recent work by [BS18] describes an algorithm that obtains an approximation arbitrarily close to 1/3 in O(logn) adaptive rounds and shows that no algorithm can obtain a constant factor approximation in õ(log n) adaptive rounds. This approach achieves an exponential speedup in adaptivity (and parallel running time) at the expense of approximation quality.

In this paper we describe a novel approach that yields an algorithm whose approximation is arbitrarily close to the optimal 1 - 1/eguarantee in O(log n) adaptive rounds. This algorithm therefore achieves an exponential speedup in parallel running time for submodular maximization at the expense of an arbitrarily small loss in approximation quality. This guarantee is optimal in both approximation and adaptivity, up to lower order terms.

References

[1]

{AAAK17} Arpit Agarwal, Shivani Agarwal, Sepehr Assadi, and Sanjeev Khanna. Learning with limited rounds of adaptivity: Coin tossing, multi-armed bandits, and ranking from pairwise comparisons. In COLT, pages 39--75, 2017.

[2]

{AM10} Saeed Alaei and Azarakhsh Malekian. Maximizing sequence-submodular functions and its application to online advertising. arXiv preprint arXiv:1009.4153, 2010.

[3]

{ANRW15} Noga Alon, Noam Nisan, Ran Raz, and Omri Weinstein. Welfare maximization with limited interaction. In Foundations of Computer Science (FOCS), 2015 IEEE 56th Annual Symposium on, pages 1499--1512. IEEE, 2015.

Digital Library

[4]

{AWZ08} Akram Aldroubi, Haichao Wang, and Kourosh Zarringhalam. Sequential adaptive compressed sampling via huffman codes. arXiv preprint arXiv:0810.4916, 2008.

[5]

{BDF+12} Ashwinkumar Badanidiyuru, Shahar Dobzinski, Hu Fu, Robert Kleinberg, Noam Nisan, and Tim Roughgarden. Sketching valuation functions. In SODA, pages 1025--1035. Society for Industrial and Applied Mathematics, 2012.

Digital Library

[6]

{BENW15} Rafael Barbosa, Alina Ene, Huy Nguyen, and Justin Ward. The power of randomization: Distributed submodular maximization on massive datasets. In ICML, pages 1236--1244, 2015.

Digital Library

[7]

{BENW16} Rafael da Ponte Barbosa, Alina Ene, Huy L Nguyen, and Justin Ward. A new framework for distributed submodular maximization. In FOCS, pages 645--654. Ieee, 2016.

[8]

{BGSMdW12} Harry Buhrman, David García-Soriano, Arie Matsliah, and Ronald de Wolf. The non-adaptive query complexity of testing k-parities. arXiv preprint arXiv:1209.3849, 2012.

[9]

{Ble96} Guy E Blelloch. Programming parallel algorithms. Communications of the ACM, 39(3):85--97, 1996.

Digital Library

[10]

{BMKK14} Ashwinkumar Badanidiyuru, Baharan Mirzasoleiman, Amin Karbasi, and Andreas Krause. Streaming submodular maximization: massive data summarization on the fly. In The 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '14, New York, NY, USA - August 24 -- 27, 2014, pages 671--680, 2014.

Digital Library

[11]

{BMW16} Mark Braverman, Jieming Mao, and S Matthew Weinberg. Parallel algorithms for select and partition with noisy comparisons. In STOC, pages 851--862, 2016.

Digital Library

[12]

{BPR+16} Ashwinkumar Badanidiyuru, Christos Papadimitriou, Aviad Rubinstein, Lior Seeman, and Yaron Singer. Locally adaptive optimization: Adaptive seeding for monotone submodular functions. In Proceedings of the Twenty-Seventh Annual ACM-SIAM Symposium on Discrete Algorithms, pages 414--429. Society for Industrial and Applied Mathematics, 2016.

Digital Library

[13]

{BPT11} Guy E Blelloch, Richard Peng, and Kanat Tangwongsan. Linear-work greedy parallel approximate set cover and variants. In SPAA, pages 23--32, 2011.

Digital Library

[14]

{BRM98} Guy E Blelloch and Margaret Reid-Miller. Fast set operations using treaps. In SPAA, pages 16--26, 1998.

Digital Library

[15]

{BRS89} Bonnie Berger, John Rompel, and Peter W Shor. Efficient nc algorithms for set cover with applications to learning and geometry. In FOCS, pages 54--59. IEEE, 1989.

Digital Library

[16]

{BS18} Eric Balkanski and Yaron Singer. The adaptive complexity of maximizing a submodular function. In STOC, 2018.

Digital Library

[17]

{BST12} Guy E Blelloch, Harsha Vardhan Simhadri, and Kanat Tangwongsan. Parallel and i/o efficient set covering algorithms. In SPAA, pages 82--90. ACM, 2012.

Digital Library

[18]

{BV14} Ashwinkumar Badanidiyuru and Jan Vondrák. Fast algorithms for maximizing submodular functions. In Proceedings of the Twenty-Fifth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2014, Portland, Oregon, USA, January 5-7, 2014, pages 1497--1514, 2014.

Digital Library

[19]

{CG17} Clement Canonne and Tom Gur. An adaptivity hierarchy theorem for property testing. arXiv preprint arXiv:1702.05678, 2017.

[20]

{Col88} Richard Cole. Parallel merge sort. SIAM Journal on Computing, 17(4):770--785, 1988.

Digital Library

[21]

{CST+17} Xi Chen, Rocco A Servedio, Li-Yang Tan, Erik Waingarten, and Jinyu Xie. Settling the query complexity of non-adaptive junta testing. arXiv preprint arXiv:1704.06314, 2017.

[22]

{CWW10} Wei Chen, Chi Wang, and Yajun Wang. Scalable influence maximization for prevalent viral marketing in large-scale social networks. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1029--1038. ACM, 2010.

Digital Library

[23]

{CWY09} Wei Chen, Yajun Wang, and Siyu Yang. Efficient influence maximization in social networks. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 199--208. ACM, 2009.

Digital Library

[24]

{DG08} Jeffrey Dean and Sanjay Ghemawat. Mapreduce: simplified data processing on large clusters. Communications of the ACM, 51(1):107--113, 2008.

Digital Library

[25]

{DGS84} Pavol Duris, Zvi Galil, and Georg Schnitger. Lower bounds on communication complexity. In STOC, pages 81--91, 1984.

Digital Library

[26]

{DHK+16} Nikhil R Devanur, Zhiyi Huang, Nitish Korula, Vahab S Mirrokni, and Qiqi Yan. Whole-page optimization and sub-modular welfare maximization with online bidders. ACM Transactions on Economics and Computation, 4(3):14, 2016.

Digital Library

[27]

{DNO14} Shahar Dobzinski, Noam Nisan, and Si-gal Oren. Economic efficiency requires interaction. In Proceedings of the forty-sixth annual ACM symposium on Theory of computing, pages 233--242. ACM, 2014.

Digital Library

[28]

{DR01} Pedro Domingos and Matt Richardson. Mining the network value of customers. In Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, pages 57--66. ACM, 2001.

Digital Library

[29]

{EMZ17} Alessandro Epasto, Vahab S. Mirrokni, and Morteza Zadimoghaddam. Bicri-teria distributed submodular maximization in a few rounds. In SPAA, pages 25--33, 2017.

Digital Library

[30]

{FJK10} Joseph D Frazier, Peter K Jimack, and Robert M Kirby. On the use of adjoint-based sensitivity estimates to control local mesh refinement. Commun Comput Phys, 7:631-8, 2010.

[31]

{GK10} Daniel Golovin and Andreas Krause. Adaptive submodularity: A new approach to active learning and stochastic optimization. In COLT, pages 333--345, 2010.

[32]

{GLL11} Amit Goyal, Wei Lu, and Laks VS Lakshmanan. Celf++: optimizing the greedy algorithm for influence maximization in social networks. In Proceedings of the 20th international conference companion on World wide web, pages 47--48. ACM, 2011.

Digital Library

[33]

{HBCN09} Jarvis D Haupt, Richard G Baraniuk, Rui M Castro, and Robert D Nowak. Compressive distilled sensing: Sparse recovery using adaptivity in compres-sive measurements. In Signals, Systems and Computers, 2009 Conference Record of the Forty-Third Asilomar Conference on, pages 1551--1555. IEEE, 2009.

Digital Library

[34]

{HNC09} Jarvis Haupt, Robert Nowak, and Rui Castro. Adaptive sensing for sparse signal recovery. In Digital Signal Processing Workshop and 5th IEEE Signal Processing Education Workshop, pages 702--707. IEEE, 2009.

[35]

{HS15} Thibaut Horel and Yaron Singer. Scalable methods for adaptively seeding a social network. In Proceedings of the 24th International Conference on World Wide Web, pages 441--451. International World Wide Web Conferences Steering Committee, 2015.

Digital Library

[36]

{IPW11} Piotr Indyk, Eric Price, and David P Woodruff. On the power of adaptivity in sparse recovery. In FOCS, pages 285--294. IEEE, 2011.

Digital Library

[37]

{JXC08} Shihao Ji, Ya Xue, and Lawrence Carin. Bayesian compressive sensing. IEEE Transactions on Signal Processing, 56(6):2346--2356, 2008.

Digital Library

[38]

{KKT03} David Kempe, Jon Kleinberg, and Éva Tardos. Maximizing the spread of influence through a social network. In Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 137--146. ACM, 2003.

Digital Library

[39]

{KMVV15} Ravi Kumar, Benjamin Moseley, Sergei Vassilvitskii, and Andrea Vattani. Fast greedy algorithms in mapreduce and streaming. ACM Transactions on Parallel Computing, 2(3):14, 2015.

Digital Library

[40]

{KSV10} Howard Karloff, Siddharth Suri, and Sergei Vassilvitskii. A model of computation for mapreduce. In SODA, pages 938--948, 2010.

Digital Library

[41]

{MBK+15} Baharan Mirzasoleiman, Ashwinkumar Badanidiyuru, Amin Karbasi, Jan Vondrák, and Andreas Krause. Lazier than lazy greedy. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, January 25--30, 2015, Austin, Texas, USA., pages 1812--1818, 2015.

Digital Library

[42]

{MBK16} Baharan Mirzasoleiman, Ashwinkumar Badanidiyuru, and Amin Karbasi. Fast constrained submodular maximization: Personalized data summarization. In ICML, pages 1358--1367, 2016.

Digital Library

[43]

{MKBK15} Baharan Mirzasoleiman, Amin Karbasi, Ashwinkumar Badanidiyuru, and Andreas Krause. Distributed submodular cover: Succinctly summarizing massive data. In NIPS, pages 2881--2889, 2015.

Digital Library

[44]

{MKSK13} Baharan Mirzasoleiman, Amin Kar-basi, Rik Sarkar, and Andreas Krause. Distributed submodular maximization: Identifying representative elements in massive data. In NIPS, pages 2049--2057, 2013.

Digital Library

[45]

{MNSW95} Peter Bro Miltersen, Noam Nisan, Shmuel Safra, and Avi Wigderson. On data structures and asymmetric communication complexity. In STOC, pages 103--111. ACM, 1995.

Digital Library

[46]

{MSW08} Dmitry M Malioutov, Sujay Sanghavi, and Alan S Willsky. Compressed sensing with sequential observations. In Acoustics, Speech and Signal Processing, 2008. ICASSP 2008. IEEE International Conference on, pages 3357--3360. IEEE, 2008.

[47]

{MZ15} Vahab Mirrokni and Morteza Zadi-moghaddam. Randomized compos-able core-sets for distributed submodu-lar maximization. In STOC, pages 153--162, 2015.

Digital Library

[48]

{NW78} George L Nemhauser and Laurence A Wolsey. Best algorithms for approximating the maximum of a submodular set function. Mathematics of operations research, 3(3):177--188, 1978.

Digital Library

[49]

{NW91} Noam Nisan and Avi Widgerson. Rounds in communication complexity revisited. In STOC, pages 419--429, 1991.

Digital Library

[50]

{NWF78} George L Nemhauser, Laurence A Wolsey, and Marshall L Fisher. An analysis of approximations for maximizing submodular set functionsi. Mathematical Programming, 14(1):265--294, 1978.

Digital Library

[51]

{PJG+14} Xinghao Pan, Stefanie Jegelka, Joseph E Gonzalez, Joseph K Bradley, and Michael I Jordan. Parallel double greedy submodular maximization. In Advances in Neural Information Processing Systems, pages 118--126, 2014.

Digital Library

[52]

{PS84} Christos H Papadimitriou and Michael Sipser. Communication complexity. Journal of Computer and System Sciences, 28(2):260--269, 1984.

[53]

{RD02} Matthew Richardson and Pedro Domin-gos. Mining knowledge-sharing sites for viral marketing. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 61--70. ACM, 2002.

Digital Library

[54]

{RS06} Sofya Raskhodnikova and Adam Smith. A note on adaptivity in testing properties of bounded degree graphs. ECCC, TR06--089, 2006.

[55]

{RV98} Sridhar Rajagopalan and Vijay V Vazi-rani. Primal-dual rnc approximation algorithms for set cover and covering integer programs. SIAM Journal on Computing, 28(2):525--540, 1998.

Digital Library

[56]

{SS13} Lior Seeman and Yaron Singer. Adaptive seeding in social networks. In Foundations of Computer Science (FOCS), 2013 IEEE 54th Annual Symposium on, pages 459--468. IEEE, 2013.

Digital Library

[57]

{STK16} Adish Singla, Sebastian Tschiatschek, and Andreas Krause. Noisy submodu-lar maximization via adaptive sampling with applications to crowdsourced image collection summarization. In AAAI, pages 2037--2043, 2016.

Digital Library

[58]

{STW15} Rocco A Servedio, Li-Yang Tan, and John Wright. Adaptivity helps for testing juntas. In Proceedings of the 30th Conference on Computational Complexity, pages 264--279. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2015.

Digital Library

[59]

{TIWB14} Sebastian Tschiatschek, Rishabh K Iyer, Haochen Wei, and Jeff A Bilmes. Learning mixtures of submodular functions for image collection summarization. In Advances in neural information processing systems, pages 1413--1421, 2014.

Digital Library

[60]

{Val75} Leslie G Valiant. Parallelism in comparison problems. SIAM Journal on Computing, 4(3):348--355, 1975.

Cited By

Chen XPeng BLeonardi SGupta A(2022)On the complexity of dynamic submodular maximizationProceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing10.1145/3519935.3519951(1685-1698)Online publication date: 9-Jun-2022
https://dl.acm.org/doi/10.1145/3519935.3519951
Wang YLiu XZheng ZZhang ZXu MYu CWu F(2022)On Designing a Two-stage Auction for Online AdvertisingProceedings of the ACM Web Conference 202210.1145/3485447.3512054(90-99)Online publication date: 25-Apr-2022
https://dl.acm.org/doi/10.1145/3485447.3512054
Balkanski ESinger YMakarychev KMakarychev YTulsiani MKamath GChuzhoy J(2020)A lower bound for parallel submodular minimizationProceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing10.1145/3357713.3384287(130-139)Online publication date: 22-Jun-2020
https://dl.acm.org/doi/10.1145/3357713.3384287
Show More Cited By

An exponential speedup in parallel running time for submodular maximization without loss in approximation
1. Computing methodologies
2. Theory of computation

Recommendations

Submodular maximization with nearly-optimal approximation and adaptivity in nearly-linear time
SODA '19: Proceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms

In this paper, we study the tradeoff between the approximation guarantee and adaptivity for the problem of maximizing a monotone submodular function subject to a cardinality constraint. The adaptivity of an algorithm is the number of sequential rounds ...
Continuous submodular maximization: beyond DR-submodularity
NIPS '20: Proceedings of the 34th International Conference on Neural Information Processing Systems

In this paper, we propose the first continuous optimization algorithms that achieve a constant factor approximation guarantee for the problem of monotone continuous submodular maximization subject to a linear constraint. We first prove that a simple ...
Submodular maximization with matroid and packing constraints in parallel
STOC 2019: Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing

We consider the problem of maximizing the multilinear extension of a submodular function subject a single matroid constraint or multiple packing constraints with a small number of adaptive rounds of evaluation queries.

We obtain the first algorithms ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

SODA '19: Proceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms

January 2019

2993 pages

Program Chair:
Timothy M. Chan
University of Illinois at Urbana-Champaign

Sponsors

SIAM Activity Group on Discrete Mathematics

In-Cooperation

SIGACT: ACM Special Interest Group on Algorithms and Computation Theory

Publisher

Society for Industrial and Applied Mathematics

United States

Publication History

Published: 06 January 2019

Check for updates

Qualifiers

Research-article

Conference

SODA '19

Sponsor:

SODA '19: Symposium on Discrete Algorithms

January 6 - 9, 2019

California, San Diego

Acceptance Rates

Overall Acceptance Rate 411 of 1,322 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
102
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chen XPeng BLeonardi SGupta A(2022)On the complexity of dynamic submodular maximizationProceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing10.1145/3519935.3519951(1685-1698)Online publication date: 9-Jun-2022
https://dl.acm.org/doi/10.1145/3519935.3519951
Wang YLiu XZheng ZZhang ZXu MYu CWu F(2022)On Designing a Two-stage Auction for Online AdvertisingProceedings of the ACM Web Conference 202210.1145/3485447.3512054(90-99)Online publication date: 25-Apr-2022
https://dl.acm.org/doi/10.1145/3485447.3512054
Balkanski ESinger YMakarychev KMakarychev YTulsiani MKamath GChuzhoy J(2020)A lower bound for parallel submodular minimizationProceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing10.1145/3357713.3384287(130-139)Online publication date: 22-Jun-2020
https://dl.acm.org/doi/10.1145/3357713.3384287
Chekuri CQuanrud KChan T(2019)Submodular function maximization in parallel via the multilinear relaxationProceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms10.5555/3310435.3310455(303-322)Online publication date: 6-Jan-2019
https://dl.acm.org/doi/10.5555/3310435.3310455
Fahrbach MMirrokni VZadimoghaddam MChan T(2019)Submodular maximization with nearly optimal approximation, adaptivity and query complexityProceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms10.5555/3310435.3310452(255-273)Online publication date: 6-Jan-2019
https://dl.acm.org/doi/10.5555/3310435.3310452
Avdiukhin DMitrović SYaroslavtsev GZhou STeredesai AKumar VLi YRosales RTerzi EKarypis G(2019)Adversarially Robust Submodular Maximization under Knapsack ConstraintsProceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining10.1145/3292500.3330911(148-156)Online publication date: 25-Jul-2019
https://dl.acm.org/doi/10.1145/3292500.3330911

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents