General Elastic Computing Acceleration Method for Semantic Segmentation
Authors: 
Yiying Wei, Yiyang Jiang, Wei Dai, Zhijie Yang, and Ao JiangAuthors Info & Claims
Yiying Wei, Yiyang Jiang, Wei Dai, Zhijie Yang, and Ao JiangAuthors Info & ClaimsMarch 2023
Pages 139 - 144
Abstract
In the field of computer vision, deep convolution neural network plays an increasingly important role, and its accuracy in various tasks has gradually exceeded the traditional method. However, real-time is important in autonomous driving and mobile robot scenarios. Most processes need to be run locally and completed in a short time with sufficient precision. Semantic segmentation based on deep convolutional neural networks consumes huge computing resources, and the balance of accuracy and speed has been a difficult problem in industrial deployment. We propose a general elastic computing acceleration method for semantic segmentation, which makes full use of the correlation between the front and back frames of the input video. We use optical flow, fuzzy prediction and the combination of these two methods to accelerate the inference of semantic segmentation. Experiments show that our method greatly saves computing resources and retains high accuracy. On platforms with limited computing resources, it can well balance accuracy and speed.
References
[1]
Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C]. Proceedings of the IEEE conference on computer vision and pattern recognition, 2015: 3431-3440.
[2]
Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[C]. International Conference on Medical image computing and computer-assisted intervention, 2015: 234-241.
[3]
Chen L-C, Papandreou G, Kokkinos I, Semantic image segmentation with deep convolutional nets and fully connected crfs[J]. arXiv preprint arXiv:1412.7062, 2014.
[4]
Chen L-C, Papandreou G, Kokkinos I, Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 40(4): 834-848.
[5]
Chen L-C, Papandreou G, Schroff F, Rethinking atrous convolution for semantic image segmentation[J]. arXiv preprint arXiv:1706.05587, 2017.
[6]
Chen L-C, Zhu Y, Papandreou G, Encoder-decoder with atrous separable convolution for semantic image segmentation[C]. Proceedings of the European conference on computer vision (ECCV), 2018: 801-818.
[7]
Yu C, Wang J, Peng C, Bisenet: Bilateral segmentation network for real-time semantic segmentation[C]. Proceedings of the European conference on computer vision (ECCV), 2018: 325-341.
[8]
Yu C, Gao C, Wang J, Bisenet v2: Bilateral network with guided aggregation for real-time semantic segmentation[J]. International Journal of Computer Vision, 2021, 129(11): 3051-3068.
[9]
Hong Y, Pan H, Sun W, Deep dual-resolution networks for real-time and accurate semantic segmentation of road scenes[J]. arXiv preprint arXiv:2101.06085, 2021.
[10]
Sonali Bhadoria, Preeti Aggarwal, C. G. Dethe, and Renu Vig, "Comparison of Segmentation Tools for Multiple Modalities in Medical Imaging," Journal of Advances in Information Technology, Vol. 3, No. 4, pp. 197-205, November, 2012.
[11]
S.S. Kumar, R.S. Moni, and J. Rajeesh, "Automatic Segmentation of Liver and Tumor for CAD of Liver," Journal of Advances in Information Technology, Vol. 2, No. 1, pp. 63-70, February,2011.
[12]
Dosovitskiy A, Fischer P, Ilg E, Flownet: Learning optical flow with convolutional networks[C]. Proceedings of the IEEE international conference on computer vision, 2015: 2758-2766.
[13]
Ilg E, Mayer N, Saikia T, Flownet 2.0: Evolution of optical flow estimation with deep networks[C]. Proceedings of the IEEE conference on computer vision and pattern recognition, 2017: 2462-2470.
[14]
Hui T-W, Tang X, Loy C C. Liteflownet: A lightweight convolutional neural network for optical flow estimation[C]. Proceedings of the IEEE conference on computer vision and pattern recognition, 2018: 8981-8989.
[15]
Hui T-W, Tang X, Loy C C. A lightweight optical flow CNN—Revisiting data fidelity and regularization[J]. IEEE transactions on pattern analysis and machine intelligence, 2020, 43(8): 2555-2569.
[16]
Hui T-W, Loy C C. Liteflownet3: Resolving correspondence ambiguity for more accurate optical flow estimation[C]. European Conference on Computer Vision, 2020: 169-184.
[17]
Kong L, Shen C, Yang J. FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation[C]. 2021 IEEE International Conference on Robotics and Automation (ICRA), 2021: 10310-10316.
[18]
Greff K, Srivastava R K, Koutník J, LSTM: A search space odyssey[J]. IEEE transactions on neural networks and learning systems, 2016, 28(10): 2222-2232.
[19]
Fayyaz M, Saffar M H, Sabokrou M, STFCN: spatio-temporal FCN for semantic video segmentation[J]. arXiv preprint arXiv:1608.05971, 2016.
[20]
Gadde R, Jampani V, Gehler P V. Semantic video cnns through representation warping[C]. Proceedings of the IEEE International Conference on Computer Vision, 2017: 4453-4462.
[21]
Zhu X, Xiong Y, Dai J, Deep feature flow for video recognition[C]. Proceedings of the IEEE conference on computer vision and pattern recognition, 2017: 2349-2358.
[22]
Li Y, Shi J, Lin D. Low-latency video semantic segmentation[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 5997-6005.
[23]
Hu P, Caba F, Wang O, Temporally distributed networks for fast video semantic segmentation[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 8818-8827.
[24]
M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, “The Cityscapes Dataset for Semantic Urban Scene Understanding,” in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
Index Terms
- General Elastic Computing Acceleration Method for Semantic Segmentation
Recommendations
Provisioning Policies for Elastic Computing Environments
IPDPSW '12: Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops & PhD ForumResources experience dynamic load as demand fluctuates. Therefore, resource providers must estimate the appropriate amount of resources to purchase in order to meet variable user demand. With the relatively recent introduction of infrastructure-as-a-...
Semantic Computing, Cloud Computing, and Semantic Search Engine
ICSC '09: Proceedings of the 2009 IEEE International Conference on Semantic ComputingSemantic Computing extends Semantic Web both in breadth and depth. It bridges, and integrates, technologies such as software engineering, user interface, natural language processing, artificial intelligence, programming language, grid computing and ...
An improved U-Net method for the semantic segmentation of remote sensing images
AbstractForemost deep neural network models trained in natural scenes cannot transfer and apply to remote sensing image semantic segmentation well. Studies have shown that fine-tuning methods containing model fusion can alleviate this dilemma. In this ...
Comments
Information & Contributors
Information
Published In
March 2023
824 pages
ISBN:9781450399029
DOI:10.1145/3594315
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 02 August 2023
Check for updates
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICCAI 2023
ICCAI 2023: 2023 9th International Conference on Computing and Artificial Intelligence
March 17 - 20, 2023
Tianjin, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 21Total Downloads
- Downloads (Last 12 months)21
- Downloads (Last 6 weeks)0
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format