Critical issues in mapping neural networks on message-passing multicomputers
Authors: 
J. Ghosh, K. HwangAuthors Info & Claims
J. Ghosh, K. HwangAuthors Info & ClaimsPages 3 - 11
Abstract
Connectionist models such as artificial neural systems, offer an intrinsically concurrent computational paradigm. We investigate the architectural requirements for efficiently simulating large neural networks on a multicomputer system with thousands of fine-grained processors and distributed memory. First, models for characterizing the structure of a neural network and the function of individual cells are developed. These models provide guidelines for efficiently mapping the network onto multicomputer topologies such as the hypercube, hypernet and torus. They are further used to estimate the amount of interprocessor communication bandwidth required, and the number of processors needed to meet a particular cost/performance goal. Design issues such as memory organization and the effect of VLSI technology are also considered.
References
[1]
J. A. Feldman and D. H. Ballard, "Connectionist models and their properties," Cognitive Science, vol. 6, pp. 2()5- 254, 1982.
[2]
D. E. Bumelhart and J. L. McClelland, eds., ParaLfef histributed Processing: Ezplorations in the Microstructure of Cognition. Cambridge, Mass: Bradford Books/MIT Press, 1986.
[3]
S. E. Fahlman and G. E. Hinton, "Connectionist architectures for artificial intelligence," IEEE Computer, pp. lOO- 109, Jan. 1987.
[4]
D. Hammerstrom, D. Maier, and S. Thakkar, "The cognitive architecture project," Computer Architecture News, vol. 14, pp. 9-21,1986.
[5]
J. Ghosh and K. Hwang, "Mapping neural networks onto highly parallel multiprocessors," Tech. Rep. CRI-87-65, Computer Research Institute, Univ. of Southern California, Los Angeles, 1987.
[6]
J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities," Proceedings National Acadamy of Science, vol. 79, pp. 2554-2558, Apr. 1982.
[7]
M. A. Cohen and S. Grossberg, "Absolute stability of global pattern formation and parallel memory storage by competitive neural networks," IEEE Trans. on Systems, Man and Cybernetics, vol. SMC-13, no. 5, pp. 815-826, Sept./Ott. 1983.
[8]
B. Kernighan and S. Lin, "An efficient heuristic procedure for partitioning graphs," Bell Systems Tech. Jl., vol. 49, no. 2, pp. 291-307,197l.
[9]
S. Amari and M. A. Arbib, eds., Competition and Cooperation in Neural Nets, Springer-Verlag, 1982.
[10]
R. Hecht-Nielsen, "Performance limits of optical, electrooptical, and electronics neurocomputers," Proc. SPIE, vol. 634, pp. 277-306,1986.
[11]
S. Toborg and K. Hwang, UExploring neural network and optical computing technologies," in Parallel Processing for Supercomputing and Artificial Intelligence, (K. Hwang and D. DeGroot, eds.), McGraw Hill, 1988.
[12]
J. A. Feldman, M. A. Fanty, N. H. Goddard, and K. J. Lynne, "Computing with structured connectionist networks," Comm. of ACM, vol. 31, no. 2, pp. 170-187,1988.
[13]
W. D. Hillis, The Connection Machine. Cambridge, Mass.: The MIT Press, 1985.
[14]
D. P. Agrawal, V. K. Janakiram, and G. C. Pathak, "Evaluating the performance of multicomputer configurations," IEEE Computer, vol. 19, pp. 23-37, May 1986.
[15]
K. Hwang and J. Ghosh, "Hypernet: a communicationefficient architecture for constructing massively parallel computers," IEEE Trans. Computers, vol. C-36, pp. 1450-1466, Dec. 1987.
[16]
F. P. Preparata and Vuillemin, "The cube-connected cycles: a versatile network for parallel computations," Comm. of ACM, pp. 300-309, May 1981.
[17]
C. H. Sequin, "Doubly twisted torus networks for VLSI proceessor arrays, n in Proc. 8th An. Symp. Computer Architecture, pp. 471-480, 1981.
[18]
C. T. Ho and S. L. Johnsson, "Distributed routing algorithms for broadcasting and personalized communication in hypercubes, in Proc. Int'l Conf. on Parallel Processing, pp. 640-648, Aug. 1986.
[19]
D. A. Reed and R. M. Fujimoto, Multicomputer Networks: Message-Based Parallel Processing. The MIT Press, 1987.
[20]
A. M. Despain and D. A. Patterson, "X-tree: a tree structured multiprocessor computer architecture," Proc. 5th Ann. Symp. on Computer Arch., pp. 144-151, Aug. 1978.
[21]
R. D. Geller and D. W. Hammerstrom, "A VLSI architecture for a neurocomputer using high-order predicates," in Workshop on Computer Architecture for Pattern Analysis and Machine Intelligence, pp. 153-161, 1987.
[22]
J. Ghosh and K. Hwang, "Optically connected multiprocessors for simulating artificial neural networks," in SPIE Proceedings vol. 882, Jan. 1988.
[23]
S. I. Gallant, "Connectionst expert systems," Comm. of ACM, vol. 31, no. 2, pp. 152-169, 1988.
Index Terms
- Critical issues in mapping neural networks on message-passing multicomputers
Recommendations
Critical issues in mapping neural networks on message-passing multicomputers
ISCA '88: Proceedings of the 15th Annual International Symposium on Computer architectureConnectionist models such as artificial neural systems, offer an intrinsically concurrent computational paradigm. We investigate the architectural requirements for efficiently simulating large neural networks on a multicomputer system with thousands of ...
Mapping of artificial neural networks onto message passing systems
Various Artificial Neural Networks (ANNs) have been proposed in recent years to mimic the human brain in solving problems involving human-like intelligence. Efficient mapping of ANNs comprising of large number of neurons onto various distributed MIMD ...
Comments
Information & Contributors
Information
Published In
Copyright © 1988 Authors.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 17 May 1988
Published in SIGARCH Volume 16, Issue 2
Check for updates
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 489Total Downloads
- Downloads (Last 12 months)37
- Downloads (Last 6 weeks)9
Reflects downloads up to 11 Aug 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in