With the rapid development of the fifthgeneration wireless communication systems, a profound revolution in terms of transmission capacity, energy efficiency, reliability, latency, and connectivity is highly expected to support a new batch of industries and applications. To achieve this goal, wireless networks are becoming extremely dynamic, heterogeneous, and complex. The modeling and optimization for the performance of realworld wireless networks are extremely challenging due to the difficulty to predict the network performance as a function of network parameters, and the prohibitively huge number of parameters to optimize. The conventional network modeling and optimization approaches rely on drive test, trial-and-error, and engineering experience, which are labor intensive, error-prone, and far from optimal. On the other hand, while the research community has spent significant efforts in understanding the fundamental limits of radio channels and developing physical layer techniques to operate close to it, very little is known about the performance limits of wireless networks, where millions of radio channels interact with one another in complex manners. This paper reviews the very recent mathematical and learning based techniques for modeling and optimizing the performance of real-world wireless networks in five aspects, including channel modeling, user demand and traffic modeling, throughput modeling and prediction, network parameter optimization, and IRS empowered performance optimization, and also presents the corresponding notable performance gains.