The crux of contemporary network design involves mathematical models to describe the quantitative impact of system components on overall performance. However, these models often grapple with trade-offs between accuracy and complexity, and the burgeoning complexity of wireless networks compounds the problem. Artificial Intelligence (AI) and Machine Learning (ML) techniques present promising solutions, albeit with their own challenges, including integration of domain knowledge, sparsity of training data, and network environment dynamism. To address these challenges, the major contribution of this dissertation is to develop novel AI-based network behavior models, which are domain-aware, interpretable, and robust to sparsity and dynamicity.

It first proposes an interpretable AI/ML-based framework to create a large-scale 3D propagation model by leveraging domain knowledge to create a novel set of key predictors (features) that can characterize the physical and geometric structure of the environment traversed by a signal in its propagation path.

Subsequently, the dissertation introduces a novel deep transfer learning framework to create network behavior models when the available network data is extremely sparse and unrepresentative, especially for system-level network behavior modeling. By leveraging network data from similar (source) cells, the proposed novel framework boosts the performance of conventional deep transfer learning by harnessing the strengths of deep neural networks and extreme gradient boosting methods.

However, in a dynamic network environment (with continuously changing user mobility, traffic demand, etc.), selection of source cells with a similar network environment becomes challenging and can hinder the performance of transfer learning. Thus, the dissertation further presents a novel AI-based domain-aware cell similarity prediction framework, designed to identify similar cells in a spatio-temporally dynamic environment to enable transfer learning for cellular network modeling.

In conclusion, the dissertation paves the way for a future of zero-touch automation solutions in network design and optimization, using AI and ML, which could yield immense economic benefits for network operators and better quality of experience for users. It encapsulates novel frameworks for network behavior modeling and paves the way for future research in this field.