Evaluating network test scenarios for network simulators systems

Abstract

Networks continue to grow as industries use both wired and wireless networks. Creating experiments to test those networks can be very expensive if conducted on production networks; therefore, the evaluation of networks and their performance is usually conducted using emulation. This growing reliance on simulation raises the risk of correctness and validation. Today, many network simulators have widely varying focuses and are employed in different fields of research. The trustworthiness of results produced from simulation models must be investigated. The goal of this work is first to compare and assess the performance of three prominent network simulators—NS-2, NS-3, and OMNet++—by considering the following qualitative characteristics: architectural design, correctness, performance, usability, features, and trends. Second, introduce the concept of mutation testing to design the appropriate network scenarios to be used for protocol evaluation. Many works still doubt if used scenarios can suit well to claim conclusions about protocol performance and effectiveness. A large-scale simulation model was implemented using ad hoc on-demand distance vector and destination-sequenced distance vector routing protocols to compare performance, correctness, and usability. This study addresses an interesting question about the validation process: “Are you building the right simulation model in the right environment?” In conclusion, network simulation alone cannot determine the correctness and usefulness of the implemented protocol. Software testing approaches should be considered to validate the quality of the network model and test scenarios being used.