Third, we have analyzed the tradeoff between delay caused by filtering of worms at routers, and the delay due to worms' excessive amount of network traffic. We have used the optimal control problem, to determine the appropriate tradeoffs between these two delays for a given rate of a worm spreading. Using our technique we can minimize the overall network delay by finding the number of routers that should perform filtering and the time at which they should start the filtering process.

Many early Internet protocols were designed without a fundamentally secure infrastructure and hence vulnerable to attacks such as denial of service (DoS) attacks and worms. DoS attacks attempt to consume the resources of a remote host or network, thereby denying or degrading service to legitimate users. Network forensics is an emerging area wherein the source or the cause of the attacker is determined using IDS tools. The problem of finding the source(s) of attack(s) is called the "trace back problem". Lately, Internet worms have become a major problem for the security of computer networks, causing considerable amount of resources and time to be spent recovering from the disruption of systems. In addition to breaking down victims, these worms create large amounts of unnecessary network data traffic that results in network congestion, thereby affecting the entire network.

In this dissertation, first we solve the trace back problem more efficiently in terms of the number of routers needed to complete the track back. We provide an efficient algorithm to decompose a network into connected components and construct a terminal network. We show that for a terminal network with n routers, the trace back can be completed in O(log n) steps.

Second, we apply two classical epidemic SIS and SIR models to study the spread of Internet Worm. The analytical models that we provide are useful in determining the rate of spread and time required to infect a majority of the nodes in the network. Our simulation results on large Internet like topologies show that in a fairly small amount of time, 80% of the network nodes is infected.