Commonly Used Metrics for Performance Evaluation

The following metrics are commonly used when evaluating scenarios related to DTN protocols.

• Delivery ratio of the messages,
• Average message delivery latency
• Overhead ratio (of the underlying routing mechanism)

Suppose that $M$ be the set of all messages created in the network and $M_d$ be the set of all messages delivered. Then, the delivery ratio is computed as $|M_d| / |M|$.

Now let the $i^{th}$ delivered message was created at time $c_i$ and delivered at time $d_i$. Then the average message delivery latency is computed as $(\sum_{i = 1}^{|M_d|} (d_i - c_i)) / |M_d|$. Note that, in Statistics, mean, median and mode are all the measures of average. But "loosely speaking", unless otherwise specified, we refer to the "mean" value when we say "average." Nevertheless, the MessageStatsReport in the ONE simulator provides a measure of both the mean and median values wherever appropriate.

One may refer the above metric as "end-to-end delay." Personally, I think such usage is inappropriate given that, by definition, DTNs typically lack end-to-end paths.

Finally, let $r_i$ be the number of replications of any message $m_i \in M$. Then the overhead ratio is determined as $(\sum_{i = 1}^{|M|} r_i - |M_d|) / M_d$.

The above definitions are generic. However, if you feel that they are helpful and wish to incorporate into your research article, you may consider citing it as:

B. K. Saha (2014, Mar.) Commonly Used Metrics for Performance Evaluation. Accessed: DD Mon. YYYY. [Online]. Available: http://delay-tolerant-networks.blogspot.com/2014/03/commonly-used-metrics.html