Skip to main content

Commonly Used Metrics for Performance Evaluation


The following metrics are commonly used when evaluating scenarios related to DTN protocols.
  • Delivery ratio of the messages,
  • Average message delivery latency
  • Overhead ratio (of the underlying routing mechanism)
Suppose that $M$ be the set of all messages created in the network and $M_d$ be the set of all messages delivered. Then, the delivery ratio is computed as $|M_d| / |M|$.

Now let the $i^{th}$ delivered message was created at time $c_i$ and delivered at time $d_i$. Then the average message delivery latency is computed as $(\sum_{i = 1}^{|M_d|} (d_i - c_i)) / |M_d|$. Note that, in Statistics, mean, median and mode are all the measures of average. But "loosely speaking", unless otherwise specified, we refer to the "mean" value when we say "average." Nevertheless, the MessageStatsReport in the ONE simulator provides a measure of both the mean and median values wherever appropriate.

One may refer the above metric as "end-to-end delay." Personally, I think such usage is inappropriate given that, by definition, DTNs typically lack end-to-end paths.

Finally, let $r_i$ be the number of replications of any message $m_i \in M$. Then the overhead ratio is determined as $(\sum_{i = 1}^{|M|} r_i - |M_d|) / M_d$.

The above definitions are generic. However, if you feel that they are helpful and wish to incorporate into your research article, you may consider citing it as:

B. K. Saha (2014, Mar.) Commonly Used Metrics for Performance Evaluation. Accessed: DD Mon. YYYY. [Online]. Available: http://delay-tolerant-networks.blogspot.com/2014/03/commonly-used-metrics.html

Comments

  1. Best Casino, Slots & Other Games for Real Money at the Casino
    If you're looking for a casino to start 안성 출장마사지 a gambling 제주 출장샵 journey, you've come to 동두천 출장샵 the right place. At 김포 출장안마 CasinoGround, you'll 구리 출장샵 find over

    ReplyDelete

Post a Comment

Popular posts from this blog

Text Highlighting in Latex

While preparing a manuscript with Latex, it is often useful to highlight the changes made in the current revision with a different color. This can be achieved using the \ textcolor command provided by Latex. For example, \textcolor {red}{Hello World} would display the string "Hello World" in red color. However, the final/published copy of the manuscript does not contain any highlighted text. Therefore, if a large volume of changes were made, it becomes tiresome at the end to find and remove all the individual portions of highlighted text. This can be circumvented by defining a utility command to switch highlighting on and off as desired. In the following, we define a new Latex command, highlighttext , for this purpose. The command takes only a single argument—the text to be highlighted.     \usepackage {color}    % For highlighting changes in this version with red color   \newcommand { \highlighttext }[1] { \textcolor {red}{#1}}   % Remove all text highlighting

The ONE Knowledge Base

The Opportunistic Network Environment (ONE) simulator has become largely popular and widely used in research on DTNs and Opportunistic Networks. The simulator has an associated friendly community, where users can ask questions and get their doubts clarified. Indeed, the community is vibrant and its members are useful. However, there are two issues with this community. First, the archives of the community's mailing list are not public. This is unlike many other similar lists hosted using Mailman, which are public and, therefore, can be searched using Google. Hence, time and again people have asked the question -- how to search the email archives? The second issue is largely due to the first. Due to the lack of searching facility over the archives, often same question tend to be asked by the new users repeatedly. People here are patient enough to reply to them. But such necessity could be entirely discounted had there been a facility to search the archives. In particular, it w

The ONE KB has a new home

The ONE Knowledge Base is now hosted at http://theonekb.pythonanywhere.com/ If you are unaware, the ONE KB allows you to search the old email archives of the simulator's community. Therefore, if you have any question related to simulation, you may query the existing database at the above link. Chances are good that your question might already have been answered previously. If not, you can still post an email to the community's mailing list. Have you tried the ONE KB already? How was your experience? Was it helpful? Let me know in the comments!