Skip to main content

Commonly Used Metrics for Performance Evaluation


The following metrics are commonly used when evaluating scenarios related to DTN protocols.
  • Delivery ratio of the messages,
  • Average message delivery latency
  • Overhead ratio (of the underlying routing mechanism)
Suppose that $M$ be the set of all messages created in the network and $M_d$ be the set of all messages delivered. Then, the delivery ratio is computed as $|M_d| / |M|$.

Now let the $i^{th}$ delivered message was created at time $c_i$ and delivered at time $d_i$. Then the average message delivery latency is computed as $(\sum_{i = 1}^{|M_d|} (d_i - c_i)) / |M_d|$. Note that, in Statistics, mean, median and mode are all the measures of average. But "loosely speaking", unless otherwise specified, we refer to the "mean" value when we say "average." Nevertheless, the MessageStatsReport in the ONE simulator provides a measure of both the mean and median values wherever appropriate.

One may refer the above metric as "end-to-end delay." Personally, I think such usage is inappropriate given that, by definition, DTNs typically lack end-to-end paths.

Finally, let $r_i$ be the number of replications of any message $m_i \in M$. Then the overhead ratio is determined as $(\sum_{i = 1}^{|M|} r_i - |M_d|) / M_d$.

The above definitions are generic. However, if you feel that they are helpful and wish to incorporate into your research article, you may consider citing it as:

B. K. Saha (2014, Mar.) Commonly Used Metrics for Performance Evaluation. Accessed: DD Mon. YYYY. [Online]. Available: http://delay-tolerant-networks.blogspot.com/2014/03/commonly-used-metrics.html

Comments

  1. Best Casino, Slots & Other Games for Real Money at the Casino
    If you're looking for a casino to start 안성 출장마사지 a gambling 제주 출장샵 journey, you've come to 동두천 출장샵 the right place. At 김포 출장안마 CasinoGround, you'll 구리 출장샵 find over

    ReplyDelete

Post a Comment

Popular posts from this blog

Text Highlighting in Latex

While preparing a manuscript with Latex, it is often useful to highlight the changes made in the current revision with a different color. This can be achieved using the \ textcolor command provided by Latex. For example, \textcolor {red}{Hello World} would display the string "Hello World" in red color. However, the final/published copy of the manuscript does not contain any highlighted text. Therefore, if a large volume of changes were made, it becomes tiresome at the end to find and remove all the individual portions of highlighted text. This can be circumvented by defining a utility command to switch highlighting on and off as desired. In the following, we define a new Latex command, highlighttext , for this purpose. The command takes only a single argument—the text to be highlighted.     \usepackage {color}    % For highlighting changes in this version with red color   \newcommand { \highlighttext }[1] { \textcolor {red}{#1}}   % Remove all text highlighting

The ONE KB has a new home

The ONE Knowledge Base is now hosted at http://theonekb.pythonanywhere.com/ If you are unaware, the ONE KB allows you to search the old email archives of the simulator's community. Therefore, if you have any question related to simulation, you may query the existing database at the above link. Chances are good that your question might already have been answered previously. If not, you can still post an email to the community's mailing list. Have you tried the ONE KB already? How was your experience? Was it helpful? Let me know in the comments!

Specifying Source and Destination of Messages

One of the frequently asked questions in the community is how to specify which particular nodes would act as source(s) and destination(s) of the messages created in the ONE simulator. The simulator, in fact, provides a pair of settings (shown below in bold face) aimed for this particular purpose. Let us consider that there are $n + 1$ nodes in an OMN.  Further, let the nodes with addresses from $x$ to $y$, both inclusive, would create messages. The nodes in the range $w$ to $z$, both inclusive, would be the destinations of those messages, where $0 \le x \le y \le n$, and $0 \le w \le z \le n$. Then, the corresponding simulation scenario can be configured as follows. ## Message creation parameters # How many event generators Events.nrof = 1 # Class of the first event generator Events1.class = MessageEventGenerator # (Following settings are specific for the MessageEventGenerator class) # Creation interval in seconds (one new message every 25 to 35 seconds) Events1.interval = 25