Skip to main content

Commonly Used Metrics for Performance Evaluation


The following metrics are commonly used when evaluating scenarios related to DTN protocols.
  • Delivery ratio of the messages,
  • Average message delivery latency
  • Overhead ratio (of the underlying routing mechanism)
Suppose that $M$ be the set of all messages created in the network and $M_d$ be the set of all messages delivered. Then, the delivery ratio is computed as $|M_d| / |M|$.

Now let the $i^{th}$ delivered message was created at time $c_i$ and delivered at time $d_i$. Then the average message delivery latency is computed as $(\sum_{i = 1}^{|M_d|} (d_i - c_i)) / |M_d|$. Note that, in Statistics, mean, median and mode are all the measures of average. But "loosely speaking", unless otherwise specified, we refer to the "mean" value when we say "average." Nevertheless, the MessageStatsReport in the ONE simulator provides a measure of both the mean and median values wherever appropriate.

One may refer the above metric as "end-to-end delay." Personally, I think such usage is inappropriate given that, by definition, DTNs typically lack end-to-end paths.

Finally, let $r_i$ be the number of replications of any message $m_i \in M$. Then the overhead ratio is determined as $(\sum_{i = 1}^{|M|} r_i - |M_d|) / M_d$.

The above definitions are generic. However, if you feel that they are helpful and wish to incorporate into your research article, you may consider citing it as:

B. K. Saha (2014, Mar.) Commonly Used Metrics for Performance Evaluation. Accessed: DD Mon. YYYY. [Online]. Available: http://delay-tolerant-networks.blogspot.com/2014/03/commonly-used-metrics.html
Labels:

Comments

  1. vaylenagyMarch 4, 2022 at 9:39 AM

    Best Casino, Slots & Other Games for Real Money at the Casino
    If you're looking for a casino to start 안성 출장마사지 a gambling 제주 출장샵 journey, you've come to 동두천 출장샵 the right place. At 김포 출장안마 CasinoGround, you'll 구리 출장샵 find over

    ReplyDelete

Post a Comment

Popular posts from this blog

Text Highlighting in Latex

While preparing a manuscript with Latex, it is often useful to highlight the changes made in the current revision with a different color. This can be achieved using the \ textcolor command provided by Latex. For example, \textcolor {red}{Hello World} would display the string "Hello World" in red color. However, the final/published copy of the manuscript does not contain any highlighted text. Therefore, if a large volume of changes were made, it becomes tiresome at the end to find and remove all the individual portions of highlighted text. This can be circumvented by defining a utility command to switch highlighting on and off as desired. In the following, we define a new Latex command, highlighttext , for this purpose. The command takes only a single argument—the text to be highlighted.     \usepackage {color}    % For highlighting changes in this version with red color   \newcommand { \highlighttext }[1] { \textcolor {red}{#1}}   % Remove...
1 comment
Read more

Cohere Aya Dataset: Exploring the Split-by-language Collection

A snapshot of the Aya collection (Bengali) . Image taken from HuggingFace. In February 2024, Cohere launched Aya , a multilingual Large Language Model (LLM). Alongside, a set of datasets used to train Aya has also been released. For example, the aya_dataset consists around 205K examples annotated by humans. On the other hand, the recently released aya_collection_language_split is a gigantic dataset with more than 500 million data points spread across more than 100 languages. As the name suggests, this dataset is split by language. For example, all data points in Bengali, irrespective of the underlying task, can be found in a single split. Apart from the original human-annotated examples from the aya_dataset, aya_collection_language_split also contains a lot of translated and templated data. The dataset is released using an Apache-2.0 license, allowing academic and commercial use. The Bengali Language Split Each language split in the Aya collection has three splits. The Bengali split,...
Post a Comment
Read more

Specifying Source and Destination of Messages

One of the frequently asked questions in the community is how to specify which particular nodes would act as source(s) and destination(s) of the messages created in the ONE simulator. The simulator, in fact, provides a pair of settings (shown below in bold face) aimed for this particular purpose. Let us consider that there are $n + 1$ nodes in an OMN.  Further, let the nodes with addresses from $x$ to $y$, both inclusive, would create messages. The nodes in the range $w$ to $z$, both inclusive, would be the destinations of those messages, where $0 \le x \le y \le n$, and $0 \le w \le z \le n$. Then, the corresponding simulation scenario can be configured as follows. ## Message creation parameters # How many event generators Events.nrof = 1 # Class of the first event generator Events1.class = MessageEventGenerator # (Following settings are specific for the MessageEventGenerator class) # Creation interval in seconds (one new message every 25 to 35 seconds) Events1.interval ...
20 comments
Read more