Residual Tsallis Entropy and Record Values: Some New Insights

The entropy measure proposed by Shannon [1] has many applications in various fields such as information science, physics, probability, statistics, communication theory, and economics. Let X represent the lifetime of a unit which has a cumulative distribution function (CDF) $K\left(x\right)$ and a probability density function (PDF) $k\left(x\right)$. The Shannon differential entropy of X is defined by $H\left(X\right)=-\mathbb{E}[logk(X\left)\right]$ if the expected value exists. This quantifies the uncertainty of a random phenomenon. The concept of the Tsallis entropy, initiated by Tsallis [2], see also [3], is a generalization of the Boltzmann–Gibbs statistic. Very recently, Tsallis and Borges [4] showed that, depending on the initial condition and the size of the time series, time reversal can enable the recovery, within a small error bar, of past information when the Lyapunov exponent is non-positive, notably at the Feigenbaum point (edge of chaos), where weak chaos is known to exist. The practical usefulness of time reversal has been very recently exhibited by decreasing error bars in the predictions of strong earthquakes [5,6]. For a non-negative continuous random variable (RV) X with PDF $k\left(x\right)$, the Tsallis entropy of order $\alpha$ is given by where $\omega \left(\alpha \right)=\frac{1}{1-\alpha }$ for all $\alpha >0,\alpha \ne 1,$ and where $K^{-1}\left(u\right)=inf\{x;K\left(x\right)\ge u\},$ for $u\in [0,1],$ denotes the quantile function. The Tsallis entropy can be negative for some values of $\alpha$, but it can also be non-negative if one chooses appropriate values for $\alpha$. The Tsallis entropy converges to the Shannon entropy when $\alpha$ approaches 1, i.e., $H\left(X\right)={lim}_{\alpha \to 1}{H}_{\alpha }\left(X\right)$. The Shannon differential entropy has the property of additivity, i.e., for RVs X and Y, which are independent and the entropy of their joint distribution is equal to the sum of their entropies, i.e., $H(X,Y)=H\left(X\right)+H\left(Y\right)$. However, the Tsallis entropy does not have this property and instead follows a non-additive rule given by $H_{\alpha }(X,Y)=H_{\alpha }\left(X\right)+H_{\alpha }\left(Y\right)+(1-\alpha )H_{\alpha }\left(X\right)H_{\alpha }\left(Y\right).$

The uncertainty in the lifetime of a new system can be measured by $H_{\alpha }\left(X\right)$, where X is the RV representing the lifetime. However, in some situations, operators already have some information about the age of the system. For example, they may know that the system is still functioning at time t and want to quantify the uncertainty in the remaining lifetime, i.e., $X_t=[X-t|X>t]$. In such cases, $H_{\alpha }\left(X\right)$ is not appropriate. Consider the PDF of $X_t,$ represented as $k_t\left(x\right)=k(x+t)/\overline{K}\left(t\right),$ where both x and t are positive. Here, $\overline{K}\left(t\right)=P(X>t)$ represents the survival function of X. Thus, the residual Tsallis entropy (RTE) is defined as where ${\overline{K}}_t^{-1}\left(u\right)=inf\{x;{\overline{K}}_t\left(x\right)\ge u\}$ is the quantile function of ${\overline{K}}_t\left(x\right)=\overline{K}(x+t)/\overline{K}\left(t\right),x,t>0.$ The characteristics, extensions, and uses of $H_{\alpha }(X;t)$ have been deeply researched by a number of scholars, including Asadi et al. [7], Nanda and Paul [8], and Zhang [9], along with further studies referenced in their work.

Records have applications in various fields, such as reliability engineering, insurance science, and others. An illustrative example can be found in the field of reliability theory. Consider a system involving k of n components to function successfully. Surprisingly, the lifetime of this system corresponds to the $(n-k+1)$-th order statistic from a sample of size n. As a result, we can interpret the n-th upper record value as the lifetime of a system that requires $(n-k+1)$ components to function properly. In other words, the nth upper record value represents the operating lifetime of a system that requires $(n-k+1)$ components to function successfully. In insurance science, the second or third highest values are relevant for some types of non-life insurance claims; see, e.g., Kamps [10] for more details. The information properties of record values have been studied by many researchers. The paper by Baratpour et al. [11] explores the information properties of record values using the Shannon differential entropy. Kumar [12] investigates the Tsallis entropy of k-record statistics in various continuous probability models and provides a characterization result for the Tsallis entropy of k-record values. Additionally, the mentioned paper examines the residual Tsallis entropy of k-record statistics in a summary context. Drawing on the information measures of record data obtained from independent and identically distributed continuous RVs, Ahmadi [13] offers new characterizations for continuous symmetric distributions based on the cumulative residual (past) entropy, Shannon entropy, Renyi entropy, Tsallis entropy, and common Kerridge inaccuracy measures. Xiong et al. [14] present the symmetric property of the extropy of record values and provide characterizations of exponential distributions. They also propose a new test for the symmetry of continuous distributions based on a characterization result, and the Monte Carlo simulation results investigate a wide range of alternative asymmetric distributions. In a recent study, Jose and Sathar [15] examine the residual extropy of k-records derived from any continuous distribution, relating it to the residual extropy of k-records derived from a uniform distribution. They establish lower bounds for the residual extropy of upper and lower k-records arising from any continuous probability distribution. Furthermore, they discuss the monotone property of the residual extropy for both upper and lower k-records. Further contributions in this area can be found in the following papers: Paul et al. [16], Cali et al. [17], Zamani et al. [18], Gupta et al. [19], Paula et al. [20], Zarezadeh [21], Baratpour et al. [22], and Qiu [23]. These papers, along with their references, provide additional insights and developments related to the topic.

This article is concerned with the study of the residual Tsallis entropy of upper data sets obtained from continuous distributions. The uniform distribution is chosen as a benchmark because of its simplicity and convenience in terms of its density function. It also serves as a versatile tool for modeling other distributions by applying appropriate transformations. Thus, by examining the residual Tsallis entropy of the upper data sets from the uniform distribution, we can gain insight into the entropy behavior of the upper data sets from any continuous distribution. Our main insight is that the RTE of upper data sets from any continuous distribution can be represented by the RTE of upper data sets from the uniform distribution over the interval $(0,1)$. To denote the uniform distribution over this interval, we use the notation $U(0,1)$.

In the present study, we will comprehensively investigate the residual Tsallis entropy of upper data sets obtained from continuous probability distributions. The following sections provide an overview of the structure of the paper and its main contributions: In Section 2, we first introduce the residual Tsallis entropy and application it provides regarding upper data sets derived from continuous probability distributions. We present a rigorous derivation of the expression for the RTE of upper data sets. Moreover, we establish a lower bound for this entropy measure that provides valuable insight into the minimum achievable entropy of upper data set values. Furthermore, we investigate how the RTE of data set changes in view of the aging behaviors of their components. Finally, we show the monotonic behavior of the residual Tsallis entropy of n-th upper record values as a function of $n.$ Section 3 provides an expression for the residual records based on the knowledge that all units consist of stresses exceeding $t>0.$ Understanding the monotonic property enhances our understanding of entropy dynamics and provides deeper insights into the system reliability and performance characteristics. In Section 4, we present a comprehensive summary of the overall results of our study. We highlight the main contributions, implications, and applications of the derived expressions, the lower bound, and the monotone property of residual Tsallis entropy.

We will use some notations throughout the paper. The order relations “${\le }_{st}$”, “${\le }_{hr}$”, “${\le }_{lr}$”, and “${\le }_d$” represent, respectively, the usual stochastic order, hazard rate order, likelihood ratio order, and dispersive order; for a detailed discussion of these stochastic orders, the reader may be referred to Shaked and Shanthikumar [24].

Let us consider a technical system that is subjected to shocks, such as voltage spikes. Then, the shocks can be modeled as a sequence of independent and identically distributed (i.i.d.) RVs $\{X_i,i\ge 1\}$, with a common continuous CDF K, PDF k, and survival function $\overline{K}\left(t\right)=1-K\left(t\right)$. The shocks represent the stresses on the system at different times. We are interested in the record statistics (the values of the highest stresses observed so far) of this sequence. Let us denote by $X_{i:n}$ the ith order statistics from the first n observations.

Then, we define the sequences of upper record times $T_n,(n\ge 1)$ and upper record values $U_n,$ respectively, as follows: where

It is well known that the PDF and the survival function of $U_n,$ denoted by $k_n\left(x\right)$ and ${\overline{K}}_n\left(x\right)$, respectively, are given by and where is known as the incomplete gamma function (see e.g., [25]). We use the notation $V\sim {\mathrm{\Gamma }}_t(a,b)$ to indicate that the RV V has a truncated Gamma distribution with the following PDF where $a>0$ and $b>0.$ In the residual part, we concentrate on examining the residual Tsallis entropy of the RV$X_{U_{\left(n\right)}}$, as it is considered a measure to quantify the uncertainty degree induced by the density of $[X_{U_{\left(n\right)}}-t|X_{U_{\left(n\right)}}>t]$ in terms of the system’s residual lifetime and its predictability. To facilitate the computations, we introduce a lemma that approves the RTE of order statistics from a uniform distribution that is linked to the incomplete beta function. This relationship is crucial from a practical point of view and allows for a more convenient computation of RTE. The proof of this lemma is omitted here since it involves simple calculations.

By leveraging this lemma, researchers and practitioners can readily compute the RTE of record values from a uniform distribution using the well-known incomplete gamma function. This computational simplification enhances the applicability and usability of the RTE in various contexts. In Figure 1, we present the plot of $H_{\alpha }(U_n^{★};t)$ for values of $\alpha =0.2$ and $\alpha =2$ and values of $n=2,\dots ,6.$

The upcoming theorem establishes a relationship between the RTE of record values $U_n$ and the RTE of record values from a uniform distribution.

Our analysis shows a significant decomposition of the remaining Tsallis entropy of the upper record values. In particular, we have shown that this entropy measure can be expressed as the product of two key components: the residual Tsallis entropy of the upper records from the uniform distribution and the expectation of a truncated gamma RV. From Equation (8), we can also see that the residual Tsallis entropy of the n-th upper data set value from an arbitrary continuous distribution F can be expressed by the residual Shannon entropy of the n-th upper data set value from $U(0,1)$ as follows: where $V_n\sim {\mathrm{\Gamma }}_{-log\overline{K}\left(t\right)}(n,1).$ The specialized version of this result for $t=0,$ was already obtained by Baratpour et al. [11].

Next, we examine how the residual Tsallis entropy of record values changes by referring to the aging aspects of the underlying distribution. The aging property of X affects the behavior of its residual Tsallis entropy of order $\alpha >0$. The forthcoming theorem is necessary to our aim. We recall that X has an increasing (decreasing) failure rate (IFR(DFR)) property if its hazard rate function $\lambda \left(t\right)=k\left(t\right)/\overline{K}\left(t\right)$ is an increasing (decreasing) function of $t>0.$

Now, we prove that the IFR property of X affects the behavior of the residual Tsallis entropy of record values.

We demonstrate how to use Theorems 3 and 6 with an example.

Now, we present a theorem that establishes a lower bound for the residual Tsallis entropy of upper records from any continuous distribution. The lower bound for the residual Tsallis entropy of upper records is influenced by two key factors: the residual Tsallis entropy of upper records from the uniform distribution on the interval $[0,1]$ and the mode of the original distribution.

We have presented a theorem that establishes a lower bound on the RTE of $U_n$, denoted $H_{\alpha }(U_n;t)$. This lower bound depends on the RTE of record values from a uniform distribution and the mode of PDF, denoted by M, of the original distribution. This result provides interesting insights into the information properties of $U_n$ and provides a measurable lower bound for the RTE with respect to the mode of the distribution. In Table 1, we show the lower bounds on the RTE of the record values for some common distributions based on Theorem 4.

In the following theorem, we show the monotonic behavior of the residual Tsallis entropy of n-th upper set values with respect to n. First, we need the following lemma.

Hereafter, we are interested in evaluating the residual records $X_{U\left(n\right)}-t,t\ge 0$ based on knowing the fact that all units are of voltages exceeding $t>0.$ It follows that the survival function of the $X_{U_n,t}^0=[X_{U\left(n\right)}-t|X_{U\left(0\right)}>t]$ can be written as (see [27]) and hence, we have

Hereafter, we will focus on studying the Tsallis entropy of the RV $X_{U_n,t}^0$ that measures the amount of uncertainty contained in the density of $[X_{U\left(n\right)}-t|X_{U\left(0\right)}>t],$ about the predictability of the system’s residual lifetime in terms of the Tsallis entropy. The probability integral transformation $V={\overline{K}}_t\left(X_{U_n,t}^0\right)$ plays a crucial role in our aim. It is clear that $U_n^{★}={\overline{K}}_t\left(X_{U_n,t}^0\right)$ had the pdf as

In the forthcoming proposition, we provide an expression for the Tsallis entropy of $X_{U_n,t}^0$ by using the earlier mentioned transforms.

In the next theorem, we investigate how the residual Tsallis entropy of record values changes with respect to the aging properties of their components.

In the following example, we provide an illustration of the results presented in Theorems 6 and 7.

In Theorems 8 and 9, we have demonstrated that the aforementioned equality holds true in the case of exponentially distributed component lifetimes. This highlights the significance of the exponential distribution in the context of record values, information theory, and reliability.

Considering the uncertainties of two records, here we discuss the partial ordering of residual records based on knowing the fact that all units are of voltages exceeding $t>0.$ The next theorem compares the residual Tsallis entropies of two records.

In this paper, we explored the concept of RTE for datasets. We presented a new approach to express RTE of dataset values from a continuous distribution in terms of RTE of dataset values from a uniform distribution. This connection provides valuable insight into the properties and behavior of RTE for different distributions. Since it is difficult to obtain closed-form expressions for the RTE of dataset values, we have derived bounds that provide practical approximations and allow for a better understanding of their properties. These bounds serve as useful tools for analyzing and comparing RTE values in different scenarios. To validate our results and demonstrate the applicability of our approach, we have provided illustrative examples. These examples show the practical implications of RTE for record values and illustrate the versatility of our methodology for different distributions. In summary, this study contributes to the understanding of RTE for record values by establishing relationships, deriving bounds, and demonstrating stochastic orderings. The results of this work provide valuable insights for researchers and practitioners working in the field of statistical inference and entropy-based analysis. For example, the results of this paper may be have practical implications in the field of goodness-of-fit tests. In particular, it can be applied to test the null hypothesis $H_0:F\left(x\right)=F_0\left(x\right)$ for all $x\ge 0,$ against the alternative hypothesis $H_1:F\left(x\right)\ne F_0\left(x\right)$ for some $x\ge 0.$ Here, $F_0\left(x\right)=1-e^{-\lambda x}$ represents an exponential distribution with $\lambda >0$ and $x\ge 0.$ According to (8), the above-mentioned goodness-of-fit problem is equivalent to testing the null hypothesis for all $t\ge 0,$ against the alternative hypothesis for some $t\ge 0.$ Here, $H_{\alpha }(U_n;t)$ represents a specific statistical measure. To perform this test, the maximum likelihood estimate of the parameter $\lambda$ is $1/\overline{x}=k/{\sum }_{i=1}^n{x}_i.$ If a reliable estimate for $H_{\alpha }(U_n;t)$, denoted as ${\widehat{H}}_{\alpha }(U_n;t)$, is available, significant deviations of from zero indicate non-exponential behavior. Consequently, the null hypothesis can be rejected. We note that Equation (8) can be rewritten as follows: where $I(u\ge F(t\left)\right)$ denotes the indicator function for $(u\ge F(t\left)\right).$ Following Park [29], a sample estimate of $H_{\alpha }(U_n;t)$ can be constructed based on a sample of size k as where $m\le k/2$ is a positive integer known as the window size, $\widehat{K}$ represents the empirical distribution function of $X,$ and $X_{i:k}=X_{1:k}$ if $i<1,$ and $X_{i:k}=X_{k:k}$ if $i>k.$