Clustering techniques are used for finding suitable groupings of samples belonging to a given set... more Clustering techniques are used for finding suitable groupings of samples belonging to a given set of data. There is no knowledge a priori about these data. Therefore, such set of samples cannot be considered as a training set, and classification techniques cannot be used in this case. The k-means algorithm is one of the most popular algorithms for clustering [103]. It is one of the most used algorithms for data mining, as it has been placed among the top 10 algorithms for data mining in [237]. The k-means algorithm partitions a set of data ...
Biclustering is a methodology allowing simultaneous partitioning of a set of samples and their fe... more Biclustering is a methodology allowing simultaneous partitioning of a set of samples and their features into classes. Samples and features classified together are supposed to have a high relevance to each other which can be observed by intensity of their expressions. The notion of consistency for biclustering is defined using interrelation between centroids of sample and feature classes. Consistent biclustering also implies separability of the classes by convex cones (see [Busygin et al.(2005)]). Previous works on biclustering concentrated ...
Global optimization has been expanding in all directions at an astonishing rate during the last f... more Global optimization has been expanding in all directions at an astonishing rate during the last few decades. Many new theoretical, algorithmic, and computational contributions of global optimization have been used to solve a wide spectrum of difficult problems in science and engineering. In particular, global optimization has recently emerged as a successful and versatile tool in many aspects of product and process design problems. The first part of the paper covers material regarding deterministic global optimization. We are going to focus on new algorithmic developments and some applications. In the second part of the paper we will outline some of the most basic stochastic techniques for global optimization and will present some elementary yet powerful approaches which might prove very useful for applications in the fields of process design, of innovative material research and of problems of biological interest; these problems share an enormous complexity which in most cases can b...
, except for brief excerpts in connection with reviews or scholarly analysis. Use in connection w... more , except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are to proprietary rights. not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject
This special issue of Optimization Letters presents selected, peer-reviewed, papers that were acc... more This special issue of Optimization Letters presents selected, peer-reviewed, papers that were accepted for presentation in the two recent International Conferences on Variable
The "No Free Lunch" theorem states that, averaged over all optimization problems, without re-samp... more The "No Free Lunch" theorem states that, averaged over all optimization problems, without re-sampling, all optimization algorithms perform equally well. Optimization, search, and supervised learning are the areas that have benefited more from this important theoretical concept. Formulation of the initial No Free Lunch theorem, very soon, gave rise to a number of research works which resulted in a suite of theorems that define an entire research field with significant results in other scientific areas where successfully exploring a search space is an essential and critical task. The objective of this paper is to go through the main research efforts that contributed to this research field, reveal the main issues, and disclose those points that are helpful in understanding the hypotheses, the restrictions, or even the inability of applying No Free Lunch theorems.
The Variable Neighborhood Search (VNS) metaheuristic is based on systematic changes in the neighb... more The Variable Neighborhood Search (VNS) metaheuristic is based on systematic changes in the neighborhood structure within a search. It has been successfully applied for the solution of various global and combinatorial optimization problems. The aim of this special issue of Journal of Global Optimization (JOGO) is to present some recent methodological developments in the field of Variable Neighborhood Search and also to publish emerging applications in this area. The VNS papers in this issue are linked to the 6th International Conference on Variable Neighborhood Search (ICVNS 2018), which was held in Sithonia, Halkidiki, Greece, during October 4-7, 2018. Each submission was peer reviewed by at least two referees, according to the editorial policy of JOGO.
A detailed review of the dynamic search trajectory methods for global optimization is given. In a... more A detailed review of the dynamic search trajectory methods for global optimization is given. In addition, a family of dynamic search trajectories methods that are created using numerical methods for solving autonomous ordinary differential equations is presented. Furthermore, a strategy for developing globally convergent methods that is applicable to the proposed family of methods is given and the corresponding theorem is proved. Finally, theoretical results for obtaining nonmonotone convergent methods that exploit the accumulated information with regard to the most recent values of the objective function are given.
Support Vector Machines (SVM) is one of the well known supervised classes of learning algorithms.... more Support Vector Machines (SVM) is one of the well known supervised classes of learning algorithms. SVM have wide applications to many fields in recent years and also many algorithmic and modeling variations. Basic SVM models are dealing with the situation where the exact values of the data points are known. This paper presents a survey of SVM when the data points are uncertain. When a direct model cannot guarantee a generally good performance on the uncertainty set, robust optimization is introduced to deal with the worst case scenario and still guarantee an optimal performance. The data uncertainty could be an additive noise which is bounded by norm, where some efficient linear programming models are presented under certain conditions; or could be intervals with support and extremum values; or a more general case of polyhedral uncertainties with formulations presented. Another field of the uncertainty analysis is chance constrained SVM which is used to ensure the small probability of misclassification for the uncertain data. The multivariate Chebyshev inequality and Bernstein bounding schemes have been used to transform the chance constraints through robust optimization. The Chebyshev based model employs moment information of the uncertain training points. The Bernstein bounds can be less conservative than the Chebyshev bounds since it employs both support and moment information, but it also makes a strong assumption that all the elements in the data set are independent.
Recently, high dimensional classification problems have been ubiquitous due to significant advanc... more Recently, high dimensional classification problems have been ubiquitous due to significant advances in technology. High dimensionality poses significant statistical chal-lenges and renders many traditional classification algorithms impractical to use. In this chapter, we present a comprehensive overview of different classifiers that have been highly successful in handling high dimensional data classification problems. We start with popular methods such as Support Vector Machines and variants of discrim-inant functions and discuss in detail their applications and modifications to several problems in high dimensional settings. We also examine regularization techniques and their integration to several existing algorithms. We then discuss more recent methods namely the hybrid classifiers and the ensemble classifiers. Feature selection techniques, as a part of hybrid classifiers, are introduced and their relative merits and drawbacks are examined. Lastly, we describe AdaBoost and Random ...
Aims and Scope Optimization has been expanding in all directions at an astonishing rate during th... more Aims and Scope Optimization has been expanding in all directions at an astonishing rate during the last few decades. New algorithmic and theoretical techniques have been developed, the diffusion into other disciplines has proceeded at a rapid pace, and our knowledge of all aspects of the field has grown even more profound. At the same time, one of the most striking trends in optimization is the constantly increasing emphasis on the interdisciplinary nature of the field. Optimization has been a basic tool in all areas of applied mathematics, engineering, medicine, economics, and other sciences. The series Springer Optimization and Its Applications publishes undergraduate and graduate textbooks, monographs and state-of-the-art expository work that focus on algorithms for solving optimization problems and also study applications involving such problems. Some of the topics covered include nonlinear optimization (convex and nonconvex), network flow problems, stochastic optimization, optimal control, discrete optimization, multi-objective programming, description of software packages, approximation techniques and heuristic approaches.
Quadratic Assignment Problems model many applications in diverse areas such as operations researc... more Quadratic Assignment Problems model many applications in diverse areas such as operations research, parallel and distributed computing, and combinatorial data analysis. In this paper we survey some of the most important techniques, applications, and methods regarding the quadratic assignment problem. We focus our attention on recent developments. 1 This paper and a separate bibliography le (bib le) is available by anonymous ftp at orion.uwaterloo.ca in the directory pub/henry/qap.
Optimization and Control of Bilinear Systems, 2008
In the last century, society was transformed by the conscious application of modern technologies ... more In the last century, society was transformed by the conscious application of modern technologies to engines of the industrial revolution. It is easy to predict that in the twenty-first century it will be quantum and biomolecular technologies that will influence all our lives. There is a sense in which quantum technology already has a profound effect (Lloyd, 1993). A large part of the national product of the industrialized countries stems from quantum mechanics. We are referring to transistors, the 'fundamental particle'of modern electronics. ...
► In biomedicine, binary classification is used for example in medical prognosis, to predict whet... more ► In biomedicine, binary classification is used for example in medical prognosis, to predict whether a patient will have a recurrence of the disease after a fixed time interval.► From a mathematical point of view, given a set of points Γ⊂ Rn, a binary classifier is a function: h (x): Γ⊆ Rn→ R, x∈ Γ, whose sign represents the class of the point x.
We give a brief overview of the methodology of modeling massive datasets arising in various appli... more We give a brief overview of the methodology of modeling massive datasets arising in various applications as networks. This approach is often useful for extracting non-trivial information from the datasets by applying standard graph-theoretic techniques. We also point out that graphs representing datasets coming from diverse practical fields have a similar power-law structure, which indicates that the global organization and evolution of massive datasets arising in various spheres of life nowadays follow similar natural principles.
During the last decade, many problems in social, biological, and financial networks require findi... more During the last decade, many problems in social, biological, and financial networks require finding cliques, or quasi-cliques. Cliques or clique partitions have also been used as clustering or classification tools in data sets represented by networks. These networks can be very large and often massive and therefore external (or semi-external) memory algorithms are needed. We discuss four applications where we identify computational challenges which are both of practical and theoretical interest.
Progressive preictal dynamical convergence and postictal divergence of dynamical EEG descriptors ... more Progressive preictal dynamical convergence and postictal divergence of dynamical EEG descriptors among brain regions has been reported in human temporal lobe epilepsy (TLE) and in a rodent model of TLE. There are also reports of anticonvulsant effects of high frequency stimulation of the hippocampus in humans. We postulate that this anticonvulsant effect is due to dynamical resetting by the electrical stimulation. The following study investigated the effects of acute hippocampal electrical stimulation on dynamical ...
Clustering techniques are used for finding suitable groupings of samples belonging to a given set... more Clustering techniques are used for finding suitable groupings of samples belonging to a given set of data. There is no knowledge a priori about these data. Therefore, such set of samples cannot be considered as a training set, and classification techniques cannot be used in this case. The k-means algorithm is one of the most popular algorithms for clustering [103]. It is one of the most used algorithms for data mining, as it has been placed among the top 10 algorithms for data mining in [237]. The k-means algorithm partitions a set of data ...
Biclustering is a methodology allowing simultaneous partitioning of a set of samples and their fe... more Biclustering is a methodology allowing simultaneous partitioning of a set of samples and their features into classes. Samples and features classified together are supposed to have a high relevance to each other which can be observed by intensity of their expressions. The notion of consistency for biclustering is defined using interrelation between centroids of sample and feature classes. Consistent biclustering also implies separability of the classes by convex cones (see [Busygin et al.(2005)]). Previous works on biclustering concentrated ...
Global optimization has been expanding in all directions at an astonishing rate during the last f... more Global optimization has been expanding in all directions at an astonishing rate during the last few decades. Many new theoretical, algorithmic, and computational contributions of global optimization have been used to solve a wide spectrum of difficult problems in science and engineering. In particular, global optimization has recently emerged as a successful and versatile tool in many aspects of product and process design problems. The first part of the paper covers material regarding deterministic global optimization. We are going to focus on new algorithmic developments and some applications. In the second part of the paper we will outline some of the most basic stochastic techniques for global optimization and will present some elementary yet powerful approaches which might prove very useful for applications in the fields of process design, of innovative material research and of problems of biological interest; these problems share an enormous complexity which in most cases can b...
, except for brief excerpts in connection with reviews or scholarly analysis. Use in connection w... more , except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are to proprietary rights. not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject
This special issue of Optimization Letters presents selected, peer-reviewed, papers that were acc... more This special issue of Optimization Letters presents selected, peer-reviewed, papers that were accepted for presentation in the two recent International Conferences on Variable
The "No Free Lunch" theorem states that, averaged over all optimization problems, without re-samp... more The "No Free Lunch" theorem states that, averaged over all optimization problems, without re-sampling, all optimization algorithms perform equally well. Optimization, search, and supervised learning are the areas that have benefited more from this important theoretical concept. Formulation of the initial No Free Lunch theorem, very soon, gave rise to a number of research works which resulted in a suite of theorems that define an entire research field with significant results in other scientific areas where successfully exploring a search space is an essential and critical task. The objective of this paper is to go through the main research efforts that contributed to this research field, reveal the main issues, and disclose those points that are helpful in understanding the hypotheses, the restrictions, or even the inability of applying No Free Lunch theorems.
The Variable Neighborhood Search (VNS) metaheuristic is based on systematic changes in the neighb... more The Variable Neighborhood Search (VNS) metaheuristic is based on systematic changes in the neighborhood structure within a search. It has been successfully applied for the solution of various global and combinatorial optimization problems. The aim of this special issue of Journal of Global Optimization (JOGO) is to present some recent methodological developments in the field of Variable Neighborhood Search and also to publish emerging applications in this area. The VNS papers in this issue are linked to the 6th International Conference on Variable Neighborhood Search (ICVNS 2018), which was held in Sithonia, Halkidiki, Greece, during October 4-7, 2018. Each submission was peer reviewed by at least two referees, according to the editorial policy of JOGO.
A detailed review of the dynamic search trajectory methods for global optimization is given. In a... more A detailed review of the dynamic search trajectory methods for global optimization is given. In addition, a family of dynamic search trajectories methods that are created using numerical methods for solving autonomous ordinary differential equations is presented. Furthermore, a strategy for developing globally convergent methods that is applicable to the proposed family of methods is given and the corresponding theorem is proved. Finally, theoretical results for obtaining nonmonotone convergent methods that exploit the accumulated information with regard to the most recent values of the objective function are given.
Support Vector Machines (SVM) is one of the well known supervised classes of learning algorithms.... more Support Vector Machines (SVM) is one of the well known supervised classes of learning algorithms. SVM have wide applications to many fields in recent years and also many algorithmic and modeling variations. Basic SVM models are dealing with the situation where the exact values of the data points are known. This paper presents a survey of SVM when the data points are uncertain. When a direct model cannot guarantee a generally good performance on the uncertainty set, robust optimization is introduced to deal with the worst case scenario and still guarantee an optimal performance. The data uncertainty could be an additive noise which is bounded by norm, where some efficient linear programming models are presented under certain conditions; or could be intervals with support and extremum values; or a more general case of polyhedral uncertainties with formulations presented. Another field of the uncertainty analysis is chance constrained SVM which is used to ensure the small probability of misclassification for the uncertain data. The multivariate Chebyshev inequality and Bernstein bounding schemes have been used to transform the chance constraints through robust optimization. The Chebyshev based model employs moment information of the uncertain training points. The Bernstein bounds can be less conservative than the Chebyshev bounds since it employs both support and moment information, but it also makes a strong assumption that all the elements in the data set are independent.
Recently, high dimensional classification problems have been ubiquitous due to significant advanc... more Recently, high dimensional classification problems have been ubiquitous due to significant advances in technology. High dimensionality poses significant statistical chal-lenges and renders many traditional classification algorithms impractical to use. In this chapter, we present a comprehensive overview of different classifiers that have been highly successful in handling high dimensional data classification problems. We start with popular methods such as Support Vector Machines and variants of discrim-inant functions and discuss in detail their applications and modifications to several problems in high dimensional settings. We also examine regularization techniques and their integration to several existing algorithms. We then discuss more recent methods namely the hybrid classifiers and the ensemble classifiers. Feature selection techniques, as a part of hybrid classifiers, are introduced and their relative merits and drawbacks are examined. Lastly, we describe AdaBoost and Random ...
Aims and Scope Optimization has been expanding in all directions at an astonishing rate during th... more Aims and Scope Optimization has been expanding in all directions at an astonishing rate during the last few decades. New algorithmic and theoretical techniques have been developed, the diffusion into other disciplines has proceeded at a rapid pace, and our knowledge of all aspects of the field has grown even more profound. At the same time, one of the most striking trends in optimization is the constantly increasing emphasis on the interdisciplinary nature of the field. Optimization has been a basic tool in all areas of applied mathematics, engineering, medicine, economics, and other sciences. The series Springer Optimization and Its Applications publishes undergraduate and graduate textbooks, monographs and state-of-the-art expository work that focus on algorithms for solving optimization problems and also study applications involving such problems. Some of the topics covered include nonlinear optimization (convex and nonconvex), network flow problems, stochastic optimization, optimal control, discrete optimization, multi-objective programming, description of software packages, approximation techniques and heuristic approaches.
Quadratic Assignment Problems model many applications in diverse areas such as operations researc... more Quadratic Assignment Problems model many applications in diverse areas such as operations research, parallel and distributed computing, and combinatorial data analysis. In this paper we survey some of the most important techniques, applications, and methods regarding the quadratic assignment problem. We focus our attention on recent developments. 1 This paper and a separate bibliography le (bib le) is available by anonymous ftp at orion.uwaterloo.ca in the directory pub/henry/qap.
Optimization and Control of Bilinear Systems, 2008
In the last century, society was transformed by the conscious application of modern technologies ... more In the last century, society was transformed by the conscious application of modern technologies to engines of the industrial revolution. It is easy to predict that in the twenty-first century it will be quantum and biomolecular technologies that will influence all our lives. There is a sense in which quantum technology already has a profound effect (Lloyd, 1993). A large part of the national product of the industrialized countries stems from quantum mechanics. We are referring to transistors, the 'fundamental particle'of modern electronics. ...
► In biomedicine, binary classification is used for example in medical prognosis, to predict whet... more ► In biomedicine, binary classification is used for example in medical prognosis, to predict whether a patient will have a recurrence of the disease after a fixed time interval.► From a mathematical point of view, given a set of points Γ⊂ Rn, a binary classifier is a function: h (x): Γ⊆ Rn→ R, x∈ Γ, whose sign represents the class of the point x.
We give a brief overview of the methodology of modeling massive datasets arising in various appli... more We give a brief overview of the methodology of modeling massive datasets arising in various applications as networks. This approach is often useful for extracting non-trivial information from the datasets by applying standard graph-theoretic techniques. We also point out that graphs representing datasets coming from diverse practical fields have a similar power-law structure, which indicates that the global organization and evolution of massive datasets arising in various spheres of life nowadays follow similar natural principles.
During the last decade, many problems in social, biological, and financial networks require findi... more During the last decade, many problems in social, biological, and financial networks require finding cliques, or quasi-cliques. Cliques or clique partitions have also been used as clustering or classification tools in data sets represented by networks. These networks can be very large and often massive and therefore external (or semi-external) memory algorithms are needed. We discuss four applications where we identify computational challenges which are both of practical and theoretical interest.
Progressive preictal dynamical convergence and postictal divergence of dynamical EEG descriptors ... more Progressive preictal dynamical convergence and postictal divergence of dynamical EEG descriptors among brain regions has been reported in human temporal lobe epilepsy (TLE) and in a rodent model of TLE. There are also reports of anticonvulsant effects of high frequency stimulation of the hippocampus in humans. We postulate that this anticonvulsant effect is due to dynamical resetting by the electrical stimulation. The following study investigated the effects of acute hippocampal electrical stimulation on dynamical ...
Uploads
Papers by Panos Pardalos