This paper develops a new algorithm for inducing cost-sensitive decision trees that is inspired b... more This paper develops a new algorithm for inducing cost-sensitive decision trees that is inspired by the multi-armed bandit problem, in which a player in a casino has to decide which slot machine (bandit) from a selection of slot machines is likely to pay out the most. Game Theory proposes a solution to this multi-armed bandit problem by using a process of exploration and exploitation in which reward is maximized. This paper utilizes these concepts to develop a new algorithm by viewing the rewards as a reduction in costs, and utilizing the exploration and exploitation techniques so that a compromise between decisions based on accuracy and decisions based on costs can be found. The algorithm employs the notion of lever pulls in the multi-armed bandit game to select the attributes during decision tree induction, using a lookahead methodology to explore potential attributes and exploit the attributes which maximizes the reward. The new algorithm is evaluated on fifteen datasets and compared to six wellknown algorithms J48, EG2, MetaCost, AdaCostM1, ICET and ACT. The results obtained show that the new multi-armed based algorithm can produce more cost-effective trees without compromising accuracy. The paper also includes a critical appraisal of the limitations of the new algorithm and proposes avenues for further research.
IoT sensors and deep learning models can widely be applied for fault prediction. Although deep le... more IoT sensors and deep learning models can widely be applied for fault prediction. Although deep learning models are considerably more potent than many conventional machine learning models, they are not transparent. This paper first examines different deep learning techniques to carry out univariate time series analysis based on vibration sensors installed on four industrial bearings to predict a fault occurring in a predefined time window. Several recurrent neural networks are used to develop fault prediction models. An empirical evaluation of these models shows that all models perform well; however, hybrid models outperform other models when the time window increases. Then, instance-wise feature selection has been considered to highlight the most contributing features for its outputs regarding any input. In this problem, the main challenge is to propose a trainable feature selection model with the minimum number of selected features whilst its performance is close to the baseline model. This paper develops a novel explainable method called the Gumbel-Sigmoid eXplanator (GSX) to tackle these problems. In a nutshell: (i) we have developed a differentiable and trainable selector, and (ii) we utilize regularization to control the number of features for each instance flexibly. The proposed method is model agnostic, and empirical evaluations on two datasets show that GSX can not only solve the problems identified with two other state-of-the-art methods but also outperform them in terms of accuracy and run-time.
This paper presents a survey of methods for pruning deep neural networks. It begins by categorisi... more This paper presents a survey of methods for pruning deep neural networks. It begins by categorising over 150 studies based on the underlying approach used and then focuses on three categories: methods that use magnitude based pruning, methods that utilise clustering to identify redundancy, and methods that use sensitivity analysis to assess the effect of pruning. Some of the key influencing studies within these categories are presented to highlight the underlying approaches and results achieved. Most studies present results which are distributed in the literature as new architectures, algorithms and data sets have developed with time, making comparison across different studied difficult. The paper therefore provides a resource for the community that can be used to quickly compare the results from many different methods on a variety of data sets, and a range of architectures, including AlexNet, ResNet, DenseNet and VGG. The resource is illustrated by comparing the results published for pruning AlexNet and ResNet50 on ImageNet and ResNet56 and VGG16 on the CIFAR10 data to reveal which pruning methods work well in terms of retaining accuracy whilst achieving good compression rates. The paper concludes by identifying some research gaps and promising directions for future research. INDEX TERMS Deep learning, neural networks, pruning deep networks.
All in-text references underlined in blue are linked to publications on ResearchGate, letting you... more All in-text references underlined in blue are linked to publications on ResearchGate, letting you access and read them immediately.
Phishing is the most prevalent method of cybercrime that convinces people to provide sensitive in... more Phishing is the most prevalent method of cybercrime that convinces people to provide sensitive information; for instance, account IDs, passwords, and bank details. Emails, instant messages, and phone calls are widely used to launch such cyber-attacks. Despite constant updating of the methods of avoiding such cyber-attacks, the ultimate outcome is currently inadequate. On the other hand, phishing emails have increased exponentially in recent years, which suggests a need for more effective and advanced methods to counter them. Numerous methods have been established to filter phishing emails, but the problem still needs a complete solution. To the best of our knowledge, this is the first survey that focuses on using Natural Language Processing (NLP) and Machine Learning (ML) techniques to detect phishing emails. This study provides an analysis of the numerous stateof-the-art NLP strategies currently in use to identify phishing emails at various stages of the attack, with an emphasis on ML strategies. These approaches are subjected to a comparative assessment and analysis. This gives a sense of the problem, its immediate solution space, and the expected future research directions.
The successful application of deep learning has led to increasing expectations of their use in em... more The successful application of deep learning has led to increasing expectations of their use in embedded systems. This, in turn, has created the need to find ways of reducing the size of neural networks. Decreasing the size of a neural network requires deciding which weights should be removed without compromising accuracy, which is analogous to the kind of problems addressed by multi-armed bandits (MABs). Hence, this paper explores the use of MABs for reducing the number of parameters of a neural network. Different MAB algorithms, namely $\epsilon $-greedy, win-stay, lose-shift, UCB1, KL-UCB, BayesUCB, UGapEb, successive rejects and Thompson sampling are evaluated and their performance compared to existing approaches. The results show that MAB pruning methods, especially those based on UCB, outperform other pruning methods.
Purpose Over 200,000 young people in the UK embark on a smoking career annually, thus continued e... more Purpose Over 200,000 young people in the UK embark on a smoking career annually, thus continued effort is required to understand the types of interventions that are most effective in changing perceptions about smoking amongst teenagers. Several authors have proposed the use of social norms programmes, where correcting misconceptions of what is considered normal behaviour lead to improved behaviours. There are a limited number of studies showing the effectiveness of such programmes for changing teenagers’ perception of smoking habits, and hence this paper reports on the results from one of the largest social norms programmes that used a variety of interventions aimed at improving teenagers’ perceptions of smoking. The paper aims to discuss this issue. Design/methodology/approach A range of interventions were adopted for 57 programmes in year nine students, ranging from passive interventions such as posters and banners to active interventions such as student apps and enterprise days. ...
Cost-sensitive learning has become an increasingly important area that recognizes that real world... more Cost-sensitive learning has become an increasingly important area that recognizes that real world classification problems need to take the costs of misclassification and accuracy into account. Much work has been done on cost-sensitive decision tree learning, but very little has been done on cost-sensitive Bayesian networks. Although there has been significant research on Bayesian networks there has been relatively little research on learning cost-sensitive Bayesian networks. Hence, this paper explores whether it is possible to develop algorithms that learn cost-sensitive Bayesian networks by taking (i) an indirect approach that changes the data distribution to reflect the costs of misclassification; and (ii) a direct approach that amends an existing accuracy based algorithm for learning Bayesian networks.An empirical comparison of the new approaches is carried out with cost-sensitive decision tree learning algorithms on 33 data sets, and the results show that the new algorithms perform better in terms of misclassification cost and maintaining accuracy.
Page 1. A Probabilistic Model for Case-Based Reasoning AndrOs F. Rodriguezl, Sunil Vadera 1 and L... more Page 1. A Probabilistic Model for Case-Based Reasoning AndrOs F. Rodriguezl, Sunil Vadera 1 and L. Enrique Sucar 2 ... 2. Fig. 2. Bayesian networks for indexing and classification. The first Bayesian network (BN) is used as an index structure to categories. ...
Diabetes mellitus is now recognised as a major worldwide public health problem. At present, about... more Diabetes mellitus is now recognised as a major worldwide public health problem. At present, about 100 million people are registered as diabetic patients. Many clinical, social and economic problems occur as a consequence of insulin-dependent diabetes. Treatment attempts to prevent or delay complications by applying 'optimal' glycaemic control. Therefore, there is a continuous need for effective monitoring of the patient. Given the popularity of decision tree learning algorithms as well as neural networks for knowledge classification which is further used for decision support, this paper examines their relative merits by applying one algorithm from each family on a medical problem; that of recommending a particular diabetes regime. For the purposes of this study, OC1 a descendant of Quinlan's ID3 algorithm was chosen as decision tree learning algorithm and a generating shrinking algorithm for learning arbitrary classifications as a neural network algorithm. These systems were trained on 646 cases derived from two countries in Europe and were tested on 100 cases which were different from the original 646 cases.
Knowledge Structure Mapping (KSM) is a method to help those responsible for any activity to more ... more Knowledge Structure Mapping (KSM) is a method to help those responsible for any activity to more effectively manage the knowledge which is held and applied by the people. Early experimental knowledge study systems lead to the foundations of Structural Knowledge Auditing (SKA) and a pilot project was undertaken at BA Systems in Samlesbury in 1999. Several more trials were carried out between 2000 and 2005. In 2005 AKRI Limited began to develop KSM and produced new software with a business delivery focus. Work at Airbus UK and several projects at Rolls-Royce plc lead to a great many more refinements. Later work with the Nuclear Industry has contributed further to the evolution of the method. Ultimately KSM has evolved through exposure to customer driven needs. The lines between research, design, development and deployment have not been drawn for this project and all four functions continue to take place.
Eduarda is a Researcher with the Integrated Systems group at Microsoft Research Cambridge. Her re... more Eduarda is a Researcher with the Integrated Systems group at Microsoft Research Cambridge. Her research interests lie in the broad areas of data mining and web information retrieval. In particular, her current work is focused on social network analysis, online communities, web link analysis and text mining.
During the last decade, one important contribution towards requirements engineering has been the ... more During the last decade, one important contribution towards requirements engineering has been the advent of formal specification languages. They offer a well‐defined notation that can improve consistency and avoid ambiguity in specifications. However, the process of obtaining formal specifications that are consistent with the requirements is itself a difficult activity. Hence, various researchers are developing systems that aid the transition
Proof by induction plays a central role in showing that recursive programs satisfy their specific... more Proof by induction plays a central role in showing that recursive programs satisfy their specification. Sometimes a key step is to generalise a lemma so that its inductive proof is easier. Existing heuristics for generalisation for induction are examined. The applicability of heuristics for generalisation is also examined, and it is shown that the kind of examples on which some of the heuristics work best form a well defined class of problems. A class of generalisation problems is identified for which none of the methods work, and directions for future research are provided.
A knowledge-based framework called LACEPRO where civil engineers are able to learn, consult and a... more A knowledge-based framework called LACEPRO where civil engineers are able to learn, consult and apply established procedures is presented. LACEPRO's knowledge base was designed to be used by three different knowledge operators: the Tutor, the Consultor, and the Expert. Each of these knowledge operators has its own mechanisms to perform their functions over the shared knowledge base. The core of
Advances in Computational Intelligence and Robotics, 2010
Product or company names used in this set are for identification purposes only. Inclusion of the ... more Product or company names used in this set are for identification purposes only. Inclusion of the names of the products or companies does not indicate a claim of ownership by IGI Global of the trademark or registered trademark. Library of Congress Cataloging-in-Publication Data Artificial intelligence applications for improved software engineering development : new prospects / Farid Meziane and Sunil Vadera, editors. p. cm. Includes bibliographical references and index.
Sign language is used by approximately 70 million 1 people throughout the world, and an automatic... more Sign language is used by approximately 70 million 1 people throughout the world, and an automatic tool for interpreting it could make a major impact on communication between those who use it and those who may not understand it. However, computer interpretation of sign language is very difficult given the variability in size, shape and position of the fingers or hands in an image. Hence, this paper explores the applicability of deep learning for interpreting sign language. The paper develops a convolutional neural network aimed at classifying fingerspelling images using both image intensity and depth data. The developed convolutional network is evaluated by applying it to the problem of finger spelling recognition for American Sign Language. The evaluation shows that the developed convolutional network performs better than previous studies and has precision of 82% and recall of 80%. Analysis of the confusion matrix from the evaluation reveals the underlying difficulties of classifying some particular signs which is discussed in the paper.
This paper develops a new algorithm for inducing cost-sensitive decision trees that is inspired b... more This paper develops a new algorithm for inducing cost-sensitive decision trees that is inspired by the multi-armed bandit problem, in which a player in a casino has to decide which slot machine (bandit) from a selection of slot machines is likely to pay out the most. Game Theory proposes a solution to this multi-armed bandit problem by using a process of exploration and exploitation in which reward is maximized. This paper utilizes these concepts to develop a new algorithm by viewing the rewards as a reduction in costs, and utilizing the exploration and exploitation techniques so that a compromise between decisions based on accuracy and decisions based on costs can be found. The algorithm employs the notion of lever pulls in the multi-armed bandit game to select the attributes during decision tree induction, using a lookahead methodology to explore potential attributes and exploit the attributes which maximizes the reward. The new algorithm is evaluated on fifteen datasets and compared to six wellknown algorithms J48, EG2, MetaCost, AdaCostM1, ICET and ACT. The results obtained show that the new multi-armed based algorithm can produce more cost-effective trees without compromising accuracy. The paper also includes a critical appraisal of the limitations of the new algorithm and proposes avenues for further research.
IoT sensors and deep learning models can widely be applied for fault prediction. Although deep le... more IoT sensors and deep learning models can widely be applied for fault prediction. Although deep learning models are considerably more potent than many conventional machine learning models, they are not transparent. This paper first examines different deep learning techniques to carry out univariate time series analysis based on vibration sensors installed on four industrial bearings to predict a fault occurring in a predefined time window. Several recurrent neural networks are used to develop fault prediction models. An empirical evaluation of these models shows that all models perform well; however, hybrid models outperform other models when the time window increases. Then, instance-wise feature selection has been considered to highlight the most contributing features for its outputs regarding any input. In this problem, the main challenge is to propose a trainable feature selection model with the minimum number of selected features whilst its performance is close to the baseline model. This paper develops a novel explainable method called the Gumbel-Sigmoid eXplanator (GSX) to tackle these problems. In a nutshell: (i) we have developed a differentiable and trainable selector, and (ii) we utilize regularization to control the number of features for each instance flexibly. The proposed method is model agnostic, and empirical evaluations on two datasets show that GSX can not only solve the problems identified with two other state-of-the-art methods but also outperform them in terms of accuracy and run-time.
This paper presents a survey of methods for pruning deep neural networks. It begins by categorisi... more This paper presents a survey of methods for pruning deep neural networks. It begins by categorising over 150 studies based on the underlying approach used and then focuses on three categories: methods that use magnitude based pruning, methods that utilise clustering to identify redundancy, and methods that use sensitivity analysis to assess the effect of pruning. Some of the key influencing studies within these categories are presented to highlight the underlying approaches and results achieved. Most studies present results which are distributed in the literature as new architectures, algorithms and data sets have developed with time, making comparison across different studied difficult. The paper therefore provides a resource for the community that can be used to quickly compare the results from many different methods on a variety of data sets, and a range of architectures, including AlexNet, ResNet, DenseNet and VGG. The resource is illustrated by comparing the results published for pruning AlexNet and ResNet50 on ImageNet and ResNet56 and VGG16 on the CIFAR10 data to reveal which pruning methods work well in terms of retaining accuracy whilst achieving good compression rates. The paper concludes by identifying some research gaps and promising directions for future research. INDEX TERMS Deep learning, neural networks, pruning deep networks.
All in-text references underlined in blue are linked to publications on ResearchGate, letting you... more All in-text references underlined in blue are linked to publications on ResearchGate, letting you access and read them immediately.
Phishing is the most prevalent method of cybercrime that convinces people to provide sensitive in... more Phishing is the most prevalent method of cybercrime that convinces people to provide sensitive information; for instance, account IDs, passwords, and bank details. Emails, instant messages, and phone calls are widely used to launch such cyber-attacks. Despite constant updating of the methods of avoiding such cyber-attacks, the ultimate outcome is currently inadequate. On the other hand, phishing emails have increased exponentially in recent years, which suggests a need for more effective and advanced methods to counter them. Numerous methods have been established to filter phishing emails, but the problem still needs a complete solution. To the best of our knowledge, this is the first survey that focuses on using Natural Language Processing (NLP) and Machine Learning (ML) techniques to detect phishing emails. This study provides an analysis of the numerous stateof-the-art NLP strategies currently in use to identify phishing emails at various stages of the attack, with an emphasis on ML strategies. These approaches are subjected to a comparative assessment and analysis. This gives a sense of the problem, its immediate solution space, and the expected future research directions.
The successful application of deep learning has led to increasing expectations of their use in em... more The successful application of deep learning has led to increasing expectations of their use in embedded systems. This, in turn, has created the need to find ways of reducing the size of neural networks. Decreasing the size of a neural network requires deciding which weights should be removed without compromising accuracy, which is analogous to the kind of problems addressed by multi-armed bandits (MABs). Hence, this paper explores the use of MABs for reducing the number of parameters of a neural network. Different MAB algorithms, namely $\epsilon $-greedy, win-stay, lose-shift, UCB1, KL-UCB, BayesUCB, UGapEb, successive rejects and Thompson sampling are evaluated and their performance compared to existing approaches. The results show that MAB pruning methods, especially those based on UCB, outperform other pruning methods.
Purpose Over 200,000 young people in the UK embark on a smoking career annually, thus continued e... more Purpose Over 200,000 young people in the UK embark on a smoking career annually, thus continued effort is required to understand the types of interventions that are most effective in changing perceptions about smoking amongst teenagers. Several authors have proposed the use of social norms programmes, where correcting misconceptions of what is considered normal behaviour lead to improved behaviours. There are a limited number of studies showing the effectiveness of such programmes for changing teenagers’ perception of smoking habits, and hence this paper reports on the results from one of the largest social norms programmes that used a variety of interventions aimed at improving teenagers’ perceptions of smoking. The paper aims to discuss this issue. Design/methodology/approach A range of interventions were adopted for 57 programmes in year nine students, ranging from passive interventions such as posters and banners to active interventions such as student apps and enterprise days. ...
Cost-sensitive learning has become an increasingly important area that recognizes that real world... more Cost-sensitive learning has become an increasingly important area that recognizes that real world classification problems need to take the costs of misclassification and accuracy into account. Much work has been done on cost-sensitive decision tree learning, but very little has been done on cost-sensitive Bayesian networks. Although there has been significant research on Bayesian networks there has been relatively little research on learning cost-sensitive Bayesian networks. Hence, this paper explores whether it is possible to develop algorithms that learn cost-sensitive Bayesian networks by taking (i) an indirect approach that changes the data distribution to reflect the costs of misclassification; and (ii) a direct approach that amends an existing accuracy based algorithm for learning Bayesian networks.An empirical comparison of the new approaches is carried out with cost-sensitive decision tree learning algorithms on 33 data sets, and the results show that the new algorithms perform better in terms of misclassification cost and maintaining accuracy.
Page 1. A Probabilistic Model for Case-Based Reasoning AndrOs F. Rodriguezl, Sunil Vadera 1 and L... more Page 1. A Probabilistic Model for Case-Based Reasoning AndrOs F. Rodriguezl, Sunil Vadera 1 and L. Enrique Sucar 2 ... 2. Fig. 2. Bayesian networks for indexing and classification. The first Bayesian network (BN) is used as an index structure to categories. ...
Diabetes mellitus is now recognised as a major worldwide public health problem. At present, about... more Diabetes mellitus is now recognised as a major worldwide public health problem. At present, about 100 million people are registered as diabetic patients. Many clinical, social and economic problems occur as a consequence of insulin-dependent diabetes. Treatment attempts to prevent or delay complications by applying 'optimal' glycaemic control. Therefore, there is a continuous need for effective monitoring of the patient. Given the popularity of decision tree learning algorithms as well as neural networks for knowledge classification which is further used for decision support, this paper examines their relative merits by applying one algorithm from each family on a medical problem; that of recommending a particular diabetes regime. For the purposes of this study, OC1 a descendant of Quinlan's ID3 algorithm was chosen as decision tree learning algorithm and a generating shrinking algorithm for learning arbitrary classifications as a neural network algorithm. These systems were trained on 646 cases derived from two countries in Europe and were tested on 100 cases which were different from the original 646 cases.
Knowledge Structure Mapping (KSM) is a method to help those responsible for any activity to more ... more Knowledge Structure Mapping (KSM) is a method to help those responsible for any activity to more effectively manage the knowledge which is held and applied by the people. Early experimental knowledge study systems lead to the foundations of Structural Knowledge Auditing (SKA) and a pilot project was undertaken at BA Systems in Samlesbury in 1999. Several more trials were carried out between 2000 and 2005. In 2005 AKRI Limited began to develop KSM and produced new software with a business delivery focus. Work at Airbus UK and several projects at Rolls-Royce plc lead to a great many more refinements. Later work with the Nuclear Industry has contributed further to the evolution of the method. Ultimately KSM has evolved through exposure to customer driven needs. The lines between research, design, development and deployment have not been drawn for this project and all four functions continue to take place.
Eduarda is a Researcher with the Integrated Systems group at Microsoft Research Cambridge. Her re... more Eduarda is a Researcher with the Integrated Systems group at Microsoft Research Cambridge. Her research interests lie in the broad areas of data mining and web information retrieval. In particular, her current work is focused on social network analysis, online communities, web link analysis and text mining.
During the last decade, one important contribution towards requirements engineering has been the ... more During the last decade, one important contribution towards requirements engineering has been the advent of formal specification languages. They offer a well‐defined notation that can improve consistency and avoid ambiguity in specifications. However, the process of obtaining formal specifications that are consistent with the requirements is itself a difficult activity. Hence, various researchers are developing systems that aid the transition
Proof by induction plays a central role in showing that recursive programs satisfy their specific... more Proof by induction plays a central role in showing that recursive programs satisfy their specification. Sometimes a key step is to generalise a lemma so that its inductive proof is easier. Existing heuristics for generalisation for induction are examined. The applicability of heuristics for generalisation is also examined, and it is shown that the kind of examples on which some of the heuristics work best form a well defined class of problems. A class of generalisation problems is identified for which none of the methods work, and directions for future research are provided.
A knowledge-based framework called LACEPRO where civil engineers are able to learn, consult and a... more A knowledge-based framework called LACEPRO where civil engineers are able to learn, consult and apply established procedures is presented. LACEPRO's knowledge base was designed to be used by three different knowledge operators: the Tutor, the Consultor, and the Expert. Each of these knowledge operators has its own mechanisms to perform their functions over the shared knowledge base. The core of
Advances in Computational Intelligence and Robotics, 2010
Product or company names used in this set are for identification purposes only. Inclusion of the ... more Product or company names used in this set are for identification purposes only. Inclusion of the names of the products or companies does not indicate a claim of ownership by IGI Global of the trademark or registered trademark. Library of Congress Cataloging-in-Publication Data Artificial intelligence applications for improved software engineering development : new prospects / Farid Meziane and Sunil Vadera, editors. p. cm. Includes bibliographical references and index.
Sign language is used by approximately 70 million 1 people throughout the world, and an automatic... more Sign language is used by approximately 70 million 1 people throughout the world, and an automatic tool for interpreting it could make a major impact on communication between those who use it and those who may not understand it. However, computer interpretation of sign language is very difficult given the variability in size, shape and position of the fingers or hands in an image. Hence, this paper explores the applicability of deep learning for interpreting sign language. The paper develops a convolutional neural network aimed at classifying fingerspelling images using both image intensity and depth data. The developed convolutional network is evaluated by applying it to the problem of finger spelling recognition for American Sign Language. The evaluation shows that the developed convolutional network performs better than previous studies and has precision of 82% and recall of 80%. Analysis of the confusion matrix from the evaluation reveals the underlying difficulties of classifying some particular signs which is discussed in the paper.
Uploads
Papers by Sunil Vadera