Inductive Inference

Inductive inference allows humans to make powerful generalizations from sparse data when learning about word meanings, unobserved properties, causal relationships, and many other aspects of the world. Traditional accounts of induction emphasize either the power of statistical learning, or the importance of strong constraints from structured domain knowledge, intuitive theories or schemas. We argue that both components are necessary to explain the nature, use and acquisition of human knowledge, and we introduce a theory-based Bayesian framework for modeling inductive learning and reasoning as statistical inferences over structured knowledge representations.

Increasingly abundant trace data provide an opportunity for information systems researchers to generate new theory. In this research commentary, we draw on the largely “manual” tradition of the grounded theory methodology and the highly “automated” process of computational theory discovery in the sciences to develop a general approach to computationally intensive theory development from trace data. This approach involves the iterative application of four general processes: sampling, synchronic analysis, lexical framing, and diachronic analysis. We provide examples from recent research in information systems.

This chapter situates Mill’s System of Logic (1843/1872) in the context of some of the meta-logical themes and disputes characteristic of the 19th century as well as Mill’s empiricism. Particularly, by placing the Logic in relation to Whately’s (1827) Elements of Logic and Mill’s response to the “great paradox” of the informativeness of syllogistic reasoning, the chapter explores the development of Mill’s views on the foundation and function of, and the relation between, ratiocination and induction. It provides a survey of the Mill-Whewell debate on the nature of induction, Mill’s account of putatively a priori disciplines such as the science of number, and Frege’s criticisms of the Logic as psychologistic.

Karl Popper lamented the prevalence of dogmatic argument in philosophy and commended the kind of critical argument that is found in the sciences. David Miller criticises the uncritical nature of so-called critical thinking because of its attachment to dogmatic arguments. I expound and clarify Popper’s distinction between critical and dogmatic arguments and the background to it. I criticise some errors in Miller’s discussion. I reaffirm the need for philosophers to eschew dogmatic arguments in favour of critical ones.

There is a widespread idea that in grounded theory (GT) research, the researcher has to delay the literature review until the end of the analysis to avoid contamination – a dictum that might turn educational researchers away from GT. Nevertheless, in this article the author (a) problematizes the dictum of delaying a literature review in classic grounded theory, (b) presents arguments for using extant literature in the substantive field within a constructivist grounded theory, and (c) suggests data sensitizing principles in using literature, which are: theoretical agnosticism, theoretical pluralism, theoretical sampling of literature, staying grounded, theoretical playfulness, memoing extant knowledge associations, and constant reflexivity.

“We hope you will find these thoughts of ours both interesting and useful.” These are words spoken to express an intention, a bearing in the mind of a person toward an object which is yet to be achieved. The readiest moment of human life involves the interplay of signs, ideas, and objects — more explicitly, the interrelation of signifying expressions, states and dispositions of the mind or person, and objects or objectives either actual or potential. Our work designing instruments to enhance the play of inquiry has attuned us to the themes of interpretation and intentionality which every inquiry seems to involve. We hear what sounds like familiar strains reaching us from the hermeneutic quarter. The purpose of this essay is to trace to their sources a few of these potentially common themes, to draw out one line of their historical development, and to gather what consequences they inspire for educational practice and continued inquiry.

This article is focused on answering the question to what extent one can be a sceptic. Sextus Empiricus’s Outlines of Scepticism serves as a guide. In section 1, it is investigated whether three logical laws have a certain foundation or are subject to doubt. In section 2, Sextus’s way to deal with these laws is examined; the question arises how dogmatic his approach is. After that, a possible ‘reply’ by Sextus to the criticism receives attention. Section 3 is concentrated on a possible alternative to Sextus’s approach. Besides logical laws, some important methods are concerned.

The common way of putting a hypothesis H to an empirical test is by drawing empirical predictions E from H and observing whether E is true. If it is indeed true, H is said to be inductively confirmed by the evidence. So presumably H is a conclusion of a strong inductive argument involving E. However, the argument we use for the above confirmation is, 1. If H then E; 2. E; 3. Therefore, H. But this does not have the form of an inductively strong argument (in fact, it is the fallacy of affirming the consequent). I propose a solution, by showing that within the context of scientific inquiry, this argument can be shown to be indeed an enumerative induction.

It is usually accepted that deductions are non-informative and monotonic, inductions are informative and nonmonotonic, abductions create hypotheses but are epistemically irrelevant, and both deductions and inductions can’t provide new insights. In this article, I attempt to provide a more cohesive view of the subject with the following hypotheses: (1) the paradigmatic examples of deductions, such as modus ponens and hypothetical syllogism, are not inferential forms, but coherence requirements for inferences; (2) since any reasoner aims to be coherent, any inference must be deductive; (3) a coherent inference is an intuitive process where the premises should be taken as sufficient evidence for the conclusion, which on its turn should be viewed as necessary evidence for the premises in some modal range; (4) inductions, properly understood, are abductions, but there are no abductions beyond the fact that in any inference the conclusion should be regarded as necessary evidence for the premises; (5) monotonicity is not only compatible with the retraction of past inferences given new information, but it is a requirement for it; (6) this explanation of inferences holds true for discovery processes, predictions and trivial inferences.

Solomonoff induction is a brute-force method of predicting future bits of a binary string given past bits. It does so by comparing the past bits to the output of every possible input program to a universal Turing machine. It uses the programs of the matching output to predict future bits, weighted by the length of the program. Because it uses every possible input to a universal Turing machine, some of the programs do not halt. Therefore this semi-measure is incomputable. It does however have some nice properties; it is enumerable and multiplicatively dominates all other enumerable semi-measures.

The technology for building knowledge-based systems by inductive inference from examples has been demonstrated successfully in several practical applications. This paper summarizes an approach to synthesizing decision trees that has been used in a variety of systems, and it describes one such system, ID3, in detail. Results from recent studies show ways in which the methodology can be modified to

Heuristics embodying limited information search and noncompensatory processing of information can yield robust performance relative to computationally more complex models. One criticism raised against heuristics is the argument that complexity is hidden in the calculation of the cue order used to make predictions. We discuss ways to order cues that do not entail individual learning. Then we propose and test the thesis that when orders are learned individually, people's necessarily limited knowledge will curtail computational complexity while also achieving robustness. Using computer simulations, we compare the performance of the take-the-best heuristic-with dichotomized or undichotomized cues-to benchmarks such as the naïve Bayes algorithm across 19 environments. Even with minute sizes of training sets, take-the-best using undichotomized cues excels. For 10 environments, we probe people's intuitions about the direction of the correlation between cues and criterion. On the basis of these intuitions, in most of the environments take-the-best achieves the level of performance that would be expected from learning cue orders from 50% of the objects in the environments. Thus, ordinary information about cues-either gleaned from small training sets or intuited-can support robust performance without requiring Herculean computations.

There is little serious doubt about the role of inductive inference in scientific inquiry, as it has proven to allow knowledge to grow. The concerning lack of agreement, however, is about the justification of induction itself: Induction has been associated with fallible reasoning and therefore its reliability has been brought into question. Philosophers, famously Hume and Popper, have been very harshly critical of inductions, for reasons that will be spelt out below. This paper presents the philosophical stance of the pragmatistic approach of induction, championed by many philosophers and scientists, but most notably and interestingly in the cases of Charles Peirce (1839–1914, the famous polymath founder of Pragmatism) and Kazimierz Ajdukiewicz (1890–1963, a prominent member of the Polish logical movement and Lvov–Warsaw School). These two philosophers belong to different historical periods of philosophical development, but share a common interest of ascribing priority to the practical consequences of adopting beliefs, attitudes, methods and practices. Both Peirce and Ajdukiewicz were brilliant logicians and, last but not least, both were deeply involved in the science of their time, for which direct experience in scientific endeavours helped them to contrast the value of their theoretical constructs. These facts make the establishment of a connection between the two philosophers philosophically interesting and illuminating, as their considerations on the value of induction seem to converge in the priority of a self-controlled scientific practice. In the following lines I will introduce
STUDIA METODOLOGICZNE NR 35 @BULLET 2015

Is the geometry of space a macroscopic manifestation of an underlying microscopic statistical structure? Is geometrodynamics-the theory of gravity-derivable from general principles for processing information? Tentative answers are suggested by a model of geometrodynamics based on the statistical concepts of entropy, the Fisher-Rao information metric, and entropic dynamics. The model shows remarkable similarities with the 3+1 formulation of general relativity. For example, the dynamical degrees of freedom are those that specify the conformal geometry of space; there is a gauge symmetry under 3d diffeomorphisms; there is no reference to an external time; and the theory is time reversible. There is, in adition, a gauge symmetry under scale transformations. I conjecture that under a suitable choice of gauge one can recover the usual notion of a relativistic space-time.

Among the three predominant types of logic—deductive, inductive, and abductive—inductive logic has long captured the attention of Western philosophy; this is due to its concomitant collection of seemingly incalcitrant challenges. In this essay, I shall discuss both the old and new riddle of induction. In so doing, my hope is to discuss the core issues associated with each, respectively, as well as lay bare the rational motivation for the transition from the old problem of induction to induction's 20th century renewed challenge. As such, I shall also provide critical reflections regarding attempts to resolve both the old and new riddles of induction, along with suggestions for further relevant avenues of research.

This article analyzes the problem of induction from its origins in Aristotle, naturally through Hume and Popper, to the frequentist and Bayesian concepts of mathematical statistics. Thus it tackles Popper's arguments about the impossibility of inductive probability and the contemporary approaches to the theory of statistical estimation. Its two most important conclusions are: 1. that Popper fails to prove that the inductive probability is impossible, and 2. that in the context of mathematical statistics statistical inferences are only deductive inferences.

Case-based reasoning is deemed an important technology to alleviate the bottleneckof knowledge acquisition in Artificial Intelligence (AI). In case-based reasoning,knowledge is represented in the form of particular cases with an appropriate similaritymeasure rather than any form of rules. The case-based reasoning paradigm adopts theview that an AI system is dynamically changing during its life-cycle which immediatelyleads to learning considerations.Within the

Recently, due to the rapid growth of electronic data having graph structures such as HTML and XML texts and chemical compounds, many researchers have been interested in data mining and machine learning techniques for finding useful patterns from graphstructured data (graph data). Since graph data contain a huge number of substructures and it tends to be computationally expensive to decide whether or not such data have given structural features, graph mining problems face computational difficulties. Let C be a graph class which satisfies a connected hereditary property and contains infinitely many different biconnected graphs, and for which a special kind of the graph isomorphism problem can be computed in polynomial time. In this paper, we consider learning and mining problems for C. Firstly, we define a new graph pattern, which is called a block preserving graph pattern (bpgraph pattern) for C. Secondly, we present a polynomial time algorithm for deciding whether or not a given bp-graph pattern matches a given graph in C. Thirdly, by giving refinement operators over bp-graph patterns, we present a polynomial time algorithm for finding a minimally generalized bp-graph pattern for C. Outerplanar graphs are planar graphs which can be embedded in the plane in such a way that all of vertices lie on the outer boundary. Many pharmacologic chemical compounds are known to be represented by outerplanar graphs.

It is increasingly acknowledged both among epidemiologists as well as regulators that the assessment of pharmaceutical harm requires specific methodological approaches which cannot simply duplicate those developed for testing efficacy. However, this intuition lacks sound epistemic bases and delivers ad hoc advice. This paper explains why the same methods of scientific inference do not fare equally well for efficacy and safety assessment by tracing them back to their epistemic foundations. In order to illustrate this, Cartwright’s distinction into clinching and vouching methods is adopted and a series of reasons is provided for preferring the latter to the former: 1) the need to take into account all available knowledge and integrate it with incoming data; 2) the awareness that a latent unknown risk may always change the safety profile of a given drug (precautionary principle); 3) cumulative learning over time; 4) requirement of probabilistic causal assessment in order to allow decision under uncertainty; 5) impartiality; 6) limited and local information provided by RCTs. Subsequently,  the clinchers/vouchers distinction is applied to a case study concerning the debated causal association between paracetamol and asthma. This study illustrates the tension between implicit epistemologies adopted in evaluating evidence and causality; furthermore it also shows that discounting causal evidence may be a result of unacknowledged low priors or lack of valid alternative options.
We conclude with a presentation of the changing landscape in pharmacology and the trend towards an increased use of Bayesian tools for assessment of harms.

A normative theory attempts to provide recommendations regarding what to do. It follows that normative theories are addressed to an audience of people facing decisions who are capable of understanding their recommendations. However, not every recommendation qualifies as normative science. There are recommendations that may be classified as moral, religious, or political preaching. These are characterized by suggesting goals to the decision makers, and, as such, are outside the realm of academic activity. There is an 3 We apply the standard usage to the title of this book as well. 2. META-THEORETICAL VOCABULARY additional type of recommendations that we do not consider as normative theories. These are recommendations that belong to the domain of social planning or engineering. They are characterized by recommending tools for achieving pre-specified goals. For instance, the design of allocation mechanisms that yield Pareto optimal outcomes accepts the given goal of Pareto optimality and solves an engineering-like problem of obtaining it. Our use of "normative science" differs from both these types of recommendations. A normative scientific claim may be viewed as an implicit descriptive statement about decision makers' preferences. The latter are about conceivable realities that are the subject of descriptive theories. For instance, whereas descriptive theory of choice investigates actual preferences, normative theory of choice analyzes the kind of preferences that the decision maker would like to have, that is, preferences over preferences. An axiom such as transitivity of preferences, when normatively interpreted, attempts to describe the way the decision maker would prefer to make choices. Similarly, Harsanyi (1953, 1955) and Rawls (1971) can be interpreted as normative theories for social choice in that they attempt to describe to what society one would like to belong. According to this definition, normative theories are also descriptive. They attempt to describe a certain reality, namely, the preferences an individual has over the reality she encounters. To avoid confusion, we will reserve the term "descriptive theory" for theories that are not normative. That is, descriptive theories would deal, by definition, with "first-order" reality, whereas normative theories would deal with "second-order" reality, namely, with preferences over first-order reality. First-order reality may be external or objective, whereas second-order reality always has to do with subjective preferences that lie within the mind of an individual. Yet, first-order reality might include actual preferences, in which case the distinction between first

The linguistic description is proposed as the centrepiece of scientific modelling, being the way by which the concepts that are considered relevant to a target system's composition and behaviour are presented, manipulated and communicated. The expression " model prescription " is used to mean the linguistic piece that describes a target system and consequently prescribes the production of synthetic data intended to model physically acquired data. The model prescription is in itself a concept of a target system that uses other systems' concepts, as well as concepts of objects, properties, relations and actions, to describe its structure and behaviour. The communication of a model prescription is therefore dependent on the intersubjective understanding and acceptance of the concepts used. Disentangled and narrower concepts of model and graphics are proposed. " Model " is restricted to representation by the same kind, categorically matched, while representations by different kind are named " graphics ". Data is identified as the sole touching point between physical systems and its human conceptualizations and the use of data-based inference by regression is analysed and some problems identified. Besides logic-mathematical relations, causal relations are an important part of modelling and a strict characterization of causal mechanisms is made based on energy transfers. Modelling examples are given to illustrate the generality of the definitions used as well as some of the problems with scientific modelling.

Previous research (e.g., Gelman & Markman, 1986; Gopnik & Sobel, 2000) suggests that children can use category labels to make inductive inferences about non-obvious causal properties of objects. However, such inductive generalizations can fail to predict objects' causal properties when A) the property being projected varies within the category; B) the category is arbitrary (e.g., things smaller than a bread box), or C) the property being projected is due to an exogenous intervention rather than intrinsic to the object kind. In four studies, we show that preschoolers (mean: 48 months; range: 42-57 months) are sensitive to these constraints on induction and selectively engage in exploration when evidence about objects' causal properties conflicts with inductive generalizations from the objects' kind to their causal powers. This suggests that the exploratory actions children generate in free play could support causal learning.

Past research suggests that young children are often reluctant to generalize about people's behavior. Three experiments involving 102 4-5-year-olds, 84 7-8-year-olds, and 107 adults explored the conditions under which inductive inferences about people are made. There was an age-based increase in propensity to predict consistency in psychological/intentional causal relations. Children often predicted change; people would behave differently in the future than they did in the past. Younger children limited predictions of consistency to non-psychological contexts. Older children showed some appreciation of stable motivations (e.g. traits, preferences). The results are consistent with the hypothesis that children's theories of mind emphasize situational influences, with personal influences appearing in middle-childhood. q

I explain how the growth of knowledge depends upon World 3 even though it involves creative discovery. I show the falsity of the oft-repeated claim that the conclusion of a valid argument is contained in its premises and I explain how that claim is a consequence of early modern conceptions of logical analysis, the structure of propositions and concepts, and the nature of necessary truth. I argue that inference, and also logical analysis, is a matter of guessing and testing, that is, a matter of discovery that depends upon World 3. I explain in what sense it is true that, as Karl Popper says, ‘we never know what we are talking about.’

In a way this chapter was one of the hardest to write, because of the possible repercussions of any error or omission. I apologize in advance to anyone I might have left out by mistake. First of all I would of course like to thank my promotores Michael Moortgat and Wojciech Buszkowski. This thesis is the result of an intellectual journey that began when I enrolled on the Artifical Intelligence and Cognitive Science program at the Faculty of Philosophy of Utrecht University, which at that time was the only place in the country that offered the opportunity of studying such an 'esoteric' subject. My time as a student has been an exciting and challenging experience. I thank Jan, Pepijn, Remco, Jeronimus and Frits especially for making this period in my life a colourful one indeed. During this time I became increasingly interested in (machine) learning, and took courses on neural networks, genetic algorithms and the like. Though this was interesting enough, I found there was a lack of a firm theoretical basis for these approaches, and also felt that most of these techniques were applied to toy problems. I was looking for something more challenging as subject for a master's thesis and felt that learning grammars would fit the bill nicely. During this time Michael Moortgat lent me a copy of Kanazawa's thesis and sent me to Dick de Jongh, only then did things start to come together. I also was fortunate enough to be able to follow the course on learnability that Makoto Kanazawa taught at the Utrecht institute of Linguistics OTS in 1998, and I thank him for his many helpful comments. I'd also like to thank Jean-Jules Meyer, Wiebe van der Hoek and Cees Witteveen for helping me out when I was looking for a Ph.D. position. During my time as a Ph.D. student I had the pleasure of meeting to the following people who have influenced me or were helpful in some way or other:

As noted by Wesley Salmon and many others, causal concepts are ubiquitous in every branch of theoretical science, in the practical disciplines and in everyday life. In the theoretical and practical sciences especially, people often base claims about causal relations on applications of statistical methods to data. However, the source and type of data place important constraints on the choice

Why do words support inductive generalization in preschoolers? The current study provides evidence that they do so, at least in part, by working with conceptual knowledge to establish kind-membership. Thirty four-year-olds learned new labels for novel items, sometimes along with additional non-obvious information, and were then asked to generalize a novel object property to a target item based on either visual similarity or shared label. Children were more likely to generalize properties based on shared labels (over perceptual similarity) if they initially learned causally-coherent properties of items referenced by those labels than if they initially learned non-causal properties of those items or learned no properties at all. This finding suggests that novel words best support inductive inference when they are known by children to reference conceptually coherent categories. Therefore, conceptual information permeates the process of inductive inference in young children. Results are discussed with respect to their implications for the 'word-as-feature' and 'knowledge-based' accounts of early inductive inference.

Susan Haack criticises the US courts’ use of Karl Popper’s epistemology in discriminating acceptable scientific testimony. She claims that acceptable testimony should be reliable and that Popper’s epistemology is useless in discriminating reliability. She says that Popper’s views have been found acceptable only because they have been misunderstood and she indicates an alternative epistemology which she says can discriminate reliable theories. However, her account of Popper’s views is a gross and gratuitous misrepresentation. Her alternative epistemology cannot do what she claims for it. The courts should not be concerned with reliability and, insofar as they use the term ‘reliability,’ it should be construed in a procedural rather than a substantive sense. Since Popper’s epistemology gives something like a characterisation of science at its best, the courts should continue to invoke Popper’s theories in their discrimination of acceptable testimony.

It is increasingly acknowledged both among epidemiologists and regulators that the assessment of pharmaceutical harm requires specific methodological approaches that cannot simply duplicate those developed for testing efficacy. However, this intuition lacks sound epistemic bases and delivers ad hoc advice. This paper explains why the same methods of scientific inference do not fare equally well for efficacy and safety assessment by tracing them back to their epistemic foundations. To illustrate this, Cartwright's distinction into clinching and vouching methods is adopted and a series of reasons is provided for preferring the latter to the former: (1) the need to take into account all available knowledge and integrate it with incoming data; (2) the awareness that a latent unknown risk may always change the safety profile of a given drug (precautionary principle); (3) cumulative learning over time; (4) requirement of probabilistic causal assessment to allow decision under uncertai...

Category-based induction is an inferential mechanism that uses knowledge of conceptual relations in order to estimate how likely is for a property to be projected from one category to another. During the last decades, psychologists have identified several features of this mechanism, and they have proposed different formal models of it. In this article; we propose a new mathematical model for category-based induction based on distances on conceptual spaces. We show how this model can predict most of the properties of this kind of reasoning while providing a solid theoretical foundation for it. We also show that it subsumes some of the previous models proposed in the literature and that it generates new predictions.

To claim that young children's biological thought is anthropocentric or that their induction depends on similarity rather than categories is to overlook the role of experience in reasoning. We tested four groups of 4-year-olds differing in two aspects of exposure to biological information: (a) their direct experience with nature (urban versus rural) and (b) biological expertise of their parents (expert biologists versus laypeople). We used a modified version of a novel feature projection task-in which projections are made from humans, mammals and insects to an array of nine targets representing various ontological kinds. Children's exposure to biology had consequences on their patterns of induction. There was an effect of parents' expertise: laypeople's children's projections were based on similarity, while experts' children's projections were based on the category animal. There was also an effect of direct experience: rural children were more restricted in their responses than were urban children. Although we found asymmetries in projections between humans and animals, humans were not a better source of knowledge about animates than were mammals or insects. These results show that the early emergence of domain specific biological thought and the use of biological categories in induction are highly contingent on children's experience.

Humans and animals make inferences about the world under limited time and knowledge. In contrast, many models of rational inference treat the mind as a Laplacean Demon, equipped with unlimited time, knowledge, and computational might. Following H. Simon's notion of satisficing, the authors have proposed a family of algorithms based on a simple psychological mechanism: onereason decision making. These fast and frugal algorithms violate fundamental tenets of classical rationality: They neither look up nor integrate all information. By computer simulation, the authors held a competition between the satisficing "Take The Best" algorithm and various "rational" inference procedures (e.g., multiple regression). The Take The Best algorithm matched or outperformed all competitors in inferential speed and accuracy. This result is an existence proof that cognitive mechanisms capable of successful performance in the real world do not need to satisfy the classical norms of rational inference. Organisms make inductive inferences. Darwin (1872/1965) observed that people use facial cues, such as eyes that waver and lids that hang low, to infer a person's guilt. Male toads, roaming through swamps at night, use the pitch of a rival's croak to infer its size when deciding whether to fight (Krebs & Davies, 1987). Stock brokers must make fast decisions about which of several stocks to trade or invest when only limited information is available. The list goes on. Inductive inferences are typically based on uncertain cues: The eyes can deceive, and so can a tiny toad with a deep croak in the darkness. How does an organism make inferences about unknown aspects of the environment? There are three directions in which to look for an answer. From Pierre Laplace to George Boole to Jean Piaget, many scholars have defended the now classical view that the laws of human inference are the laws of probability and statistics (and to a lesser degree logic, which does not deal as easily with uncertainty). Indeed, the Enlightenment probabilists derived the laws of probability from what they believed to be the laws of human reasoning (Daston, 1988). Following this time-honored tradition, much contemporary research in psychology, behavioral ecology, and economics assumes standard

Abstract—Scenarios are increasingly recognized as an effective means for eliciting, validating, and documenting software requirements. This paper concentrates on the use of scenarios for requirements elicitation and explores the process of inferring formal specifications of ...

We discuss how the method of maximum entropy, MaxEnt, can be extended beyond its original scope, as a rule to assign a probability distribution, to a full-fledged method for inductive inference. The main concept is the (relative) entropy S[p|q] which is designed as a tool to update from a prior probability distribution q to a posterior probability distribution p when new information in the form of a constraint becomes available. The extended method goes beyond the mere selection of a single posterior p, but also addresses the question of how much less probable other distributions might be. Our approach clarifies how the entropy S[p|q] is used while avoiding the question of its meaning. Ultimately, entropy is a tool for induction which needs no interpretation. Finally, being a tool for generalization from special examples, we ask whether the functional form of the entropy depends on the choice of the examples and we find that it does. The conclusion is that there is no single general theory of inductive inference and that alternative expressions for the entropy are possible.