This text describes regression-based approaches to analyzing longitudinal and repeated measures data. It emphasizes statistical models, discusses the relationships between different approaches, and uses real data to illustrate practical applications. It uses commercially available software when it exists and illustrates the program code and output. The data appendix provides many real data sets-beyond those used for the examples-which can serve as the basis for exercises.

Repeated measures data arise when the same characteristic is measured on each case or subject at several times or under several conditions. There is a multitude of techniques available for analysing such data and in the past this has led to some confusion. This book describes the whole spectrum of approaches, beginning with very simple and crude methods, working through intermediate techniques commonly used by consultant statisticians, and concluding with more recent and advanced methods. Those covered include multiple testing, response feature analysis, univariate analysis of variance approaches, multivariate analysis of variance approaches, regression models, two-stage line models, approaches to categorical data and techniques for analysing crossover designs. The theory is illustrated with examples, using real data brought to the authors during their work as statistical consultants.

Since ROC curves have become ubiquitous in many application areas, the various advances have been scattered across disparate articles and texts. ROC Curves for Continuous Data is the first book solely devoted to the subject, bringing together all the relevant material to provide a clear understanding of how to analyze ROC curves. The fundamental theory of ROC curvesThe book first discusses the relationship between the ROC curve and numerous performance measures and then extends the theory into practice by describing how ROC curves are estimated. Further building on the theory, the authors present statistical tests for ROC curves and their summary statistics. They consider the impact of covariates on ROC curves, examine the important special problem of comparing two ROC curves, and cover Bayesian methods for ROC analysis. Special topicsThe text then moves on to extensions of the basic analysis to cope with more complex situations, such as the combination of multiple ROC curves and problems induced by the presence of more than two classes. Focusing on design and interpretation issues, it covers missing data, verification bias, sample size determination, the design of ROC studies, and the choice of optimum threshold from the ROC curve. The final chapter explores applications that not only illustrate some of the techniques but also demonstrate the very wide applicability of these techniques across different disciplines. With nearly 5,000 articles published to date relating to ROC analysis, the explosive interest in ROC curves and their analysis will continue in the foreseeable future. Embracing this growth of interest, this timely book will undoubtedly guide present and future users of ROC analysis.

Since ROC curves have become ubiquitous in many application areas, the various advances have been scattered across disparate articles and texts. ROC Curves for Continuous Data is the first book solely devoted to the subject, bringing together all the relevant material to provide a clear understanding of how to analyze ROC curves. The fundamental theory of ROC curves The book first discusses the relationship between the ROC curve and numerous performance measures and then extends the theory into practice by describing how ROC curves are estimated. Further building on the theory, the authors present statistical tests for ROC curves and their summary statistics. They consider the impact of covariates on ROC curves, examine the important special problem of comparing two ROC curves, and cover Bayesian methods for ROC analysis. Special topics The text then moves on to extensions of the basic analysis to cope with more complex situations, such as the combination of multiple ROC curves and problems induced by the presence of more than two classes. Focusing on design and interpretation issues, it covers missing data, verification bias, sample size determination, the design of ROC studies, and the choice of optimum threshold from the ROC curve. The final chapter explores applications that not only illustrate some of the techniques but also demonstrate the very wide applicability of these techniques across different disciplines. With nearly 5,000 articles published to date relating to ROC analysis, the explosive interest in ROC curves and their analysis will continue in the foreseeable future. Embracing this growth of interest, this timely book will undoubtedly guide present and future users of ROC analysis.

This book should be of interest to statistics lecturers who want ready-made data sets complete with notes for teaching.

This book aims to present a summary of recent work on the interface between Artificial Intelligence and Statistics. It does this through presenting a series of papers by different authors working in different areas of this interface. These papers are a selected and referenced subset of papers presented at the 3rd International Workshop on Artificial Intelligence and Statistics, Florida, January 1991.

Repeated measures data arise when the same characteristic is measured on each case or subject at several times or under several conditions. There is a multitude of techniques available for analysing such data and in the past this has led to some confusion. This book describes the whole spectrum of approaches, beginning with very simple and crude methods, working through intermediate techniques commonly used by consultant statisticians, and concluding with more recent and advanced methods. Those covered include multiple testing, response feature analysis, univariate analysis of variance approaches, multivariate analysis of variance approaches, regression models, two-stage line models, approaches to categorical data and techniques for analysing crossover designs. The theory is illustrated with examples, using real data brought to the authors during their work as statistical consultants.

A practical guide to making good decisions in a world of missing data In the era of big data, it is easy to imagine that we have all the information we need to make good decisions. But in fact the data we have are never complete, and may be only the tip of the iceberg. Just as much of the universe is composed of dark matter, invisible to us but nonetheless present, the universe of information is full of dark data that we overlook at our peril. In Dark Data, data expert David Hand takes us on a fascinating and enlightening journey into the world of the data we don't see. Dark Data explores the many ways in which we can be blind to missing data and how that can lead us to conclusions and actions that are mistaken, dangerous, or even disastrous. Examining a wealth of real-life examples, from the Challenger shuttle explosion to complex financial frauds, Hand gives us a practical taxonomy of the types of dark data that exist and the situations in which they can arise, so that we can learn to recognize and control for them. In doing so, he teaches us not only to be alert to the problems presented by the things we don't know, but also shows how dark data can be used to our advantage, leading to greater understanding and better decisions. Today, we all make decisions using data. Dark Data shows us all how to reduce the risk of making bad ones.

This book should be of interest to statistics lecturers who want ready-made data sets complete with notes for teaching.

A practical guide to making good decisions in a world of missing data In the era of big data, it is easy to imagine that we have all the information we need to make good decisions. But in fact the data we have are never complete, and may be only the tip of the iceberg. Just as much of the universe is composed of dark matter, invisible to us but nonetheless present, the universe of information is full of dark data that we overlook at our peril. In Dark Data, data expert David Hand takes us on a fascinating and enlightening journey into the world of the data we don't see. Dark Data explores the many ways in which we can be blind to missing data and how that can lead us to conclusions and actions that are mistaken, dangerous, or even disastrous. Examining a wealth of real-life examples, from the Challenger shuttle explosion to complex financial frauds, Hand gives us a practical taxonomy of the types of dark data that exist and the situations in which they can arise, so that we can learn to recognize and control for them. In doing so, he teaches us not only to be alert to the problems presented by the things we don't know, but also shows how dark data can be used to our advantage, leading to greater understanding and better decisions. Today, we all make decisions using data. Dark Data shows us all how to reduce the risk of making bad ones.

This book is about the function and use of official statistics. It welcomes the aspiration for official statistics to be an indispensable element in the information system of a democratic society, serving the government, the economy and the public with data about the economic, demographic, social and environmental situation. The book identifies the political role of official statisticians, who decided what gets measured as well as how it is measured. While thousands of official statistics are published every year, and some are quoted by politicians, used by policy-makers or reported in the media, the authors observe that, in the main, official statistics do not feature much in everyday lives of people and businesses. The book concludes with suggestions for more that should be done, especially in the context of improving wellbeing and helping meet the worldwide set of sustainable development goals set for 2030.

This monograph is a detailed introductory presentation of the key classes of intelligent data analysis methods. The twelve coherently written chapters by leading experts provide complete coverage of the core issues. The first half of the book is devoted to the discussion of classical statistical issues, ranging from the basic concepts of probability, through general notions of inference, to advanced multivariate and time series methods, as well as a detailed discussion of the increasingly important Bayesian approaches and Support Vector Machines. The following chapters then concentrate on the area of machine learning and artificial intelligence and provide introductions into the topics of rule induction methods, neural networks, fuzzy logic, and stochastic search methods. The book concludes with a chapter on Visualization and a higher-level overview of the IDA processes, which illustrates the breadth of application of the presented ideas.

This book constitutes the refereed proceedings of an international workshop on Pattern Detection and Discovery organized by the European Science Foundation in London, UK in September 2002.The 17 revised full papers presented were carefully selected and reviewed for inclusion in this state-of-the-art book. Six papers present an introduction and general issues in the emerging field. Four papers are devoted to association rules. Four papers deal with various aspects of text mining and Web mining, and three papers explore advanced applications.

ThesearetheproceedingsofthefourthbiennialconferenceintheIntelligentData Analysisseries. TheconferencetookplaceinCascais,Portugal,13-15September 2001. Thethemeofthisconferenceseriesistheuseofcomputersinintelligent waysindataanalysis,includingtheexplorationofintelligentprogramsfordata analysis. Dataanalytictoolscontinuetodevelop,drivenbythecomputerrevo- tion. Methodswhichwouldhaverequiredunimaginableamountsofcomputing power,andwhichwouldhavetakenyearstoreachaconclusion,cannowbe appliedwitheaseandvirtuallyinstantly. Suchmethodsarebeingdevelopedby avarietyofintellectualcommunities,includingstatistics,arti?cialintelligence, neuralnetworks,machinelearning,datamining,andinteractivedynamicdata visualization. Thisconferenceseriesseekstobringtogetherresearchersstudying theuseofintelligentdataanalysisinthesevariousdisciplines,tostimulate- teractionsothateachdisciplinemaylearnfromtheothers. Soastoencourage suchinteraction,wedeliberatelykepttheconferencetoasingletrackmeeting. Thismeantthat,ofthealmost150submissionswereceived,wewereableto selectonly23fororalpresentationand16forposterpresentation. Inaddition tothesecontributedpapers,therewasakeynoteaddressfromDarylPregibon, invitedpresentationsfromKatharinaMorik,RolfBackhofen,andSunilRao,and aspecial'datachallenge'session,whereresearchersdescribedtheirattemptsto analyseachallengingdatasetprovidedbyPaulCohen. Thisacceptancerate enabledustoensureahighqualityconference,whilealsopermittingustop- videgoodcoverageofthevarioustopicssubsumedwithinthegeneralheading ofintelligentdataanalysis. Wewouldliketoexpressourthanksandappreciationtoeveryoneinvolved intheorganizationofthemeetingandtheselectionofthepapers. Itisthe behind-the-scenese?ortswhichensurethesmoothrunningandsuccessofany conference. Wewouldalsoliketoexpressourgratitudetothesponsors:Fundac" ,ao paraaCienciaeaTecnologia,Minist'eriodaCienciaedaTecnologia,Faculdade deCienciaseTecnologia,UniversidadeNovadeLisboa,Funda,c"aoCalousteG- benkianandIPEInvestimentoseParticipac" ,oesEmpresariais,S. A. September2001 FrankHo?mann DavidJ. Hand NiallAdams GabrielaGuimaraes DougFisher Organization IDA2001wasorganizedbythedepartmentofComputerScience,NewUniversity ofLisbon. ConferenceCommittee GeneralChair: DouglasFisher(VanderbiltUniversity,USA) ProgramChairs: DavidJ. Hand(ImperialCollege,UK) NiallAdams(ImperialCollege,UK) ConferenceChair: GabrielaGuimaraes(NewUniversityofLisbon,Portugal) PublicityChair: FrankHoppner(Univ. ofAppl. SciencesEmden,Germany) PublicationChair: FrankHo?mann(RoyalInstituteofTechnology,Sweden) LocalChair: FernandoMoura-Pires(UniversityofEvora,Portugal) AreaChairs: RobertaSiciliano(UniversityofNaples,Italy) ArnoSiebes(CWI,TheNetherlands) PavelBrazdil(UniversityofPorto,Portugal) ProgramCommittee NiallAdams(ImperialCollege,UK) PieterAdriaans(Syllogic,TheNetherlands) RussellAlmond(EducationalTestingService,USA) ThomasBack(InformatikCentrumDortmund,Germany) RiccardoBellazzi(UniversityofPavia,Italy) MichaelBerthold(Tripos,USA) LiuBing(NationalUniversityofSingapore) PaulCohen(UniversityofMassachusetts,USA) PaulDarius(LeuvenUniversity,Belgium) FazelFamili(NationalResearchCouncil,Canada) DouglasFisher(VanderbiltUniversity,USA) KarlFroeschl(UniversityofVienna,Austria) AlexGammerman(RoyalHolloway,UK) AdolfGrauel(UniversityofPaderborn,Germany) GabrielaGuimaraes(NewUniversityofLisbon,Portugal) LawrenceO. Hall(UniversityofSouthFlorida,USA) FrankHo?mann(RoyalInstituteofTechnology,Sweden) AdeleHowe(ColoradoStateUniversity,USA) Klaus-PeterHuber(SASInstitute,Germany) DavidJensen(UniversityofMassachusetts,USA) JoostKok(LeidenUniversity,TheNetherlands) RudolfKruse(UniversityofMagdeburg,Germany) FrankKlawonn(UniversityofAppliedSciencesEmden,Germany) VIII Organization HansLenz(FreeUniversityofBerlin,Germany) DavidMadigan(Soliloquy,USA) RainerMalaka(EuropeanMediaLaboratory,Germany) HeikkiMannila(Nokia,Finland) FernandoMouraPires(UniversityofEvora,Portugal) SusanaNascimento(UniversityofLisbon,Portugal) WayneOldford(UniversityofWaterloo,Canada) AlbertPrat(TechnicalUniversityofCatalunya,Spain) PeterProtzel(TechnicalUniversityChemnitz,Germany) GiacomodellaRiccia(UniversityofUdine,Italy) RosannaSchiavo(UniversityofVenice,Italy) KaisaSere(AboAkademiUniversity,Finland) RobertaSiciliano(UniversityofNaples,Italy) RosariaSilipo(Nuance,USA) FloorVerdenius(ATO-DLO,TheNetherlands) StefanWrobel(UniversityofMagdeburg,Germany) HuiXiaoLiu(BrunelUniversity,UK) NevinZhang(HongKongUniversityofScienceandTechnology,HongKong) SponsoringInstitutions Fundac" ,aoparaaCienciaeaTecnologia,Minist'eriodaCienciaedaTecnologia FaculdadedeCienciaseTecnologia,UniversidadeNovadeLisboa Fundac" ,aoCalousteGulbenkian IPEInvestimentoseParticipac" ,15September 2001. Thethemeofthisconferenceseriesistheuseofcomputersinintelligent waysindataanalysis,includingtheexplorationofintelligentprogramsfordata analysis. Dataanalytictoolscontinuetodevelop,drivenbythecomputerrevo- tion. Methodswhichwouldhaverequiredunimaginableamountsofcomputing power,andwhichwouldhavetakenyearstoreachaconclusion,cannowbe appliedwitheaseandvirtuallyinstantly. Suchmethodsarebeingdevelopedby avarietyofintellectualcommunities,includingstatistics,arti?cialintelligence, neuralnetworks,machinelearning,datamining,andinteractivedynamicdata visualization. Thisconferenceseriesseekstobringtogetherresearchersstudying theuseofintelligentdataanalysisinthesevariousdisciplines,tostimulate- teractionsothateachdisciplinemaylearnfromtheothers. Soastoencourage suchinteraction,wedeliberatelykepttheconferencetoasingletrackmeeting. Thismeantthat,ofthealmost150submissionswereceived,wewereableto selectonly23fororalpresentationand16forposterpresentation. Inaddition tothesecontributedpapers,therewasakeynoteaddressfromDarylPregibon, invitedpresentationsfromKatharinaMorik,RolfBackhofen,andSunilRao,and aspecial'datachallenge'session,whereresearchersdescribedtheirattemptsto analyseachallengingdatasetprovidedbyPaulCohen. Thisacceptancerate enabledustoensureahighqualityconference,whilealsopermittingustop- videgoodcoverageofthevarioustopicssubsumedwithinthegeneralheading ofintelligentdataanalysis. Wewouldliketoexpressourthanksandappreciationtoeveryoneinvolved intheorganizationofthemeetingandtheselectionofthepapers. Itisthe behind-the-scenese?ortswhichensurethesmoothrunningandsuccessofany conference. Wewouldalsoliketoexpressourgratitudetothesponsors:Fundac" cao paraaCienciaeaTecnologia,Minist'eriodaCienciaedaTecnologia,Faculdade deCienciaseTecnologia,UniversidadeNovadeLisboa,Fundacc"aoCalousteG- benkianandIPEInvestimentoseParticipac" coesEmpresariais,S. A. September2001 FrankHo?mann DavidJ. Hand NiallAdams GabrielaGuimaraes DougFisher Organization IDA2001wasorganizedbythedepartmentofComputerScience,NewUniversity ofLisbon. ConferenceCommittee GeneralChair: DouglasFisher(VanderbiltUniversity,USA) ProgramChairs: DavidJ. Hand(ImperialCollege,UK) NiallAdams(ImperialCollege,UK) ConferenceChair: GabrielaGuimaraes(NewUniversityofLisbon,Portugal) PublicityChair: FrankHoppner(Univ. ofAppl. SciencesEmden,Germany) PublicationChair: FrankHo?mann(RoyalInstituteofTechnology,Sweden) LocalChair: FernandoMoura-Pires(UniversityofEvora,Portugal) AreaChairs: RobertaSiciliano(UniversityofNaples,Italy) ArnoSiebes(CWI,TheNetherlands) PavelBrazdil(UniversityofPorto,Portugal) ProgramCommittee NiallAdams(ImperialCollege,UK) PieterAdriaans(Syllogic,TheNetherlands) RussellAlmond(EducationalTestingService,USA) ThomasBack(InformatikCentrumDortmund,Germany) RiccardoBellazzi(UniversityofPavia,Italy) MichaelBerthold(Tripos,USA) LiuBing(NationalUniversityofSingapore) PaulCohen(UniversityofMassachusetts,USA) PaulDarius(LeuvenUniversity,Belgium) FazelFamili(NationalResearchCouncil,Canada) DouglasFisher(VanderbiltUniversity,USA) KarlFroeschl(UniversityofVienna,Austria) AlexGammerman(RoyalHolloway,UK) AdolfGrauel(UniversityofPaderborn,Germany) GabrielaGuimaraes(NewUniversityofLisbon,Portugal) LawrenceO. Hall(UniversityofSouthFlorida,USA) FrankHo?mann(RoyalInstituteofTechnology,Sweden) AdeleHowe(ColoradoStateUniversity,USA) Klaus-PeterHuber(SASInstitute,Germany) DavidJensen(UniversityofMassachusetts,USA) JoostKok(LeidenUniversity,TheNetherlands) RudolfKruse(UniversityofMagdeburg,Germany) FrankKlawonn(UniversityofAppliedSciencesEmden,Germany) VIII Organization HansLenz(FreeUniversityofBerlin,Germany) DavidMadigan(Soliloquy,USA) RainerMalaka(EuropeanMediaLaboratory,Germany) HeikkiMannila(Nokia,Finland) FernandoMouraPires(UniversityofEvora,Portugal) SusanaNascimento(UniversityofLisbon,Portugal) WayneOldford(UniversityofWaterloo,Canada) AlbertPrat(TechnicalUniversityofCatalunya,Spain) PeterProtzel(TechnicalUniversityChemnitz,Germany) GiacomodellaRiccia(UniversityofUdine,Italy) RosannaSchiavo(UniversityofVenice,Italy) KaisaSere(AboAkademiUniversity,Finland) RobertaSiciliano(UniversityofNaples,Italy) RosariaSilipo(Nuance,USA) FloorVerdenius(ATO-DLO,TheNetherlands) StefanWrobel(UniversityofMagdeburg,Germany) HuiXiaoLiu(BrunelUniversity,UK) NevinZhang(HongKongUniversityofScienceandTechnology,HongKong) SponsoringInstitutions Fundac" caoparaaCienciaeaTecnologia,Minist'eriodaCienciaedaTecnologia FaculdadedeCienciaseTecnologia,UniversidadeNovadeLisboa Fundac" caoCalousteGulbenkian IPEInvestimentoseParticipac" coesEmpresariais,S. A. TableofContents TheFourthInternationalSymposiumonIntelligentData Analysis FeatureCharacterizationinScienti?cDatasets...1 ElizabethBradley(UniversityofColorado),NancyCollins(University ofColorado),W. PhilipKegelmeyer(SandiaNationalLaboratories) RelevanceFeedbackintheBayesianNetworkRetrievalModel: AnApproachBasedonTermInstantiation...13 LuisM. deCampos(UniversityofGranada),JuanM. Fernan ' dez-Luna (UniversityofJa'en),JuanF. Huete(UniversityofGranada) GeneratingFuzzySummariesfromFuzzyMultidimensionalDatabases...24 AnneLaurent(Universit'ePierreetMarieCurie) AMixture-of-ExpertsFrameworkforLearningfromImbalancedData Sets...34 AndrewEstabrooks(IBM),NathalieJapkowicz(UniversityofOttawa) PredictingTime-VaryingFunctionswithLocalModels...44 AchimLewandowski(ChemnitzUniversity),PeterProtzel(Chemnitz University) BuildingModelsofEcologicalDynamicsUsingHMMBasedTemporal DataClustering-APreliminaryStudy...53 CenLi(TennesseeStateUniversity),GautamBiswas(Vanderbilt University),MikeDale(Gri?thUniversity),PatDale(Gri?th University) TaggingwithSmallTrainingCorpora...63 NunoC. Marques(UniversidadeAberta),GabrielPereiraLopes (Centria) ASearchEngineforMorphologicallyComplexLanguages...7 3 UdoHahn(UniversitatFreiburg),MartinHoneck(Universitat sklinikum Freiburg),StefanSchulz(Universitat oppner(Univ. ofAppl. SciencesEmden,Germany) PublicationChair: FrankHo?mann(RoyalInstituteofTechnology,Sweden) LocalChair: FernandoMoura-Pires(UniversityofEvora,Portugal) AreaChairs: RobertaSiciliano(UniversityofNaples,Italy) ArnoSiebes(CWI,TheNetherlands) PavelBrazdil(UniversityofPorto,Portugal) ProgramCommittee NiallAdams(ImperialCollege,UK) PieterAdriaans(Syllogic,TheNetherlands) RussellAlmond(EducationalTestingService,USA) ThomasBack(InformatikCentrumDortmund,Germany) RiccardoBellazzi(UniversityofPavia,Italy) MichaelBerthold(Tripos,USA) LiuBing(NationalUniversityofSingapore) PaulCohen(UniversityofMassachusetts,USA) PaulDarius(LeuvenUniversity,Belgium) FazelFamili(NationalResearchCouncil,Canada) DouglasFisher(VanderbiltUniversity,USA) KarlFroeschl(UniversityofVienna,Austria) AlexGammerman(RoyalHolloway,UK) AdolfGrauel(UniversityofPaderborn,Germany) GabrielaGuimaraes(NewUniversityofLisbon,Portugal) LawrenceO. Hall(UniversityofSouthFlorida,USA) FrankHo?mann(RoyalInstituteofTechnology,Sweden) AdeleHowe(ColoradoStateUniversity,USA) Klaus-PeterHuber(SASInstitute,Germany) DavidJensen(UniversityofMassachusetts,USA) JoostKok(LeidenUniversity,TheNetherlands) RudolfKruse(UniversityofMagdeburg,Germany) FrankKlawonn(UniversityofAppliedSciencesEmden,Germany) VIII Organization HansLenz(FreeUniversityofBerlin,Germany) DavidMadigan(Soliloquy,USA) RainerMalaka(EuropeanMediaLaboratory,Germany) HeikkiMannila(Nokia,Finland) FernandoMouraPires(UniversityofEvora,Portugal) SusanaNascimento(UniversityofLisbon,Portugal) WayneOldford(UniversityofWaterloo,Canada) AlbertPrat(TechnicalUniversityofCatalunya,Spain) PeterProtzel(TechnicalUniversityChemnitz,Germany) GiacomodellaRiccia(UniversityofUdine,Italy) RosannaSchiavo(UniversityofVenice,Italy) KaisaSere(AboAkademiUniversity,Finland) RobertaSiciliano(UniversityofNaples,Italy) RosariaSilipo(Nuance,USA) FloorVerdenius(ATO-DLO,TheNetherlands) StefanWrobel(UniversityofMagdeburg,Germany) HuiXiaoLiu(BrunelUniversity,UK) NevinZhang(HongKongUniversityofScienceandTechnology,HongKong) SponsoringInstitutions Fundac" caoparaaCienciaeaTecnologia,Minist'eriodaCienciaedaTecnologia FaculdadedeCienciaseTecnologia,UniversidadeNovadeLisboa Fundac" caoCalousteGulbenkian IPEInvestimentoseParticipac" coesEmpresariais,S. A. TableofContents TheFourthInternationalSymposiumonIntelligentData Analysis FeatureCharacterizationinScienti?cDatasets...1 ElizabethBradley(UniversityofColorado),NancyCollins(University ofColorado),W. PhilipKegelmeyer(SandiaNationalLaboratories) RelevanceFeedbackintheBayesianNetworkRetrievalModel: AnApproachBasedonTermInstantiation...13 LuisM. deCampos(UniversityofGranada),JuanM. Fernan ' dez-Luna (UniversityofJa'en),JuanF. Huete(UniversityofGranada) GeneratingFuzzySummariesfromFuzzyMultidimensionalDatabases...24 AnneLaurent(Universit'ePierreetMarieCurie) AMixture-of-ExpertsFrameworkforLearningfromImbalancedData Sets...34 AndrewEstabrooks(IBM),NathalieJapkowicz(UniversityofOttawa) PredictingTime-VaryingFunctionswithLocalModels...44 AchimLewandowski(ChemnitzUniversity),PeterProtzel(Chemnitz University) BuildingModelsofEcologicalDynamicsUsingHMMBasedTemporal DataClustering-APreliminaryStudy...53 CenLi(TennesseeStateUniversity),GautamBiswas(Vanderbilt University),MikeDale(Gri?thUniversity),PatDale(Gri?th University) TaggingwithSmallTrainingCorpora...63 NunoC. Marques(UniversidadeAberta),GabrielPereiraLopes (Centria) ASearchEngineforMorphologicallyComplexLanguages...7 3 UdoHahn(UniversitatFreiburg),MartinHoneck(Universitat sklinikum Freiburg),StefanSchulz(Universitat sklinikumFreiburg) ErrorsDetectionandCorrectioninLargeScaleDataCollecting...84 RenatoBruni(Universit'adiRoma),AntonioSassano(Universit'adi Roma) X TableofContents ANewFrameworktoAssessAssociationRules ...95 FernandoBerzal(UniversityofGranada),IgnacioBlanco(University ofGranada),DanielS'anchez(UniversityofGranada), Mar'?a-AmparoVila(UniversityofGranada) CommunitiesofInterest ...105 CorinnaCortes(AT&TShannonResearchLabs),DarylPregibon (AT&TShannonResearchLabs),ChrisVolinsky(AT&TShannon ResearchLabs) AnEvaluationofGradingClassi?ers ...115 AlexanderK. Seewald(AustrianResearchInstituteforArti?cial Intelligence),JohannesFur nkranz(AustrianResearchInstitutefor Arti?cialIntelligence) FindingInformativeRulesinIntervalSequences ...125 FrankHoppner(UniversityofAppliedSciencesEmden), FrankKlawonn(UniversityofAppliedSciencesEmden) Correlation-BasedandContextualMerit-BasedEnsembleFeature Selection...135 SeppoPuuronen(UniversityofJyvaskyla),AlexeyTsymbal(University ofJyvaskyla),IrynaSkrypnyk(UniversityofJyvas kyla) NonmetricMultidimensionalScalingwithNeuralNetworks...145 MichielC. vanWezel(UniversiteitLeiden),WalterA. Kosters (UniversiteitLeiden),PetervanderPutten(UniversiteitLeiden), JoostN. Kok(UniversiteitLeiden) FunctionalTreesforRegression...156 Joao " Gama(UniversityofPorto) DataMiningwithProductsofTrees...167 Jos'eTom'eA. S. Ferreira(ImperialCollege),DavidG. T. Denison (ImperialCollege),DavidJ. Hand(ImperialCollege) 3 SBagging:FastClassi?erInductionMethodwithSubsamplingand Bagging ...177 MasahiroTerabe(MitshibishiResearchInstitute,Inc. ),TakashiWashio (I. S. I. R. ,OsakaUniversity),HiroshiMotoda(I. S. I. R.

Formanyyearstheintersectionofcomputing anddataanalysiscontainedme- based statistics packages and not much else. Recently, statisticians have - braced computing, computer scientists have started using statistical theories and methods, and researchers in all corners have invented algorithms to nd structure in vast online datasets. Data analysts now have access to tools for exploratory data analysis, decision tree induction, causal induction, function - timation, constructingcustomizedreferencedistributions, andvisualization, and thereareintelligentassistantsto adviseonmatters ofdesignandanalysis.There aretoolsfortraditional, relativelysmallsamples, andalsoforenormousdatasets. In all, the scope for probing data in new and penetrating ways has never been so exciting. The IDA-99 conference brings together a wide variety of researchers c- cerned with extracting knowledge from data, including people from statistics, machine learning, neural networks, computer science, pattern recognition, da- base management, and other areas.The strategiesadopted by people from these areas are often di erent, and a synergy results if this is recognized. The IDA series of conferences is intended to stimulate interaction between these di erent areas, sothatmorepowerfultoolsemergeforextractingknowledgefromdataand a better understanding is developed of the process of intelligent data analysis. The result is a conference that has a clear focus (one application area: intelligent data analysis) and a broad scope (many di erent methods and techn

Measurement is a fundamental concept that underpins almost every aspect of the modern world. It is central to the sciences, social sciences, medicine, and economics, but it affects everyday life. We measure everything - from the distance of far-off galaxies to the temperature of the air, levels of risk, political majorities, taxes, blood pressure, IQ, and weight. The history of measurement goes back to the ancient world, and its story has been one of gradual standardization. Today there are different types of measurement, levels of accuracy, and systems of units, applied in different contexts. Measurement involves notions of variability, accuracy, reliability, and error, and challenges such as the measurement of extreme values. In this Very Short Introduction, David Hand explains the common mathematical framework underlying all measurement, the main approaches to measurement, and the challenges involved. Following a brief historical account of measurement, he discusses measurement as used in the physical sciences and engineering, the life sciences and medicine, the social and behavioural sciences, economics, business, and public policy. ABOUT THE SERIES: The Very Short Introductions series from Oxford University Press contains hundreds of titles in almost every subject area. These pocket-sized books are the perfect way to get ahead in a new subject quickly. Our expert authors combine facts, analysis, perspective, new ideas, and enthusiasm to make interesting and challenging topics highly readable.

Statistical ideas and methods underlie just about every aspect of modern life. From randomized clinical trials in medical research, to statistical models of risk in banking and hedge fund industries, to the statistical tools used to probe vast astronomical databases, the field of statistics has become centrally important to how we understand our world. But the discipline underlying all these is not the dull statistics of the popular imagination. Long gone are the days of manual arithmetic manipulation. Nowadays statistics is a dynamic discipline, revolutionized by the computer, which uses advanced software tools to probe numerical data, seeking structures, patterns, and relationships. This Very Short Introduction sets the study of statistics in context, describing its history and giving examples of its impact, summarizes methods of gathering and evaluating data, and explains the role played by the science of chance, of probability, in statistical methods. The book also explores deep philosophical issues of induction--how we use statistics to discern the true nature of reality from the limited observations we necessarily must make.
About the Series: Combining authority with wit, accessibility, and style, Very Short Introductions offer an introduction to some of life's most interesting topics. Written by experts for the newcomer, they demonstrate the finest contemporary thinking about the central problems and issues in hundreds of key topics, from philosophy to Freud, quantum theory to Islam.

In "The Improbability Principle," the renowned statistician David J. Hand argues that extraordinarily rare events are anything but. In fact, they're commonplace. Not only that, we should all expect to experience a miracle roughly once every month.
But Hand is no believer in superstitions, prophecies, or the paranormal. His definition of "miracle" is thoroughly rational. No mystical or supernatural explanation is necessary to understand why someone is lucky enough to win the lottery twice, or is destined to be hit by lightning three times and still survive. All we need, Hand argues, is a firm grounding in a powerful set of laws: the laws of inevitability, of truly large numbers, of selection, of the probability lever, and of near enough.
Together, these constitute Hand's groundbreaking Improbability Principle. And together, they explain why we should not be so surprised to bump into a friend in a foreign country, or to come across the same unfamiliar word four times in one day. Hand wrestles with seemingly less explicable questions as well: what the Bible and Shakespeare have in common, why financial crashes are par for the course, and why lightning does strike the same place (and the same person) twice. Along the way, he teaches us how to use the Improbability Principle in our own lives--including how to cash in at a casino and how to recognize when a medicine is truly effective.
An irresistible adventure into the laws behind "chance" moments and a trusty guide for understanding the world and universe we live in, "The Improbability Principle" will transform how you think about serendipity and luck, whether it's in the world of business and finance or you're merely sitting in your backyard, tossing a ball into the air and wondering where it will land.