000 04553nam a22005655i 4500
001 978-3-031-02174-9
003 DE-He213
005 20240730165216.0
007 cr nn 008mamaa
008 220601s2020 sz | s |||| 0|eng d
020 _a9783031021749
_9978-3-031-02174-9
024 7 _a10.1007/978-3-031-02174-9
_2doi
050 4 _aQ334-342
050 4 _aTA347.A78
072 7 _aUYQ
_2bicssc
072 7 _aCOM004000
_2bisacsh
072 7 _aUYQ
_2thema
082 0 4 _a006.3
_223
100 1 _aDror, Rotem.
_eauthor.
_4aut
_4http://id.loc.gov/vocabulary/relators/aut
_987852
245 1 0 _aStatistical Significance Testing for Natural Language Processing
_h[electronic resource] /
_cby Rotem Dror, Lotem Peled-Cohen, Segev Shlomov, Roi Reichart.
250 _a1st ed. 2020.
264 1 _aCham :
_bSpringer International Publishing :
_bImprint: Springer,
_c2020.
300 _aXVII, 98 p.
_bonline resource.
336 _atext
_btxt
_2rdacontent
337 _acomputer
_bc
_2rdamedia
338 _aonline resource
_bcr
_2rdacarrier
347 _atext file
_bPDF
_2rda
490 1 _aSynthesis Lectures on Human Language Technologies,
_x1947-4059
505 0 _aPreface -- Acknowledgments -- Introduction -- Statistical Hypothesis Testing -- Statistical Significance Tests -- Statistical Significance in NLP -- Deep Significance -- Replicability Analysis -- Open Questions and Challenges -- Conclusions -- Bibliography -- Authors' Biographies.
520 _aData-driven experimental analysis has become the main evaluation tool of Natural Language Processing (NLP) algorithms. In fact, in the last decade, it has become rare to see an NLP paper, particularly one that proposes a new algorithm, that does not include extensive experimental analysis, and the number of involved tasks, datasets, domains, and languages is constantly growing. This emphasis on empirical results highlights the role of statistical significance testing in NLP research: If we, as a community, rely on empirical evaluation to validate our hypotheses and reveal the correct language processing mechanisms, we better be sure that our results are not coincidental. The goal of this book is to discuss the main aspects of statistical significance testing in NLP. Our guiding assumption throughout the book is that the basic question NLP researchers and engineers deal with is whether or not one algorithm can be considered better than another one. This question drivesthe field forward as it allows the constant progress of developing better technology for language processing challenges. In practice, researchers and engineers would like to draw the right conclusion from a limited set of experiments, and this conclusion should hold for other experiments with datasets they do not have at their disposal or that they cannot perform due to limited time and resources. The book hence discusses the opportunities and challenges in using statistical significance testing in NLP, from the point of view of experimental comparison between two algorithms. We cover topics such as choosing an appropriate significance test for the major NLP tasks, dealing with the unique aspects of significance testing for non-convex deep neural networks, accounting for a large number of comparisons between two NLP algorithms in a statistically valid manner (multiple hypothesis testing), and, finally, the unique challenges yielded by the nature of the data and practices of the field.
650 0 _aArtificial intelligence.
_93407
650 0 _aNatural language processing (Computer science).
_94741
650 0 _aComputational linguistics.
_96146
650 1 4 _aArtificial Intelligence.
_93407
650 2 4 _aNatural Language Processing (NLP).
_931587
650 2 4 _aComputational Linguistics.
_96146
700 1 _aPeled-Cohen, Lotem.
_eauthor.
_4aut
_4http://id.loc.gov/vocabulary/relators/aut
_987854
700 1 _aShlomov, Segev.
_eauthor.
_4aut
_4http://id.loc.gov/vocabulary/relators/aut
_987856
700 1 _aReichart, Roi.
_eauthor.
_4aut
_4http://id.loc.gov/vocabulary/relators/aut
_987857
710 2 _aSpringerLink (Online service)
_987859
773 0 _tSpringer Nature eBook
776 0 8 _iPrinted edition:
_z9783031001857
776 0 8 _iPrinted edition:
_z9783031010460
776 0 8 _iPrinted edition:
_z9783031033025
830 0 _aSynthesis Lectures on Human Language Technologies,
_x1947-4059
_987861
856 4 0 _uhttps://doi.org/10.1007/978-3-031-02174-9
912 _aZDB-2-SXSC
942 _cEBK
999 _c86160
_d86160