Central Library

Normal view MARC view ISBD view

An Introduction to Duplicate Detection [electronic resource] / by Felix Nauman, Melanie Herschel.

By: Nauman, Felix [author.].
Contributor(s): Herschel, Melanie [author.] | SpringerLink (Online service).
Material type: materialTypeLabelBookSeries: Synthesis Lectures on Data Management: Publisher: Cham : Springer International Publishing : Imprint: Springer, 2010Edition: 1st ed. 2010.Description: IX, 77 p. online resource.Content type: text Media type: computer Carrier type: online resourceISBN: 9783031018350.Subject(s): Computer networks  | Data structures (Computer science) | Information theory | Computer Communication Networks | Data Structures and Information TheoryAdditional physical formats: Printed edition:: No title; Printed edition:: No titleDDC classification: 004.6 Online resources: Click here to access online
Contents:
Data Cleansing: Introduction and Motivation -- Problem Definition -- Similarity Functions -- Duplicate Detection Algorithms -- Evaluating Detection Success -- Conclusion and Outlook -- Bibliography.
In: Springer Nature eBookSummary: With the ever increasing volume of data, data quality problems abound. Multiple, yet different representations of the same real-world objects in data, duplicates, are one of the most intriguing data quality problems. The effects of such duplicates are detrimental; for instance, bank customers can obtain duplicate identities, inventory levels are monitored incorrectly, catalogs are mailed multiple times to the same household, etc. Automatically detecting duplicates is difficult: First, duplicate representations are usually not identical but slightly differ in their values. Second, in principle all pairs of records should be compared, which is infeasible for large volumes of data. This lecture examines closely the two main components to overcome these difficulties: (i) Similarity measures are used to automatically identify duplicates when comparing two records. Well-chosen similarity measures improve the effectiveness of duplicate detection. (ii) Algorithms are developed to perform on very large volumes of data in search for duplicates. Well-designed algorithms improve the efficiency of duplicate detection. Finally, we discuss methods to evaluate the success of duplicate detection. Table of Contents: Data Cleansing: Introduction and Motivation / Problem Definition / Similarity Functions / Duplicate Detection Algorithms / Evaluating Detection Success / Conclusion and Outlook / Bibliography.
    average rating: 0.0 (0 votes)
No physical items for this record

Data Cleansing: Introduction and Motivation -- Problem Definition -- Similarity Functions -- Duplicate Detection Algorithms -- Evaluating Detection Success -- Conclusion and Outlook -- Bibliography.

With the ever increasing volume of data, data quality problems abound. Multiple, yet different representations of the same real-world objects in data, duplicates, are one of the most intriguing data quality problems. The effects of such duplicates are detrimental; for instance, bank customers can obtain duplicate identities, inventory levels are monitored incorrectly, catalogs are mailed multiple times to the same household, etc. Automatically detecting duplicates is difficult: First, duplicate representations are usually not identical but slightly differ in their values. Second, in principle all pairs of records should be compared, which is infeasible for large volumes of data. This lecture examines closely the two main components to overcome these difficulties: (i) Similarity measures are used to automatically identify duplicates when comparing two records. Well-chosen similarity measures improve the effectiveness of duplicate detection. (ii) Algorithms are developed to perform on very large volumes of data in search for duplicates. Well-designed algorithms improve the efficiency of duplicate detection. Finally, we discuss methods to evaluate the success of duplicate detection. Table of Contents: Data Cleansing: Introduction and Motivation / Problem Definition / Similarity Functions / Duplicate Detection Algorithms / Evaluating Detection Success / Conclusion and Outlook / Bibliography.

There are no comments for this item.

Log in to your account to post a comment.