Topographic Processing of Very Large Text Datasets


A common challenge today, arising especially in the field of text and web mining, is the proper visualization of very large datasets to explore their structure and gain information that otherwise would remain concealed, buried due to the sheer amount of data. These large non-Euclidean datasets cannot be held at once within random-access memory during computation, so fast batch variants of common mapping methods like Self-Organizing Maps (SOM) or Neural Gas (NG) cannot be applied. In this work, we present fast approximate (semi-)supervised versions of SOM and NG for non-Euclidean data that are able to handle large text datasets by a single pass technique. The introduced methods are based on patches that can be chosen in accordance to the size of the available random-access memory. These algorithms are running in constant space and linear time and provide different visualizations of the data space. Moreover, the patch technique opens the way for long-term learning and scalability, what is especially suited for information systems where the database is updated perpetually, e.g. in visual search engines etc. We demonstrate the methods on several typical problems from text mining.

  • Abstract
  • I. Introduction
  • II. Topographic Maps
  • III. Relational Data
  • IV. Patch Relational Methods
  • V. Experiments
  • VI. Conclusions
  • VII. Appendix

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In