![Scientific and Technical Journal of Information Technologies, Mechanics and Optics](/images/mag-ntv.png)
METHOD OF RARE TERM CONTRASTIVE EXTRACTION FROM NATURAL LANGUAGE TEXTS
![Scientific and Technical Journal of Information Technologies, Mechanics and Optics](/images/mag-ntv.png)
Annotation
The paper considers a problem of automatic domain term extraction from documents corpus by means of a contrast collection. Existing contrastive methods successfully extract often used terms but mishandle rare terms. This could yield poorness of the resulting thesaurus. Assessment of point-wise mutual information is one of the known statistical methods of term extraction and it finds rare terms successfully. Although, it extracts many false terms at that. The proposed approach consists of point-wise mutual information application for rare terms extraction and filtering of candidates by criterion of joint occurrence with the other candidates. We build “documents-by-terms” matrix that is subjected to singular value decomposition to eliminate noise and reveal strong interconnections. Then we pass on to the resulting matrix “terms-by-terms” that reproduces strength of interconnections between words. This approach was approved on a documents collection from “Geology” domain with the use of contrast documents from such topics as “Politics”, “Culture”, “Economics” and “Accidents” on some Internet resources. The experimental results demonstrate operability of this method for rare terms extraction
Keywords
Постоянный URL
Articles in current issue
- NANOSCALE STRUCTURES GENERATION WITHIN THE SURFACE LAYER OF METALS WITH SHORT UV LASER PULSES
- INKJET PRINTING OF ALUMOOXIDE SOL FOR DEPOSITION OF ANTIREFLECTING COATINGS
- INCREASED IMAGE QUALITY BY SYNTHESIZING SPACE PHOTOS WITH DIFFERENT EXPOSURES
- ROBUST CONTROL ALGORITHM FOR MULTIVARIABLE PLANTS WITH QUANTIZED OUTPUT
- COLLAPSE KINETIC OF COMPOSITES BASED ON COPOLYMERS OF ACRYLIC ACID AND ACRYLAMIDE FILLED WITH BENTONITE IN AQUEOUS SOLUTIONS OF POLYVALENT METALS
- FORMATION OF NANOSTRUCTURED CuO FILM ON FLUOROPHOSPHATE GLASS SURFACE
- VIRTUAL REALITY FOR MANAGEMENT OF SITUATIONAL AWARENESS DURING GLOBAL MASS GATHERINGS
- MUTUAL IMAGE TRANSFORMATION ALGORITHMS FOR VISUAL INFORMATION PROCESSING AND RETRIEVAL
- AUTOMATIC ANALYSIS OF LOCAL ROUTES AND ADJACENT HOUSE TERRITORY FOR URBAN PLANNING SUPPORT
- ANALYSIS OF STATISTICAL DATA FROM NETWORK INFRASTRUCTURE MONITORING TO DETECT ABNORMAL BEHAVIOR OF SYSTEM LOCAL SEGMENTS
- ON INFORMATION SECURITY SOLUTIONS APPLICABLE TO D2D COMMUNICATIONS WITHIN THE 5G DOMAIN: ANALYZING THE INFLUENCE OF USER MOBILITY
- PROBABILITY DISTRIBUTION OVER THE SET OF CLASSES IN ARABIC DIALECT CLASSIFICATION TASK
- DYNAMIC FEATURE SELECTION FOR WEB USER IDENTIFICATION ON LINGUISTIC AND STYLISTIC FEATURES OF ONLINE TEXTS
- DEEP LEARNING MODEL FOR BILINGUAL SENTIMENT CLASSIFICATION OF SHORT TEXTS
- INNOVATIVE HEAT FLUX SENSOR
- APPROACH TO SYNTHESIS OF PASSIVE INFRARED DETECTORS BASED ON QUASI-POINT MODEL OF QUALIFIED INTRUDER
- NUMERICAL SIMULATION OF MASS TRANSFER IN CENTRIFUGAL EVAPORATOR
- A CALCULATION OF SEMI-EMPIRICAL ONE-ELECTRON WAVE FUNCTIONS FOR MULTI-ELECTRON ATOMS USED FOR ELEMENTARY PROCESS SIMULATION IN NONLOCAL PLASMA
- PARAMETRICAL IDENTIFICATION OF DIFFERENTIAL-DIFFERENCE HEAT TRANSFER MODEL DURING LIDAR TEMPERATURE MONITORING
- EFFECT OF UV LASER ON SPECTRAL PROPERTIES OF BORATE GLASSES DOPED WITH COPPER CHLORIDE NANOCRYSTALS
- PROJECT ENGINEERING DATA MANAGEMENT AT AUTOMATED PREPARATION OF DESIGN DOCUMENTATION
- CONTROL SYSTEM FOR TILTABLE PLATE WITH TWO DEGREES OF FREEDOM FOR RESEARCH OF DYNAMIC MANIPULATION PROBLEMS
- APPARATUS FOR SURFACE TREATMENT OF FREE-FORM OBJECT BY LASER RADIATION
- AUTOMATED REMOTE MANAGEMENT AND CONTROL SYSTEM OF THE LABORATORY EQUIPMENT