Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## com.gisgraphy.domain.geoloc.importer
Class AbstractGeonamesProcessor

    java.lang.Object
      com.gisgraphy.domain.geoloc.importer.AbstractGeonamesProcessor

All Implemented Interfaces:
  IGeonamesProcessor

Direct Known Subclasses:
  AdmExtracter, GeonamesAdm1Importer, GeonamesAdm2Importer, GeonamesAdm3Importer, GeonamesAdm4Importer, GeonamesAlternateNamesImporter, GeonamesCountryImporter, GeonamesFeatureImporter, GeonamesLanguageImporter

    public abstract class AbstractGeonamesProcessor

extends java.lang.Object
implements IGeonamesProcessor

Base class for all geonames processor. it provides session management and the ability to process one or more CSV file

Author:
  David Masclet

Field Summary
`protected java.lang.String` | `COMMENT_START` Lines starting with this prefix are considered as comments
`protected java.io.File` | `currentFile` The current processed file
`protected ImporterConfig` | `importerConfig`
`protected java.io.BufferedReader` | `in` The bufferReader for the current read Geonames file
`protected static org.slf4j.Logger` | `logger` The logger
`protected int` | `readFileLine`
`protected java.lang.String` | `statusMessage`
`protected int` | `totalReadLine`

Constructor Summary
`AbstractGeonamesProcessor()` Default constructor

Method Summary
`protected void` | `checkNumberOfColumn(java.lang.String[] fields)` Utility method which throw an exception if the number of fields is not the one expected (retrieved by `getNumberOfColumns()`)
`protected int` | `countLines()`
`protected static java.lang.String` | `dumpFields(java.lang.String[] fields)`
`protected abstract void` | `flushAndClear()` Should flush and clear all the Daos that are used by the processor.
`java.lang.String` | `getCurrentFileName()`
`protected abstract java.io.File[]` | `getFiles()`
`protected int` | `getMaxInsertsBeforeFlush()`
`protected abstract int` | `getNumberOfColumns()`
`int` | `getNumberOfLinesToProcess()`
`int` | `getReadFileLine()` The number of read line for the current processed file
`ImporterStatus` | `getStatus()`
`java.lang.String` | `getStatusMessage()`
`int` | `getTotalReadLine()` The number of read line for all the processed file
`protected static boolean` | `isEmptyField(java.lang.String[] fields, int position, boolean required)` Check that the array is not null, and the fields of the specified position is not empty (after been trimed)
`protected boolean` | `isEndOfDocument()`
`void` | `process()` Manage the transaction, flush Daos, and process all files to be processed
`protected abstract void` | `processData(java.lang.String line)` Process a read line of the geonames file, must be implemented by the concrete class
`int` | `readLineAndProcessData()` Process the line if needed (is not a comment, should ignore first line, is end of document,...)
`protected void` | `resetStatusFields()`
`protected abstract void` | `setCommitFlushMode()` Will flush after every commit
`void` | `setImporterConfig(ImporterConfig importerConfig)`
`void` | `setTransactionManager(org.springframework.transaction.PlatformTransactionManager transactionManager)`
`protected void` | `setup()` Template method that can be override.
`protected abstract boolean` | `shouldIgnoreComments()` Whether the filter should ignore the comments (i.e. lines starting with #)
`protected abstract boolean` | `shouldIgnoreFirstLine()` Template Method : Whether the processor should ignore the first line of the input
`protected void` | `tearDown()` Template method that can be override.

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Methods inherited from interface com.gisgraphy.domain.geoloc.importer.IGeonamesProcessor
`rollback`

Field Detail

### totalReadLine

    protected int totalReadLine

### readFileLine

    protected int readFileLine

### statusMessage

    protected java.lang.String statusMessage

### importerConfig

    protected ImporterConfig importerConfig

### logger

    protected static final org.slf4j.Logger logger

  The logger

### COMMENT_START

    protected java.lang.String COMMENT_START

  Lines starting with this prefix are considered as comments

### in

    protected java.io.BufferedReader in

  The bufferReader for the current read Geonames file

### currentFile

    protected java.io.File currentFile

  The current processed file

Constructor Detail

### AbstractGeonamesProcessor

    public AbstractGeonamesProcessor()

  Default constructor

Method Detail

### shouldIgnoreFirstLine

    protected abstract boolean shouldIgnoreFirstLine()

  Template Method : Whether the processor should ignore the first line of the input

  Returns:
    true if the processor should ignore first line

### flushAndClear

    protected abstract void flushAndClear()

  Should flush and clear all the Daos that are used by the processor. This avoid memory leak

### setCommitFlushMode

    protected abstract void setCommitFlushMode()

  Will flush after every commit

  See Also:
    `flushAndClear()`

### getNumberOfColumns

    protected abstract int getNumberOfColumns()

  Returns:
    the number of fields the processed Geonames file should have

### shouldIgnoreComments

    protected abstract boolean shouldIgnoreComments()

  Whether the filter should ignore the comments (i.e. lines starting with #)

  See Also:
    `COMMENT_START`

### setup

    protected void setup()

  Template method that can be override. This method is called before the process start. it is not called for each file processed.

### getFiles

    protected abstract java.io.File[] getFiles()

  Returns:
    The files to be process
  See Also:
    `ImporterHelper`

### getCurrentFileName

    public java.lang.String getCurrentFileName()

  Specified by:
    `getCurrentFileName` in interface `IGeonamesProcessor`
  Returns:
    The name of the file currently processed or null if no file is processed

### readLineAndProcessData

    public int readLineAndProcessData()
                               throws GeonamesProcessorException

  Process the line if needed (is not a comment, should ignore first line, is end of document,...)

  Returns:
    The number of lines that have been processed for the current processed file
  Throws:
    `GeonamesProcessorException` - if an error occurred

### processData

    protected abstract void processData(java.lang.String line)
                                 throws GeonamesProcessorException

  Process a read line of the geonames file, must be implemented by the concrete class

  Parameters:
    `line` - the line to process
  Throws:
    `GeonamesProcessorException`

### process

    public void process()

  Manage the transaction, flush Daos, and process all files to be processed

  Specified by:
    `process` in interface `IGeonamesProcessor`

### tearDown

    protected void tearDown()

  Template method that can be override. This method is called after the end of the process. it is not called for each file processed.

### isEmptyField

    protected static boolean isEmptyField(java.lang.String[] fields,
                                          int position,
                                          boolean required)

  Check that the array is not null, and the fields of the specified position is not empty (after been trimed)

  Parameters:
    `fields` - The array to test
    `position` - the position of the field to test in the array
    `required` - if an exception should be thrown if the field is empty
  Returns:
    true is the field of the specifed position is empty
  Throws:
    `MissingRequiredFieldException` - if the fields is empty and required is true

### dumpFields

    protected static java.lang.String dumpFields(java.lang.String[] fields)

  Parameters:
    `fields` - The array to process
  Returns:
    a string which represent a human readable string of the Array

### checkNumberOfColumn

    protected void checkNumberOfColumn(java.lang.String[] fields)

  Utility method which throw an exception if the number of fields is not the one expected (retrieved by `getNumberOfColumns()`)

  Parameters:
    `fields` - The array to check
  See Also:
    `getNumberOfColumns()`

### isEndOfDocument

    protected boolean isEndOfDocument()

  Returns:
    true if the end of the document for the current processed file is reached

### getReadFileLine

    public int getReadFileLine()

  Description copied from interface: `IGeonamesProcessor`
  The number of read line for the current processed file

  Specified by:
    `getReadFileLine` in interface `IGeonamesProcessor`
  See Also:
    `IGeonamesProcessor.getTotalReadLine()`

### getTotalReadLine

    public int getTotalReadLine()

  Description copied from interface: `IGeonamesProcessor`
  The number of read line for all the processed file

  Specified by:
    `getTotalReadLine` in interface `IGeonamesProcessor`
  See Also:
    `IGeonamesProcessor.getReadFileLine()`

### setTransactionManager

    public void setTransactionManager(org.springframework.transaction.PlatformTransactionManager transactionManager)

### setImporterConfig

    public void setImporterConfig(ImporterConfig importerConfig)

### countLines

    protected int countLines()

  Returns:
    the number of line to process

### getNumberOfLinesToProcess

    public int getNumberOfLinesToProcess()

  Specified by:
    `getNumberOfLinesToProcess` in interface `IGeonamesProcessor`
  Returns:
    The number of line the processor will process. (it is not the number of lines left!)

### getStatus

    public ImporterStatus getStatus()

  Specified by:
    `getStatus` in interface `IGeonamesProcessor`
  Returns:
    The current status of the importer

### getMaxInsertsBeforeFlush

    protected int getMaxInsertsBeforeFlush()

  Returns:
    The option
  See Also:
    `ImporterConfig.setMaxInsertsBeforeFlush(int)`

### resetStatusFields

    protected void resetStatusFields()

### getStatusMessage

    public java.lang.String getStatusMessage()

  Specified by:
    `getStatusMessage` in interface `IGeonamesProcessor`
  Returns:
    A text Message for the importer

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2009. All Rights Reserved.