Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## com.gisgraphy.importer
Class AbstractSimpleImporterProcessor

    java.lang.Object
      com.gisgraphy.importer.AbstractSimpleImporterProcessor

All Implemented Interfaces:
  IImporterProcessor

Direct Known Subclasses:
  AbstractAdvancedImporterProcessor, AdmExtracter, GeonamesAdmExtracter, GeonamesAlternateNamesExtracter, GeonamesAlternateNamesSimpleImporter, GeonamesCountryImporter, GeonamesFeatureSimpleImporter, GeonamesLanguageImporter, GeonamesZipCodeSimpleImporter, OpenStreetMapSimpleImporter

    public abstract class AbstractSimpleImporterProcessor

extends Object
implements IImporterProcessor

Base class for all geonames processor. it provides session management and the ability to process one or more CSV file

Author:
  David Masclet

Field Summary
`protected String` | `COMMENT_START` Lines starting with this prefix are considered as comments
`protected File` | `currentFile` The current processed file
`protected ImporterConfig` | `importerConfig`
`protected BufferedReader` | `in` The bufferReader for the current read Geonames file
`protected IInternationalisationService` | `internationalisationService`
`protected static org.slf4j.Logger` | `logger` The logger
`protected int` | `readFileLine`
`protected ImporterStatus` | `status`
`protected String` | `statusMessage`
`protected int` | `totalReadLine`
`protected org.springframework.transaction.PlatformTransactionManager` | `transactionManager` The transaction manager
`protected org.springframework.transaction.support.DefaultTransactionDefinition` | `txDefinition`
`protected org.springframework.transaction.TransactionStatus` | `txStatus`

Constructor Summary
`AbstractSimpleImporterProcessor()` Default constructor

Method Summary
`protected void` | `checkNumberOfColumn(String[] fields)` Utility method which throw an exception if the number of fields is not the one expected (retrieved by `getNumberOfColumns()`)
`protected void` | `commit()`
`protected int` | `countLines(File[] files)`
`protected int` | `decrementReadedFileLine(int decrement)`
`protected static String` | `dumpFields(String[] fields)`
`protected abstract void` | `flushAndClear()` Should flush and clear all the Daos that are used by the processor.
`String` | `getCurrentFileName()`
`protected abstract File[]` | `getFiles()`
`protected int` | `getMaxInsertsBeforeFlush()`
`protected abstract int` | `getNumberOfColumns()`
`long` | `getNumberOfLinesToProcess()`
`long` | `getReadFileLine()` The number of read line for the current processed file
`ImporterStatus` | `getStatus()`
`String` | `getStatusMessage()`
`long` | `getTotalReadLine()` The number of read line for all the processed file
`protected int` | `incrementReadedFileLine(int increment)`
`protected static boolean` | `isEmptyField(String[] fields, int position, boolean required)` Check that the array is not null, and the fields of the specified position is not empty (after been trimed)
`protected boolean` | `isEndOfDocument()`
`protected boolean` | `needCommit()`
`protected void` | `onFileProcessed(File file)`
`void` | `process()` Manage the transaction, flush Daos, and process all files to be processed
`protected abstract void` | `processData(String line)` Process a read line of the geonames file, must be implemented by the concrete class
`protected void` | `processError(Exception e)`
`int` | `readLineAndProcessData()` Process the line if needed (is not a comment, should ignore first line, is end of document,...)
`void` | `resetStatus()` Reset status fields, it should be done when the import has been canceled
`protected void` | `rollbackTransaction()`
`protected abstract void` | `setCommitFlushMode()` Will flush after every commit
`void` | `setImporterConfig(ImporterConfig importerConfig)`
`void` | `setInternationalisationService(IInternationalisationService internationalisationService)`
`void` | `setTransactionManager(org.springframework.transaction.PlatformTransactionManager transactionManager)`
`protected void` | `setup()` Template method that can be override.
`boolean` | `shouldBeSkipped()`
`protected abstract boolean` | `shouldIgnoreComments()` Whether the filter should ignore the comments (i.e. lines starting with #)
`protected abstract boolean` | `shouldIgnoreFirstLine()` Template Method : Whether the processor should ignore the first line of the input
`protected void` | `startTransaction()`
`protected void` | `tearDown()` Template method that can be override.

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Methods inherited from interface com.gisgraphy.importer.IImporterProcessor
`rollback`

Field Detail

### totalReadLine

    protected int totalReadLine

### readFileLine

    protected int readFileLine

### statusMessage

    protected String statusMessage

### status

    protected ImporterStatus status

### internationalisationService

    protected IInternationalisationService internationalisationService

### importerConfig

    protected ImporterConfig importerConfig

### logger

    protected static final org.slf4j.Logger logger

  The logger

### COMMENT_START

    protected String COMMENT_START

  Lines starting with this prefix are considered as comments

### in

    protected BufferedReader in

  The bufferReader for the current read Geonames file

### transactionManager

    protected org.springframework.transaction.PlatformTransactionManager transactionManager

  The transaction manager

### txStatus

    protected org.springframework.transaction.TransactionStatus txStatus

### txDefinition

    protected org.springframework.transaction.support.DefaultTransactionDefinition txDefinition

### currentFile

    protected File currentFile

  The current processed file

Constructor Detail

### AbstractSimpleImporterProcessor

    public AbstractSimpleImporterProcessor()

  Default constructor

Method Detail

### shouldIgnoreFirstLine

    protected abstract boolean shouldIgnoreFirstLine()

  Template Method : Whether the processor should ignore the first line of the input

  Returns:
    true if the processor should ignore first line

### flushAndClear

    protected abstract void flushAndClear()

  Should flush and clear all the Daos that are used by the processor. This avoid memory leak

### setCommitFlushMode

    protected abstract void setCommitFlushMode()

  Will flush after every commit

  See Also:
    `flushAndClear()`

### getNumberOfColumns

    protected abstract int getNumberOfColumns()

  Returns:
    the number of fields the processed Geonames file should have

### shouldIgnoreComments

    protected abstract boolean shouldIgnoreComments()

  Whether the filter should ignore the comments (i.e. lines starting with #)

  See Also:
    `COMMENT_START`

### setup

    protected void setup()

  Template method that can be override. This method is called before the process start. it is not called for each file processed.

### getFiles

    protected abstract File[] getFiles()

  Returns:
    The files to be process
  See Also:
    `ImporterHelper`

### getCurrentFileName

    public String getCurrentFileName()

  Specified by:
    `getCurrentFileName` in interface `IImporterProcessor`
  Returns:
    The name of the file currently processed or null if no file is processed

### readLineAndProcessData

    public int readLineAndProcessData()
                               throws ImporterException

  Process the line if needed (is not a comment, should ignore first line, is end of document,...)

  Returns:
    The number of lines that have been processed for the current processed file
  Throws:
    `ImporterException` - if an error occurred

### processData

    protected abstract void processData(String line)
                                 throws ImporterException

  Process a read line of the geonames file, must be implemented by the concrete class

  Parameters:
    `line` - the line to process
  Throws:
    `ImporterException`

### process

    public void process()

  Manage the transaction, flush Daos, and process all files to be processed

  Specified by:
    `process` in interface `IImporterProcessor`

### processError

    protected void processError(Exception e)

### shouldBeSkipped

    public boolean shouldBeSkipped()

  Specified by:
    `shouldBeSkipped` in interface `IImporterProcessor`
  Returns:
    true if the processor should Not be executed

### incrementReadedFileLine

    protected int incrementReadedFileLine(int increment)

### decrementReadedFileLine

    protected int decrementReadedFileLine(int decrement)

### rollbackTransaction

    protected void rollbackTransaction()

### needCommit

    protected boolean needCommit()

### startTransaction

    protected void startTransaction()

### tearDown

    protected void tearDown()

  Template method that can be override. This method is called after the end of the process. it is not called for each file processed. You should always call super.tearDown() when you overide this method

### onFileProcessed

    protected void onFileProcessed(File file)

### commit

    protected void commit()

### isEmptyField

    protected static boolean isEmptyField(String[] fields,
                                          int position,
                                          boolean required)

  Check that the array is not null, and the fields of the specified position is not empty (after been trimed)

  Parameters:
    `fields` - The array to test
    `position` - the position of the field to test in the array
    `required` - if an exception should be thrown if the field is empty
  Returns:
    true is the field of the specifed position is empty
  Throws:
    `MissingRequiredFieldException` - if the fields is empty and required is true

### dumpFields

    protected static String dumpFields(String[] fields)

  Parameters:
    `fields` - The array to process
  Returns:
    a string which represent a human readable string of the Array

### checkNumberOfColumn

    protected void checkNumberOfColumn(String[] fields)

  Utility method which throw an exception if the number of fields is not the one expected (retrieved by `getNumberOfColumns()`)

  Parameters:
    `fields` - The array to check
  See Also:
    `getNumberOfColumns()`

### isEndOfDocument

    protected boolean isEndOfDocument()

  Returns:
    true if the end of the document for the current processed file is reached

### getReadFileLine

    public long getReadFileLine()

  Description copied from interface: `IImporterProcessor`
  The number of read line for the current processed file

  Specified by:
    `getReadFileLine` in interface `IImporterProcessor`
  See Also:
    `IImporterProcessor.getTotalReadLine()`

### getTotalReadLine

    public long getTotalReadLine()

  Description copied from interface: `IImporterProcessor`
  The number of read line for all the processed file

  Specified by:
    `getTotalReadLine` in interface `IImporterProcessor`
  See Also:
    `IImporterProcessor.getReadFileLine()`

### setTransactionManager

    public void setTransactionManager(org.springframework.transaction.PlatformTransactionManager transactionManager)

### setImporterConfig

    public void setImporterConfig(ImporterConfig importerConfig)

### countLines

    protected int countLines(File[] files)

  Returns:
    the number of line to process

### getNumberOfLinesToProcess

    public long getNumberOfLinesToProcess()

  Specified by:
    `getNumberOfLinesToProcess` in interface `IImporterProcessor`
  Returns:
    The number of line the processor will process. (it is not the number of lines left!)

### getStatus

    public ImporterStatus getStatus()

  Specified by:
    `getStatus` in interface `IImporterProcessor`
  Returns:
    The current status of the importer

### getMaxInsertsBeforeFlush

    protected int getMaxInsertsBeforeFlush()

  Returns:
    The option
  See Also:
    `ImporterConfig.setMaxInsertsBeforeFlush(int)`

### resetStatus

    public void resetStatus()

  Description copied from interface: `IImporterProcessor`
  Reset status fields, it should be done when the import has been canceled

  Specified by:
    `resetStatus` in interface `IImporterProcessor`

### getStatusMessage

    public String getStatusMessage()

  Specified by:
    `getStatusMessage` in interface `IImporterProcessor`
  Returns:
    A text Message for the importer

### setInternationalisationService

    public void setInternationalisationService(IInternationalisationService internationalisationService)

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2012. All Rights Reserved.