Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## com.gisgraphy.importer
Class AbstractAdvancedImporterProcessor

    java.lang.Object
      com.gisgraphy.importer.AbstractSimpleImporterProcessor
          com.gisgraphy.importer.AbstractAdvancedImporterProcessor

All Implemented Interfaces:
  IImporterProcessor

    public abstract class AbstractAdvancedImporterProcessor

extends AbstractSimpleImporterProcessor

Import the street from an (pre-processed) openStreet map data file in with all the processor capacity.

Author:
  David Masclet

Field Summary
`protected ImporterException` | `exception`
`protected FileSpliter` | `fileSpliter`
`protected IIdGenerator` | `idGenerator`
`protected static org.slf4j.Logger` | `logger` The logger
`protected List<File>` | `splitedFiles`
`protected org.springframework.scheduling.backportconcurrent.ThreadPoolTaskExecutor` | `threadPoolTaskExecutor`
`protected AtomicInteger` | `totalReadLineMultithreaded`

Fields inherited from class com.gisgraphy.importer.AbstractSimpleImporterProcessor
`COMMENT_START, currentFile, importerConfig, in, internationalisationService, readFileLine, status, statusMessage, totalReadLine, transactionManager, txDefinition, txStatus`

Constructor Summary
`AbstractAdvancedImporterProcessor()`

Method Summary
`abstract Runnable` | `createTask(File file)`
`protected void` | `flushAndClear()` Should flush and clear all the Daos that are used by the processor.
`String` | `getCurrentFileName()`
`protected int` | `getNumberOfColumns()`
`long` | `getNumberOfLinesToProcess()`
`int` | `getNumberOfProcessors()`
`int` | `getNumberOfTaskExecutors()`
`long` | `getReadFileLine()` The number of read line for the current processed file
`protected abstract List<File>` | `getSplitedFiles()`
`protected int` | `getSplitLength(int nblines)`
`long` | `getTotalReadLine()` The number of read line for all the processed file
`int` | `incrementReadedFileLine(int increment)`
`void` | `process()` Manage the transaction, flush Daos, and process all files to be processed
`protected void` | `processData(String line)` Process a read line of the geonames file, must be implemented by the concrete class
`void` | `setIdGenerator(IIdGenerator idGenerator)`
`protected void` | `setup()` Template method that can be override.
`protected boolean` | `shouldIgnoreComments()` Whether the filter should ignore the comments (i.e. lines starting with #)
`protected boolean` | `shouldIgnoreFirstLine()` Template Method : Whether the processor should ignore the first line of the input
`protected void` | `splitFiles()`

Methods inherited from class com.gisgraphy.importer.AbstractSimpleImporterProcessor
`checkNumberOfColumn, commit, countLines, decrementReadedFileLine, dumpFields, getFiles, getMaxInsertsBeforeFlush, getStatus, getStatusMessage, isEmptyField, isEndOfDocument, needCommit, onFileProcessed, processError, readLineAndProcessData, resetStatus, rollbackTransaction, setCommitFlushMode, setImporterConfig, setInternationalisationService, setTransactionManager, shouldBeSkipped, startTransaction, tearDown`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Methods inherited from interface com.gisgraphy.importer.IImporterProcessor
`rollback`

Field Detail

### idGenerator

    protected IIdGenerator idGenerator

### totalReadLineMultithreaded

    protected AtomicInteger totalReadLineMultithreaded

### threadPoolTaskExecutor

    protected org.springframework.scheduling.backportconcurrent.ThreadPoolTaskExecutor threadPoolTaskExecutor

### fileSpliter

    protected FileSpliter fileSpliter

### splitedFiles

    protected List<File> splitedFiles

### logger

    protected static final org.slf4j.Logger logger

  The logger

### exception

    protected ImporterException exception

Constructor Detail

### AbstractAdvancedImporterProcessor

    public AbstractAdvancedImporterProcessor()

Method Detail

### setup

    protected void setup()

  Description copied from class: `AbstractSimpleImporterProcessor`
  Template method that can be override. This method is called before the process start. it is not called for each file processed.

  Overrides:
    `setup` in class `AbstractSimpleImporterProcessor`

### getSplitedFiles

    protected abstract List<File> getSplitedFiles()

### splitFiles

    protected void splitFiles()

### getSplitLength

    protected int getSplitLength(int nblines)

### process

    public void process()

  Description copied from class: `AbstractSimpleImporterProcessor`
  Manage the transaction, flush Daos, and process all files to be processed

  Specified by:
    `process` in interface `IImporterProcessor`
  Overrides:
    `process` in class `AbstractSimpleImporterProcessor`

### createTask

    public abstract Runnable createTask(File file)

### incrementReadedFileLine

    public int incrementReadedFileLine(int increment)

  Overrides:
    `incrementReadedFileLine` in class `AbstractSimpleImporterProcessor`
  Parameters:
    `increment` -
  Returns:
    the incremented value

### processData

    protected void processData(String line)
                        throws ImporterException

  Description copied from class: `AbstractSimpleImporterProcessor`
  Process a read line of the geonames file, must be implemented by the concrete class

  Specified by:
    `processData` in class `AbstractSimpleImporterProcessor`
  Parameters:
    `line` - the line to process
  Throws:
    `ImporterException`

### getCurrentFileName

    public String getCurrentFileName()

  Specified by:
    `getCurrentFileName` in interface `IImporterProcessor`
  Overrides:
    `getCurrentFileName` in class `AbstractSimpleImporterProcessor`
  Returns:
    The name of the file currently processed or null if no file is processed

### getReadFileLine

    public long getReadFileLine()

  Description copied from interface: `IImporterProcessor`
  The number of read line for the current processed file

  Specified by:
    `getReadFileLine` in interface `IImporterProcessor`
  Overrides:
    `getReadFileLine` in class `AbstractSimpleImporterProcessor`
  See Also:
    `IImporterProcessor.getTotalReadLine()`

### getTotalReadLine

    public long getTotalReadLine()

  Description copied from interface: `IImporterProcessor`
  The number of read line for all the processed file

  Specified by:
    `getTotalReadLine` in interface `IImporterProcessor`
  Overrides:
    `getTotalReadLine` in class `AbstractSimpleImporterProcessor`
  See Also:
    `IImporterProcessor.getReadFileLine()`

### shouldIgnoreComments

    protected boolean shouldIgnoreComments()

  Description copied from class: `AbstractSimpleImporterProcessor`
  Whether the filter should ignore the comments (i.e. lines starting with #)

  Specified by:
    `shouldIgnoreComments` in class `AbstractSimpleImporterProcessor`
  See Also:
    `AbstractSimpleImporterProcessor.COMMENT_START`

### shouldIgnoreFirstLine

    protected boolean shouldIgnoreFirstLine()

  Description copied from class: `AbstractSimpleImporterProcessor`
  Template Method : Whether the processor should ignore the first line of the input

  Specified by:
    `shouldIgnoreFirstLine` in class `AbstractSimpleImporterProcessor`
  Returns:
    true if the processor should ignore first line

### getNumberOfColumns

    protected int getNumberOfColumns()

  Specified by:
    `getNumberOfColumns` in class `AbstractSimpleImporterProcessor`
  Returns:
    the number of fields the processed Geonames file should have

### getNumberOfProcessors

    public int getNumberOfProcessors()

### getNumberOfTaskExecutors

    public int getNumberOfTaskExecutors()

### setIdGenerator

    public void setIdGenerator(IIdGenerator idGenerator)

### flushAndClear

    protected void flushAndClear()

  Description copied from class: `AbstractSimpleImporterProcessor`
  Should flush and clear all the Daos that are used by the processor. This avoid memory leak

  Specified by:
    `flushAndClear` in class `AbstractSimpleImporterProcessor`

### getNumberOfLinesToProcess

    public long getNumberOfLinesToProcess()

  Specified by:
    `getNumberOfLinesToProcess` in interface `IImporterProcessor`
  Overrides:
    `getNumberOfLinesToProcess` in class `AbstractSimpleImporterProcessor`
  Returns:
    The number of line the processor will process. (it is not the number of lines left!)

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2012. All Rights Reserved.