Package com.stardog.stark.io

Class LineSupplier

java.lang.Object

com.stardog.stark.io.LineSupplier

All Implemented Interfaces:

java.util.function.Supplier<LineBatch>

public class LineSupplier
extends java.lang.Object
implements java.util.function.Supplier<LineBatch>

A utility class that reads lines from a stream in one thread and supplies the output as batches for multiple threads to consume. The line numbers associated with the batches will start at 1. The definition of a line follows the BufferedReader.readLine() definition. The lines in the resulting batches will not have any line-termination characters in them.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	LineSupplier.InputStreamSupplier

Constructor Summary

Constructors

Constructor	Description
LineSupplier(LineSupplier.InputStreamSupplier theInput, FileFormat.Compression theCompression)
LineSupplier(java.io.InputStream theInput, FileFormat.Compression theCompression)
LineSupplier(java.nio.file.Path theFile, FileFormat.Compression theCompression)

Method Summary

Modifier and Type	Method	Description
long	bytesRead()	The number of bytes that have been read so far.
LineBatch	get()	Retrieve the next batch available.
void	setBatchSize(int theBatchSize)	Sets the number of lines that should be in every batch.
void	setQueueSize(int theQueueSize)	Sets the maximum number of batches that should be buffered internally.
void	start()	Starts the reading process.
void	stop()	Stops reading the lines as soon as possible.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LineSupplier

public LineSupplier(java.nio.file.Path theFile,
                    FileFormat.Compression theCompression)

LineSupplier

public LineSupplier(java.io.InputStream theInput,
                    FileFormat.Compression theCompression)

LineSupplier

public LineSupplier(LineSupplier.InputStreamSupplier theInput,
                    FileFormat.Compression theCompression)

Method Detail

setBatchSize

public void setBatchSize(int theBatchSize)

Sets the number of lines that should be in every batch. Should be set before reading starts.

setQueueSize

public void setQueueSize(int theQueueSize)

Sets the maximum number of batches that should be buffered internally. Should be set before reading starts.

start

public void start()
           throws java.io.IOException

Starts the reading process. This function will return immediately and reading the input stream will be done in a thread in the shared executor until the stream is consumed or this supplier is stopped. If this function is called multiple times subsequent calls will have no effect.

Throws:

java.io.IOException

stop

public void stop()

Stops reading the lines as soon as possible. Calling this function will ensure that no more batches will be produced but stopping the thread that reads the lines might not happen instantaneously.

bytesRead

public long bytesRead()

The number of bytes that have been read so far.

get

public LineBatch get()

Retrieve the next batch available. This function will block until the next batch is available. The batches will be returned in the order they are read from the stream so if they are consumed sequentially this will be equivalent to reading the stream sequentially. If the input stream has been consumed completely an empty batch will be returned for all subsequent calls.

Specified by:

get in interface java.util.function.Supplier<LineBatch>