public class DeltaBinaryPackingValuesWriter extends ValuesWriter
delta-binary-packing: <page-header> <block>*
page-header := <block size in values> <number of miniblocks in a block> <total value count> <first value>
block := <min delta> <list of bitwidths of miniblocks> <miniblocks>
min delta : zig-zag var int encoded
bitWidthsOfMiniBlock : 1 byte little endian
blockSizeInValues,blockSizeInValues,totalValueCount,firstValue : unsigned varint
The algorithm and format is inspired by D. Lemire's paper: http://lemire.me/blog/archives/2012/09/12/fast-integer-compression-decoding-billions-of-integers-per-second/| Modifier and Type | Field and Description |
|---|---|
static int |
DEFAULT_NUM_BLOCK_VALUES |
static int |
DEFAULT_NUM_MINIBLOCKS |
static int |
MAX_BITWIDTH
max bitwidth for a mini block, it is used to allocate miniBlockByteBuffer which is
reused between flushes.
|
| Constructor and Description |
|---|
DeltaBinaryPackingValuesWriter(int slabSize,
int pageSize) |
DeltaBinaryPackingValuesWriter(int blockSizeInValues,
int miniBlockNum,
int slabSize,
int pageSize) |
| Modifier and Type | Method and Description |
|---|---|
long |
getAllocatedSize()
( >
#getBufferedMemorySize ) |
long |
getBufferedSize()
used to decide if we want to work to the next page
|
BytesInput |
getBytes()
getBytes will trigger flushing block buffer, DO NOT write after getBytes() is called without calling reset()
|
Encoding |
getEncoding()
called after getBytes() and before reset()
|
String |
memUsageString(String prefix) |
void |
reset()
called after getBytes() to reset the current buffer and start writing the next page
|
void |
writeInteger(int v) |
createDictionaryPage, resetDictionary, writeBoolean, writeByte, writeBytes, writeDouble, writeFloat, writeLongpublic static final int MAX_BITWIDTH
public static final int DEFAULT_NUM_BLOCK_VALUES
public static final int DEFAULT_NUM_MINIBLOCKS
public DeltaBinaryPackingValuesWriter(int slabSize,
int pageSize)
public DeltaBinaryPackingValuesWriter(int blockSizeInValues,
int miniBlockNum,
int slabSize,
int pageSize)
public long getBufferedSize()
ValuesWritergetBufferedSize in class ValuesWriterpublic void writeInteger(int v)
writeInteger in class ValuesWriterpublic BytesInput getBytes()
getBytes in class ValuesWriterpublic Encoding getEncoding()
ValuesWritergetEncoding in class ValuesWriterpublic void reset()
ValuesWriterreset in class ValuesWriterpublic long getAllocatedSize()
ValuesWriter#getBufferedMemorySize )getAllocatedSize in class ValuesWriterpublic String memUsageString(String prefix)
memUsageString in class ValuesWriterCopyright © 2015 The Apache Software Foundation. All rights reserved.