Thread safe data storage.
Thread safe data storage. The purpose of DataPool is to provide a uniform interface for accessing data from decoding routines running in a multi-threaded environment. Depending on the mode of operation it may contain the actual data, may be connected to another DataPool or may be mapped to a file. Regardless of the mode, the class returns data in a thread-safe way, blocking reading threads if there is no data of interest available. This blocking is especially useful in the networking environment (plugin) when there is a running decoding thread, which wants to start decoding as soon as there is just one byte available blocking if necessary.Access to data in a DataPool may be direct (Using get_data() function) or sequential (See get_stream() function).
If the DataPool is not connected to anything, that is it contains some real data, this data can be added to it by means of two add_data() functions. One of them adds data sequentially maintaining the offset of the last block of data added by it. The other can store data anywhere. Thus it's important to realize, that there may be "white spots" in the data storage.
There is also a way to test if data is available for some given data range (See has_data()). In addition to this mechanism, there are so-called trigger callbacks, which are called, when there is all data available for a given data range.
Let us consider all modes of operation in details:
- Not connected DataPool. In this mode the DataPool contains some real data. As mentioned above, it may be added by means of two functions add_data() operating independent of each other and allowing to add data sequentially and directly to any place of data storage. It's important to call function set_eof() after all data has been added.
Functions like get_data() or get_stream() can be used to obtain direct or sequential access to the data. As long as is_eof() is FALSE, DataPool will block every reader, which is trying to read unavailable data until it really becomes available. But as soon as is_eof() is TRUE, any attempt to read non-existing data will read 0 bytes.
Taking into account the fact, that DataPool was designed to store DjVu files, which are in IFF formats, it becomes possible to predict the size of the DataPool as soon as the first 32 bytes have been added. This is invaluable for estimating download progress. See function get_length() for details. If this estimate fails (which means, that stored data is not in IFF format), get_length() returns -1.
Triggers may be added and removed by means of add_trigger() and del_trigger() functions. add_trigger() takes a data range. As soon as all data in that data range is available, the trigger callback will be called.
All trigger callbacks will be called when EOF condition has been set.
- DataPool connected to another DataPool. In this slave mode you can map a given DataPool to any offsets range inside another DataPool. You can connect the slave DataPool even if there is no data in the master DataPool. Any get_data() request will be forwarded to the master DataPool, and it will be responsible for blocking readers trying to access unavailable data.
The usage of add_data() functions is prohibited for connected DataPools.
The offsets range used to map a slave DataPool can be fully specified (both start offset and length are positive numbers) or partially specified (the length is negative). In this mode the slave DataPool is assumed to extend up to the end of the master DataPool.
Triggers may be used with slave DataPools as well as with the master ones.
Calling stop() function of a slave will stop only the slave (and any other slave connected to it), but not the master.
set_eof() function is meaningless for slaves. They obtain the EOF status from their master.
Depending on the offsets range passed to the constructor, get_length() returns different values. If the length passed to the constructor was positive, then it is returned by get_length() all the time. Otherwise the value returned is either -1 if master's length is still unknown (it didn't manage to parse IFF data yet) or it is calculated as masters_length-slave_start.
- DataPool connected to a file. This mode is quite similar to the case, when the DataPool is connected to another DataPool. Similarly, the DataPool stores no data inside. It just forwards all get_data() requests to the underlying source (a file in this case). Thus these requests will never block the reader. But they may return 0 if there is no data available at the requested offset.
The usage of add_data() functions is meaningless and is prohibited.
is_eof() function always returns TRUE. Thus set_eof() us meaningless and does nothing.
get_length() function always returns the file size.
Calling stop() function will stop this DataPool and any other slave connected to it.
Trigger callbacks passed through add_trigger() function are called immediately.
This mode is useful to read and decode DjVu files without reading and storing them in full in memory.
If only_blocked flag is TRUE then only those requests will be processed, which would not block. Any attempt to get non-existing data would result in a STOP exception (instead of blocking until data is available).
If only_blocked flag is FALSE then any further attempt to read from this DataPool (as well as from any DataPool connected to this one) will result in a STOP exception.
The function will unblock readers waiting for data if this data
arrives with this block. It may also trigger some trigger
callbacks, which may have been added by means of add_trigger()
function. Note: After all the data has been added, it's necessary
to call set_eof() to tell the DataPool that nothing else
is expected. Note: This function may not be called if the DataPool
has been connected to something.
Note: After all the data has been added, it's necessary
to call set_eof() to tell the DataPool that nothing else
is expected. Note: This function may not be called if the DataPool
has been connected to something.
Note: This function is meaningless and does nothing
when the DataPool is connected to another DataPool or to
a file.
If there is no data available, and is_eof() returns
FALSE, the reader (and the thread) will be blocked
until the data actually arrives. Please note, that since
the reader is blocked, it should run in a separate thread
so that other threads have a chance to call add_data().
If there is no data available, but is_eof() is TRUE
the behavior is different and depends on the DataPool's
estimate of the file size:
You can add a trigger callback in two ways:
Note: The callback may be called immediately if all
data for the given range is already available or EOF is TRUE.
This function is a simplified version of the function above.
The callback will be called when there is data available for
every offset from 0 to thresh, if thresh is positive, or
when EOF condition has been set otherwise.
This may be useful when you want to overwrite the file and leave
existing DataPools with valid data.
Adding data.
void add_data(const void * buffer, int size)
size - length of the buffervoid add_data(const void * buffer, int offset, int size)
offset - where to store the data
size - length of the buffer void set_eof(void)
Accessing data.
int get_data(void * buffer, int offset, int size)
.
.
EOF The requested data is not there and will not be added,
although it should have been.
offset - Offset in the DataPool to read data at
size - Size of the buffer
GP<ByteStream> get_stream(void)
State querying functions.
bool is_connected(void) const
bool has_data(int start, int length)
int get_length(void) const
.
int get_size(void) const
Trigger callbacks.
void add_trigger(int start, int length, void (* callback)(void *), void * cl_data)
length - If the length is not negative then the callback
will be called when there is data available for every
offset from start to start+length-1.
If thresh is negative, the callback is called after
EOF condition has been set.
callback - Function to call
cl_data - Argument to pass to the callback when it's called. void add_trigger(int thresh, void (* callback)(void *), void * cl_data)
void del_trigger(void (* callback)(void *), void * cl_data)
void load_file(void)
static void load_file(const char * name)
static void close_all(void)
Alphabetic index HTML hierarchy of classes or Java