|
Message
From: nec556 at retena.com<nec556@r...>
Date: Thu Apr 28 22:21:39 CEST 2005
Subject: [oc] open lossless data compression core
----- Original Message -----
From: abgoyal@g...<abgoyal@g...>
To:
Date: Fri Apr 15 15:11:51 CEST 2005
Subject: [oc] open lossless data compression core
> Hi,
> I have been trying to see if i can implement a lossless data
> compression core (like gzip, etc) in HDL. I searched a lot for some
> simple compression cores already in HDL that I could use. The only
> one
> i found was the static huffman encoder/decoder in jpeg core from
> opencores. That is not really usable for general data as its a
> static
> encoder. I am interested in a general purpose data compression
> core.
> Has anyone here ever written any kind of data compression cores in
> HDL? From my study of the "deflate" algorithm, i think it
> will be
> extremely tough to code it in verilog but, any other simpler
> compressor implementation, that I can learn from?
> I am surprised that no such simple cores exist. Is it very tough to
> develop such a core?
> Any opinion on this would be welcome. If by any luck, i get a large
> number of replies, I also volunteer to edit and submit to OC as a
> FAQ/HOWTO (depending on the results).
> Thanks,
> Abgoyal.
>
Hello, you have thousands of lossles compression algorithms. If you
post at nntp://comp.compression the answer will be like this:
<answer>
What type of data you want to compress:?
Is a stream or blocks of data:? Size:?
Do you know something about data (are text, image, voice, numbers...
has structure, is a mathematical function behind it):?
What speed you need:?
What compression you need:?
</answer>
If i were you i try a static statistical as golomb/rice codes plus a
model (MTF, etc..). Or an arithmethic coder (dinamic statistical) with
Finite context model of order 0, 1 or even 2 (256 bytes, 64KB or 16MB
of memory) plus a model. If you already has a sort algorithm
implemented in hdl, you can try BWT as model. Check
http://www.arturocampos.com and the comp.compression newsgroup for
more info and tutorials. Deflate, other dictionary algorithms (called
also LZ) need data structures for dfast search substrings. Also, FCM
with order >3 need a lot of memory (multiply by 256 for next order) or
use linked lists. A FCM widely used is PPM, check it.
HTH
|
 |
Browse
