Cat on (RAM) steroids

[Eli Billauer]

Hi all,


I need a simple command-line program, which works as a plain FIFO stream 
buffer with a huge RAM. Something I can do:


$ fatcat -b 256M /dev/datasource | ./my_shaky_data_sink


The idea is that "fatcat" reads data whenever available and stores it to 
non-swappable RAM. It then pushes the data to stdout. So it's just like 
good old "cat", only with a potentially large tummy.


Rationale: The (kernel) device /dev/datasource has limited RAM it can 
allocate in kernel space. If data arrives at a high constant rate (say, 
100 MB/sec), the kernel buffers (4 MB total?) overflow rather quickly if 
not constantly drained. A real-life data sink may write the data to a 
disk with sufficient *average* rate, but with breaks every now and then 
to mind its own business.


So if data is loaded into a huge RAM array (what is 256 MB these days?) 
by a high-priority thread (dare I say "real-time"?), there's a good 
chance the kernel buffers are emptied quick enough.


It's rather straightforward to write a program doing this with POSIX 
threads, but why bother if it's already available? After all, I can't be 
the first one in the world with this need.


My own search lead me at best to cstream, which doesn't look like it's 
tailored for solving realtime issues.


Ideas, anyone?


   Eli


-- 
Web: http://www.billauer.co.il