<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html style="direction: ltr;">
<head>
<meta content="text/html;charset=UTF-8" http-equiv="Content-Type">
</head>
<body style="direction: ltr;" bgcolor="#ffffff" text="#000000">
<p style="margin-bottom: 0cm; margin-top: 0pt;">Thanks for that one.
"yum install buffer". How simple.</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;"><br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;">So I tried it out:</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;"><br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;">$ dd if=/dev/zero bs=1M
count=256 > /dev/null<br>
256+0 records in<br>
256+0 records out<br>
268435456 bytes (268 MB) copied, 0.0258608 s, 10.4 GB/s</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;"><br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;">Original throughput is
10.4 GB/sec. Looks not so bad.<br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;"><br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;">$ dd if=/dev/zero bs=1M
count=256 | cat > /dev/null</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;">256+0 records in<br>
256+0 records out</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;">268435456 bytes (268
MB) copied, 0.199799 s, 1.3 GB/s</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;"><br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;">The regular cat is
somewhat slower, then.</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;"><br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;">Now let's try buffer:<br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;"><br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;">$ dd if=/dev/zero bs=1M
count=256 | buffer -m 256m > /dev/null<br>
Cannot handle that many blocks, aborting!</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;"><br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;">Hmmm... The default
buffer size is too small, so asking for 256MB didn't work well. Let's
ask for 128kByte buffers:<br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;"><br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;">$ dd if=/dev/zero bs=1M
count=256 | buffer -s 128k -m 256m > /dev/null</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;">256+0 records in<br>
256+0 records out<br>
268435456 bytes (268 MB) copied, 0.379354 s, 708 MB/s<br>
<br>
On one hand, it's not all that impressive that the rate went down. But
it's still way above the 100 MB/sec needed. On the other hand, going</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;"><br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;">$ dd if=/dev/zero bs=1M
count=256 | buffer -s 128k -m 256m | hexdump -C > /dev/null</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;"></p>
<p style="margin-bottom: 0cm; margin-top: 0pt;"><br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;">gives exactly the same
result, except that command prompt returns after a second or so, which
is the time it took for hexdump to consume the data.</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;"><br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;">So I suppose we have a
winner. Thanks again.<br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;"><br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;"> Eli<br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;"><br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;"><br>
</p>
<p style="margin-bottom: 0cm; margin-top: 0pt;">Baruch Even wrote:</p>
<blockquote
cite="mid:CAC9+anL7_8hXuegnnRKBz+Zd-5FDKqxPTY8QWTrjJkUOj3VE1A@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div class="gmail_quote">On Sat, Feb 4, 2012 at 4:27 PM, Eli Billauer
<span dir="ltr"><<a moz-do-not-send="true"
href="mailto:eli@billauer.co.il">eli@billauer.co.il</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Hi
all,<br>
<br>
<br>
I need a simple command-line program, which works as a plain FIFO
stream buffer with a huge RAM. Something I can do:<br>
<br>
<br>
$ fatcat -b 256M /dev/datasource | ./my_shaky_data_sink<br>
<br>
<br>
The idea is that "fatcat" reads data whenever available and stores it
to non-swappable RAM. It then pushes the data to stdout. So it's just
like good old "cat", only with a potentially large tummy.<br>
</blockquote>
<div><br>
</div>
<div>I've used pv or buffer for such purposes.</div>
<div><br>
</div>
<div>Baruch</div>
</div>
</div>
</div>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Web: <a class="moz-txt-link-freetext" href="http://www.billauer.co.il">http://www.billauer.co.il</a>
</pre>
</body>
</html>