script to upload from URL

[shimi]

2011/12/13 Hetz Ben Hamo <hetzbh at gmail.com>

> Hi,
>
> I've written a simple bash script to upload a file from a remote server as
> a CGI script (yes, I know, I should use another language, but it's just a
> proof-of-concept).
>
> It goes like this: A simple HTML page gives the user a text line to enter
> a URL and "upload" button, which submits the data using POST to a bash
> script (I use the proccgi for transferring the values).
>
> The scripts fetches the URL and launches wget to grab the file, rename it
> and move it to a specific directory.
>
> So far, so good. The script works well.
>
> But I have one issue with it: those files are pretty big (1-3 GB) and wget
> doesn't show anything while it uploads - in the web browser. I tried using
> some redirect tricks, but it still doesn't show anything on the screen. I
> can redirect the output to a text file and show it after the upload, but it
> defeats the purpose of showing some activity.
>
> So my question: how can I make WGET (or CURL) show anything on my browser
> while it downloads the file (uploading it to the server)?
>
> Thanks,
> Hetz
>
>
>
I am assuming I understood you correctly (and will answer upon that
assumption) :

You have a server that gets a request to upload a file from the client (the
file is NOT uploaded by the client, the client just asks for the file to be
uploaded)  - then the server uses cURL via system() or similar to do the
actual upload - and you want the CLIENT to see the progess of the cURL
process in real-time or close to real-time.

If so, how about this:

1. Redirect cURL's output to a temp file via -o - and background it
2. As a response to the client's request containing the form filled with
the filename to upload [which you started the cURL for in #1], you'll
return a simple HTML page with a DIV placeholder to show the progress. In
the end of the page, you'll call a JS function to run call an XHR that will
call a SECOND URL with the temp file in a parameter (you'll think on how to
secure it, if needed) - a URL that will run a script on the same said
server to read the temp file from #1 and display it (or a tail -n 5 of it,
or whatever).
3. After a successful return of the XHR, use
document.getElementyById('name-of-div-of-status').innerHTML=XHRObject.responseText
4. Repeat 2-3 until XHRObject.responseText contains some magic string from
SECOND URL that says "OK, I finished"

If I didn't get you right... please tell me where I was wrong :)

Hope this helps,

-- Shimi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.huji.ac.il/pipermail/linux-il/attachments/20111213/25f8d8d1/attachment.html>