Before you declare me an insane old man, I have to say, that I needed it for an embedded board with only 64MiBs of RAM. And (mostly due to the specifics of the tasks), I ended up using bash hooked to lighttpd via cgi. Bringing on heavy artillery (php or python), really complicated the task, since I mostly needed the outputs of different shell utils.
Anyway, I quickly got to the part where I needed to handle POST requests to upload binary files. First, I googled for a solution, and one of them eventually did what I needed. But alas, it had a drawback.
If you have a look at the source from the link
if [ "$REQUEST_METHOD" = "POST" ]; then TMPOUT=/tmp/fwupdate cat >$TMPOUT # Get the line count LINES=$(wc -l $TMPOUT | cut -d ' ' -f 1) # Remove the first four lines tail -$((LINES - 4)) $TMPOUT >$TMPOUT.1 # Remove the last line head -$((LINES - 5)) $TMPOUT.1 >$TMPOUT # Copy everything but the new last line to a temporary file head -$((LINES - 6)) $TMPOUT >$TMPOUT.1 # Copy the new last line but remove trailing \r\n tail -1 $TMPOUT | perl -p -i -e 's/\r\n$//' >>$TMPOUT.1 fi |
You will see that a temporary file is used. And that pretty much sucks, since my flash media on that board was horribly slow, I didn’t want to waste erase cycles, nor I had enough RAM to handle all of that in tmpfs. Ideas? Here goes my implementation of that… thing.
#!/bin/bash OIFS="$IFS" read boundary read disposition read ctype read junk #Holy fuck, this sucks! #Due to \n\r line breaks we have 2 extra bytes per line read, #6 + 2 newlines == 10 junk bytes a=${#boundary} b=${#disposition} c=${#ctype} a=$((a*2+b+c+d+10)) IFS="${IFS}&:" set $QUERY_STRING dir=$1 file=$2 SIZE=$((HTTP_CONTENT_LENGTH-a)) echo "Content-Type: text/html" echo "" echo "Upload complete, $SIZE bytes stored<br>" echo "Written to $dir, filename $2 ($dir/$file)<br>" dd ibs=1 obs=512 count=$SIZE of=$dir/$file |
That’s it! No temporary files, no need to buffer the whole file in memory, like PHP does. Just one plain dd and some utter voodoo.
No POST variables can be handled, so instead I pump two variables via GET, separated by a ‘:’.
In this raw way it’s pretty much a hell of a security hole, but since the thing I’m working on will unlikely ever be exposed to the internets, and online only when there’s a user nearby – why not?
Hi, thanks for the tips!
Here’s how I do it, with a POST form upload (slight variation of your script) :
echo "Content-type: text/html"
echo ""
echo ""
cat header.html
maintenance
debugmsg "HTTPS=$HTTPS"
if [[ "$REQUEST_METHOD" == "POST" ]]; then
debugmsg "CONTENT_LENGTH=$CONTENT_LENGTH"
if [[ $CONTENT_LENGTH -gt 0 && $CONTENT_LENGTH -le $MAX_FILE_UPLOAD_SIZE ]]; then
echo "Uploading file, please wait ..."
# single file upload
# we must strip the first 4 lines (http headers) of received data, and the last 2 at the end
read hline1 </dev/stdin && debugmsg "$hline1"
read hline2 </dev/stdin && debugmsg "$hline2"
read hline3 </dev/stdin && debugmsg "$hline3"
read hline4 </dev/stdin && debugmsg "$hline4"
# calculate remaining extra bytes to strip at the end of data (boundary string + extra cr/lf)
size=$(( ${#hline1} + 5 ))
outfile="uploads/tmpfile"
# read remaining data from stdin, except the last $size bytes (http headers)
head -c -$size "$outfile"
if [[ $? -eq 0 ]];then
echo "File has been uploaded successfully"
else
echo "Error : Unable to write file!"
fi
else
echo "File is too big for uploading, maximum allowed file size is $MAX_FILE_UPLOAD_SIZE bytes"
fi
else
echo "Error: Invalid method"
fi
@Fredo: Nice, thanks, looks like a cleaner solution.
For info, I noticed that some http clients do not work well with this method : the native web browser on Android 2.3.6 on my smartphone, for exemple (badly formatted http headers, although I’m not sure if they are really breaking the rules, I would need to read the RFC’s!).
But Firefox Mobile and Opera on the same platform work OK 🙂