Both pyTivo and TiVoDotNet suffer the same problem with transfers.  The result is that files which are transfered but not watched are deleted instantly as soon as they finish.  The problem is caused by how we transfer files. 

When TiVo asks for a XviD file from pyTivo TiVo asks for the file size of the file before it starts transfering.  This of course raises an issue, since pyTivo is transcoding the file realtime pyTivo has no idea how large he file will be when the transcoding is done.  I have tried numerous settings and I can’t figure out a command to force FFMpeg to an exact bit rate.  So we usually estimate the file size.

There is also an additional problem with file size estimation.  Estimating the file size too small will cause files to be clipped at the end.  Estimating the file to large may cause TiVo to unnecessarily delete other videos to make room for a file that is not that large.  Also, when a user reached the end of the video TiVo would display an error stating that the transfer had unexpectedly ended even though we have reached the end of the video.

However if pyTivo estimates the file even 1 bit too large(pyTivo never gets it exact, we always have a 2% buffer to prevent the under size issue) TiVo will continue asking for bytes that dont exist.  This really messes with TiVo.  So our temporary hack was to return a 404 error when TiVo asked for the overage of the file.  However this doesn’t appear to be perfect, most of us suffer the early deletion problem discussed above.

So I spent a weekend on this and solved the issue!!

First here is a breakout of what a request for a file looks like:

1. TiVo asks for the information about a video
   1. This request is in the form of a webpage request
2. pyTivo sends back a file size that is as large as the file could possibly be (Example 1Gig)
   1. This data is returned in an XML formated page
3. TiVo sends a get request for the Video
   1. Again this is sent as an HTTP GET request
4. pyTivo streams the video as it transcodes
   1. During this time pyTivo does the following:
      1. Determines based on user settings and the dimensions of the file the correct AspectRatio and padding
      2. Pipes the output of ffmpeg directly into TCP packets sent to TiVo
5. Stream abrubtly ends at the end of the video (Example 800MB)
   1. FFMpeg has reached the end of the file and transcoding is complete. However because the size of the trancoded file always varies the end of file comes before the total number of bytes TiVo is expecting.
   2. So at this point TiVo is still expecting another 200MB
6. TiVo waits thinking that network traffic has slowed down the packets, TiVo seems to wait 30 seconds or so.
7. TiVo then requests the file again asking for the last 200MB.
   1. This request is sent in a second HTTP GET request, however this time TiVo request the range of the file it is still missing. In this case that would be 800MB – 1000MB.
8. pyTivo responds with a 404 and TiVo stops requesting the file

So I realized that this was a bad solution.  So I went and looked into a 206 response, or partial content.  The 206 RFC can be found here.

This response tells TiVo that there is no more file coming.  It lists the range of bytes which are empty.

This alone worked.  It stopped the error from occuring at the end of playing a file.  But for some reason the file would still get deleted earlier than it was supposed to.  So I tinkered more.

What I discovered is that TiVo needs and EOF response in the 206 as well.  This would seem to go without saying, but apparently not.  So by addint the Hex codes 300D0A to the body of the 206. TiVo would properly end the file.

And that did it!! Everything worked perfectly after that.  So replace step 8 above with the following and everything works perfectly

8.  pyTivo responds with a 206 which tells TiVo that the file has ended and sends a short bit of code which I believe equals and End of File line.

1. pyTivo receives the request and sees that the byte request is beyond the length of the file and is out of range.
2. pyTivo then responds with an HTTP 206 response, partial content.
   1. This response contains headers stating that there is no more file and to close the connection.
   2. It tells TiVo specifically the range of bytes which it can forget about, in this case 800MB – 1000MB
   3. It sends back three hexadecimal characters in the body of the TCP packet. These are "300D0A". I have no idea what this means, I only saw it in responses from TiVo Desktop and copied it. My assumption is that this is an End of File signal?? Without it TiVo will still recognize that the transfer was not interrupted, but the file will be deleted from the TiVo immediately, which makes me think that TiVo interprets the lack of those characters as a corrupt file.

The actual python code looks like this:

if handler.headers.getheader(‘Range’) and not handler.headers.getheader(‘Range’) == ‘bytes=0-‘:
  handler.send_response(206)
  handler.send_header(‘Connection’, ‘close’)
  handler.send_header(‘Content-Type’, ‘video/x-tivo-mpeg’)
  handler.send_header(‘Transfer-Encoding’, ‘chunked’)
  handler.send_header(‘Server’, ‘TiVo Server/1.4.257.475’)
  handler.end_headers()
  handler.wfile.write("\x30\x0D\x0A")
  return