Dealing with and printing large text files

Question

I'm working on a corporate art project that requires me to:

1. Take every line of code in our git repo (hundreds of thousands) and join them into a single line in a single file. - This has been completed but the file is quite large (122mb)
2. That file needs to be converted into a print shop-ready format (PDF, EPS, etc) in four very large (3' x 8' / 300dpi) posters.

Here's what I've tried:

• Illustrator (copy/paste crashes)

• InDesign (crashes)

• Acrobat (crashes)

• PHP (I can get the file to display on a large enough webpage but any kind of print-to-pdf, etc just makes my browser die - See code below)

-

$READ_LEN = 64*1024; 
$MAX_BUF_LEN = 2*$READ_LEN; 

$url = "../thisisit.txt"; 
$handle = fopen($url, "r") or die("Couldn't get handle");
if ($handle) {
    while (!feof($handle)) {
        $buffer = fgets($handle, 4096);
        // Process buffer here..
        echo htmlentities($buffer);
    }
    fclose($handle);
}

My questions are as follows:

• Is there a better way to deal with a text file of this size?

• Is there a way to stream the output of the text file (via a shell command like cat) into a PDF wrapper directly?

• Am I insane for attempting to do this?

Answer 1

I did a short test for you with XSL FO (using RenderX XEP). I am fairly confident it could BUT I believe you need to do some calculations. I created a page that is of the size you state, no margins at all with some text in it (about 3.2MB). Formatting that at 1pt font-size takes about 1/6 of the 96inx36in page size.

The XSL FO is simple:

<fo:root xmlns:fo="http://www.w3.org/1999/XSL/Format" 
    font-family="Courier" font-size="1pt">
    <fo:layout-master-set>
        <fo:simple-page-master master-name="backgrad" page-width="8*12in" page-height="3*12in">
            <fo:region-body/>
            <fo:region-before extent="11in"/>
        </fo:simple-page-master>
    </fo:layout-master-set>
    <fo:page-sequence master-reference="backgrad">
        <fo:flow flow-name="xsl-region-body">
            <fo:block>
                hjdskjhdkh dskjhdskjhsd kjsdkjsd jksd ksdkjsdkj 
    <!-- snipped -->

You do need 64bit machine and 64bit Java with about 2GB memory. I did another test of 23MB and it nearly filled the page of the PDF. You could probably fit it at 0.7 or 0.8pt font.

Here's a sample 28MB text file @ 1pt font -- it spills to two pages. HINT: DO not use the browser to view it, it cannot handle this PDF even though it is only 127kb. Download and use Adobe Reader. HINT2: If you do download and open it, it may look like a big black blob. You need to zoom in to about 400% at least to start seeing things. It is 1/72inch font on 36inch high paper or about 2500 lines of text per page. Poster PDF