Why isn't HummusJS displaying pdf fill values in Acrobat Reader (but does in browser)?

Question

Using HummusJS to fill government pdf forms I cannot figure out why the filled values do appear when I open the filled pdf forms in Chrome/Firefox but do not appear when I open them in Acrobat Reader.

Using filling-form-values and modifying the main.js code for the fields to be filled and an example I-130 pdf form:

var hummus = require('hummus'),
fillForm = require('./pdf-form-fill').fillForm;

var filename = 'i-130.pdf';
var writer = hummus
.createWriterToModify(__dirname + '/sample-forms/' + filename, {
    modifiedFilePath: __dirname + '/output/' + filename + '_out.pdf'
});

var data = {
    "form1[0].#subform[0].Pt2Line4a_FamilyName[0]" : "LAST filled",
    "form1[0].#subform[0].Pt2Line4b_GivenName[0]" : 'FIRST filled ',
    "form1[0].#subform[0].Pt2Line4c_MiddleName[0]" : 'MIDDLE filled',
}

fillForm(writer,data);
writer.end();

Here's how it looks in Chrome with the correct filled values (no concerns on alignment - I know how to fix that) :

And here's how it appears in Acrobat Reader - sad empty fields :

If I parse the filled PDF using HummusJS Samples parsing-form-values, the values are there in those filled fields. eg:

{
  "name": "Pt2Line4a_FamilyName[0]",
  "fullName": "form1[0].#subform[0].Pt2Line4a_FamilyName[0]",
  "alternateName": "Part 2. Information About You (Petitioner). Your Full Name. 4. A. Enter Family Name (Last Name).",
  "isNoExport": false,
  "isFileSelect": false,
  "type": "plaintext",
  "value": "LAST filled"
},
{
  "name": "Pt2Line4b_GivenName[0]",
  "fullName": "form1[0].#subform[0].Pt2Line4b_GivenName[0]",
  "alternateName": "Part 2. Information About You (Petitioner). Your Full Name. 4. B. Enter Given Name (First Name).",
  "isNoExport": false,
  "isFileSelect": false,
  "type": "plaintext",
  "value": "FIRST filled "
},
{
  "name": "Pt2Line4c_MiddleName[0]",
  "fullName": "form1[0].#subform[0].Pt2Line4c_MiddleName[0]",
  "alternateName": "Part 2. Information About You (Petitioner). Your Full Name. 4. C. Enter Middle Name.",
  "isNoExport": false,
  "isFileSelect": false,
  "type": "plaintext",
  "value": "MIDDLE filled"
},

I've tried running this on both Windows and Linux with the same result.

The values don't appear in Acrobat Reader even if you click into those fields.

I've also tried the loading fonts to rule that out with same result.

var font = handles.writer.getFontForFile(__dirname + '/arial.ttf');
xobjectForm.getContentContext()
.BT()           // Begin Text writing
.k(0,0,0,1)     // set color to Black (cmyk = 0,0,0,1) 
.Tf(font,10)    // set font; size 20
.Tm(1,0,0,1,10,10) // set position to 0,40 in this object
.Tj('YO YO YO!!')   // set text
.ET()               // end text writing

Anyone have any ideas or pointers?

Answer 1

The cause for the issue is that the PDF form is a hybrid AcroForm/XFA form. HummusJS (at least the version the OP used) apparently only recognizes and changes the AcroForm form definition. Most non-Adobe PDF viewers also only recognize the AcroForm form definition which is why Chrome and Firefox show your changes: they show the AcroForm form. Adobe Reader on the other hand prefers the XFA form definition and shows the unchanged XFA form.

Backgrounds

AcroForm form definitions are the native form definitions in PDFs. Form labels, explanations, and other fixed parts are drawn as regular page content and the visible form fields themselves are widget annotations on the respective PDF pages. Form values are stored in PDF objects.

XFA form definitions are XML streams defining fields, static texts, lines, images, decorations, etc. They can be embedded in a PDF and use the PDF merely as a transport container (making people think they have a "normal" PDF document and making computers opening them in their standard PDF viewer). Form values are stored in the XML.

Hybrid form PDFs contain both an AcroForm and a XFA form definition which appear and behave more or less identical.

In comparison XFA form definitions can be more flexible than AcroForm form definitions; e.g. they can have variable length tables. In hybrid forms, though, they cannot make much use of that greater flexibility as they are intended to appear the same as they do in the AcroForm variant.

AcroForm form definitions are specified in the PDF specification ISO 32000 itself. For XFA form definitions the PDF specification only specifies the object (the target dictionary and key therein) in which to store the form definition and references an external, proprietary document that specifies the contents of the form definition.

Beginning with PDF 2.0 XFA forms are deprecated in PDFs, cf. ISO 32000-2:

XFA stream or array (Optional; deprecated in PDF 2.0) A stream or array containing an XFA resource, whose format shall conform to the Data Package (XDP) Specification.

See Annex K, “(normative) XFA forms”.

(Table 224 — Entries in the interactive form dictionary)

XFA forms were deprecated with PDF 2.0.

(Annex K (normative) XFA forms)

The PDF specification explicitly allows PDF processors not to support XFA forms:

The implementation of such a schema driven page generation involves considerable effort beyond that for a simple PDF viewer and therefore a PDF processor may choose to not implement this feature.

(Annex K (normative) XFA forms)

Most PDF viewers do not implement XFA form processing (and, therefore, only store values in the AcroForm form definition), the obvious exceptions being Adobe Acrobat (Reader).

Adobe Acrobat (Reader), when confronted with a hybrid form, favors the XFA definition and displays it. It stores values in both form definitions.

Work-around

In case of hybrid form definitions the best option usually is to remove the XFA form definition. Furthermore, hybrid form PDFs often are signed with a usage rights signature to make Adobe Acrobat Reader allow saving of filled-in XFA forms. After removing the XFA form one should also remove the usage rights signature.

I don't know whether that is possible with HummusJS, let alone how. But even if it is not possible, one can do this in some other software before and add the cut down PDF to one's project.