How to create inline-records (IRRE) using DataHandler in TYPO3?

Question

In our case we have a table A which contains IRRE records using table B. Inside a backend module we import an XML file to import those records for table B.

All records/data for table A is available. All data for table B is available, except the new uids/identifiers.

Based on https://docs.typo3.org/typo3cms/CoreApiReference/6.2/ApiOverview/Typo3CoreEngine/Database/ I have to set the identifier NEWxxxx for all new created records.

I'm importing a large number of records at once. Can I generate those identifiers in a loop and process all records at once or do I have to run the whole datamap-handling record by record ?

Besides the identifier, is there any field i have to set on the parent record which contains the IRRE record ?

No translations/workspaces/other relations are involved.

Thanks for your help.

Answer 1

The DataHandler in TYPO3 is using the following array structure to create new or update existing records - this is valid up to and including TYPO3 CMS 8:

$dataMap = ['<table-name>' => [
    '<record-uid>' => ['<field-name>' => '<field-value>']
];

Existing records use the integer value of the record's uid field, e.g. 123, new records use some random but unique identifier that are prefixed with NEW, e.g. NEWa2b3c4f8 created by uniqid('NEW', true) - since TYPO3 CMS 7 StringUtility::getUniqueId('NEW') can and should be used for that.

Generic Example

Let's assume the following records shall be created:

• a new content element in table tt_content
• two new inline file references for table sys_file_reference for field tt_content.image
  • referencing to existing sys_file record with uid 123
  • referencing to existing sys_file record with uid 234

---

// generating unique identifiers for records to be created
$ttContentId = 'NEW58d5079c8741c822627844'; // StringUtility::getUniqueId('NEW')
$fileRefId1st = 'NEW58d506f3cd0c4159344142'; // StringUtility::getUniqueId('NEW')
$fileRefId2nd = 'NEW58d50714c1226092562338'; // StringUtility::getUniqueId('NEW')

---

Preparing data-map

Hava a close look to tt_content.image, this is actually defining the (new) inline references, defined by a comma separated values of new records or existing records - this could either be NEWabc,NEWdef, 123,234,345 or NEWabc,123,NEWdef, mixing new and existing record references.

$dataMap = [
  'tt_content' => [
    'NEW58d5079c8741c822627844' => [
      'title' => 'My new content element',
      'bodytext' => 'Look at the following images...',
      'CType' => 'textpic',
      // $fileRefId1st & $fileRefId2nd, the sorting order is defined by this as well
      'image' => 'NEW58d506f3cd0c4159344142,NEW58d50714c1226092562338',
    ],
  ],
  'sys_file_reference' => [
    'NEW58d506f3cd0c4159344142' => [
      'uid_local' => 123,
      'title' => 'Image #123',
    ],
    'NEW58d50714c1226092562338' => [
      'uid_local' => 234,
      'title' => 'Image #234',
    ],
  ]
];

Preparing command-map

// the command-maps is similar to the data-map to copy, localize, move records
// however, it's not required in this scenario and thus stays empty
$commandMap = [];

Executing DataHandler

$dataHandler = new \TYPO3\CMS\Core\DataHandling\DataHandler();
$dataHandler->start($dataMap, $commandMap);
$dataHandler->process_datamap();
// $dataHandler->process_cmdmap(); // if $commandMap should be processed as well

If you need the uid of the created records, this can be resolved from the internal DataHandler record mapping. For example, the following code resolves the new uid of the created tt_content record:

// fetching the actual record ID, e.g. results in 333
$ttContentId = $dataHandler->substNEWwithIDs['NEW58d5079c8741c822627844'];

Notes

Defining the references happens in the example above directly for the field tt_content.image, which can contain NEW... ids as well as existing integer ids. The behaviour for the is the same for all reference types in TYPO3:

• TCA type inline, for all variants (plain, foreign_field, MM)
• TCA type select, for all variants (plain, MM)
• TCA type group, for all variants (plain, MM)

Passing data through DataHandler ensures that log entries are created, and the in most cases modifications can be reverted using TYPO3's history/rollback module.

Besides that, it's possible to execute mass actions - the invocation of DataHandler is not limited to just on aggregate (the tt_content record in the example above). However, the NEW... ids have to be unique and must not be re-used during mass-executions to avoid side-effects.

Transformed to table_a & table_b scenario

Transforming this to the table_a and table_b scenario of the initial question, the $dataMap might look like the following. Of course you have to determine which references to table_b are bound to table_a.

$dataMap = [
  // existing records of table_a, thus using the real ids
  'table_a' => [
    '11' => [ 'reference_field' => 'NEWb1,NEWb2' ],
    '22' => [ 'reference_field' => 'NEWb3,NEWb4' ],
    '33' => [ 'reference_field' => 'NEWb5,NEWb6' ],
  ],
  // new records to be references for table_b, thus using NEW... ids
  'table_b' => [
    'NEWb1' => [ ... field values of this particular table_b record ... ],
    'NEWb2' => [ ... field values of this particular table_b record ... ],
    'NEWb3' => [ ... field values of this particular table_b record ... ],
    'NEWb4' => [ ... field values of this particular table_b record ... ],
    'NEWb5' => [ ... field values of this particular table_b record ... ],
    'NEWb6' => [ ... field values of this particular table_b record ... ],
  ],
];
  

Notes on identifiers

Identifiers like NEWb1 are kept simple intentionally - usually those identifiers are composed by prefix NEW and a (pseudo-)random hexadecimal string abdc....

The TYPO3 core is using $id = StringUtility::getUniqueId('NEW') to create those unique identifiers. However, that also can be achieved using $id = 'NEW' . bin2hex(random_bytes(10); - identifiers just have to be unique for this particular process.