I want to gulp concat the same file multiple times but {'nounique': true} is not working

Question

I want to use gulp to concat some html files in order, and some of them are duplicates.

For example:

gulp.task('Generate HTML', function(cb) {
  const srcset = [
    'header.html',
    'content.html',
    'section2.html',
    'banner-cat.html',
    'section3.html',
    'banner-cat.html',
    'section4.html',
    'content.html',
    'content.html',
    'banner.html',
    'footer.html'
  ];

  gulp.src(srcset, {'allowEmpty': true, 'nounique': true, 'nosort': true})
    .pipe(concat('index.html'))
    .pipe(gulp.dest('public/'));
    
  cb();
});

I found the options no-unique and no-sort in the documentation so I thought that would solve my problem, however I am not getting the desired result, as every html snippet is only concat-ed one time, so any duplicated snippets will only show up once.

Please help me with any advice or if you know any other gulp package that can help me achieve this desired outcome using gulp and javascript

Answer 1

gulp.src treats its argument as a list of glob patterns rather than file paths. Each pattern could match one or multiple files or none at all, and if the same file is matched by multiple patterns, it's up to the implementation to decide what to do.

It's better here to load each file individually and then join the streams.

To use merge2, install the package with:

npm install -D merge2

Then your code could look like this:

const merge2 = require('merge2');

gulp.task('Generate HTML', function(cb) {
  const srcset = [
    'header.html',
    'content.html',
    'columns-left.html',
    'columns-right.html',
    'columns-left.html',
    'columns-right.html',
    'banner.html',
    'content.html',
    'content.html',
    'banner.html',
    'footer.html'
  ];

  const streams = srcset.map(src => gulp.src(src));
  merge2(...streams)
    .pipe(concat('generated-page.html'))
    .pipe(gulp.dest('public/'));
    
  cb();
});