hostvars in playbook is not accessible in zuul while it works in ansible

Question

I am using include_vars to load variables from several .yaml files

- hosts: localhost

  vars:

    files: "{{ query('varnames', 'files_[0-9]+')|
               map('extract', hostvars.localhost, 'files')|
               flatten }}"

  tasks:

    - find:
        paths: "{{ playbook_dir }}"
        recurse: true
        patterns: test.yaml
      register: files_from_dirs

    - include_vars:
        file: "{{ item }}"
        name: "{{ name }}"
      loop: "{{ files_from_dirs.files|map(attribute='path')|list }}"
      loop_control:
        extended: true
      vars:
        name: "files_{{ ansible_loop.index }}"

    - debug:
        var: files

while this works in ansible when I run it in zuul it doesn't work. Either zuul protects hostvars for security reasons or it loads the vars in another namespace

is there a way to use another variable with include_vars instead of hostvars so I can have a reliable name handler to load the variables

for example something akin to (the code below doesn't work but I am trying to explain the concept)

 - local_vars: {
               'name': 'This acts like a pointer', 
               }

    files: "{{ query('varnames', 'files_[0-9]+')|
               map('extract', local_vars, 'files')|
               flatten }}"
   

and to load into that dictionary as keys, or another method where I can have a local var to point to those dictionaries without using hostvar

- include_vars:
        file: "{{ item }}"
        name: "{{ name }}"
      loop: "{{ files_from_dirs.files|map(attribute='path')|list }}"
      loop_control:
        extended: true
      vars:
        name: "local_vars.folders_{{ ansible_loop.index }}"

Answer 1

Q: "Is there a way to use another variable with include_vars instead of hostvars?"

A: Yes. It is. Create on your own a dictionary that will keep the variables. For example, given the data

shell> cat dir1/test.yaml 
files:
  - file1
  - file2

shell> cat dir2/test.yaml 
files:
  - file3
  - file4

Find the files

    - find:
        paths: "{{ playbook_dir }}"
        recurse: true
        patterns: test.yaml
      register: files_from_dirs
    - debug:
        msg: "{{ files_from_dirs.files|map(attribute='path')|list }}"

gives

  msg:
  - /export/scratch/tmp7/test-002/dir1/test.yaml
  - /export/scratch/tmp7/test-002/dir2/test.yaml

Iterate the list of files and combine the dictionary

    - set_fact:
        files: "{{ files|d({})|
                   combine({name: lookup('template', item)|from_yaml}) }}"
      loop: "{{ files_from_dirs.files|map(attribute='path')|list }}"
      loop_control:
        extended: true
      vars:
        name: "files_{{ ansible_loop.index }}"

gives

  files:
    files_1:
      files:
      - file1
      - file2
    files_2:
      files:
      - file3
      - file4

Process the dictionary as you like. For example, put the below declaration into the vars to obtain a flat list

files_list: "{{ files|json_query('*.files')|flatten }}"

gives

files_list:
  - file1
  - file2
  - file3
  - file4

---

Example of a complete playbook for testing

- hosts: localhost

  vars:

    files_list: "{{ files|json_query('*.files')|flatten }}"

  tasks:

    - find:
        paths: "{{ playbook_dir }}"
        recurse: true
        patterns: test.yaml
      register: files_from_dirs

    - debug:
        msg: "{{ files_from_dirs.files|map(attribute='path')|list }}"

    - set_fact:
        files: "{{ files|d({})|
                   combine({name: lookup('template', item)|from_yaml}) }}"
      loop: "{{ files_from_dirs.files|map(attribute='path')|list }}"
      loop_control:
        extended: true
      vars:
        name: "files_{{ ansible_loop.index }}"

    - debug:
        var: files

    - debug:
        var: files_list

---

The next option is creating a list instead of a dictionary. For example, the playbook

- hosts: localhost

  vars:

    files_list: "{{ files|json_query('[].files')|flatten }}"

  tasks:

    - find:
        paths: "{{ playbook_dir }}"
        recurse: true
        patterns: test.yaml
      register: files_from_dirs

    - debug:
        msg: "{{ files_from_dirs.files|map(attribute='path')|list }}"

    - set_fact:
        files: "{{ files|d([]) + [lookup('template', item)|from_yaml] }}"
      loop: "{{ files_from_dirs.files|map(attribute='path')|list }}"

    - debug:
        var: files

    - debug:
        var: files_list

gives

files:
  - files:
    - file1
    - file2
  - files:
    - file3
    - file4

files_list:
  - file1
  - file2
  - file3
  - file4