Get a set of files from multiple servers in a directory structure, manipulate the files, and put them all in one file

Question

I cannot figure this out because my bash skills aren't that strong and I haven't been able to find an existing script to do this after much googling.

I use pslurp to grab the /etc/passwd file from multiple servers. That puts the /etc/passwd in a directory structure /servername/password

pslurp -h serverlist -L localdir /etc/passwd password

Then I need to take each of those password files that I just grabbed and adjust them to only show the usernames of users with the ability to login interactively. I do that by excluding any users with a shell of /sbin/nologin for each of the /servername/password files and output to a new file

grep -v "/sbin/nologin$" password | cut -d: -f1 > /servername/newfile

Once I've got that, I'll need to get all of the /servername/newfile contents into 1 combined file with a format of

servername

list contents of /servername/newfile under each servername

servername

list contents of /servername/newfile under each servername

and so on ...

I've been working on this for a couple of days but just can't get it to happen. Please help!

Answer 1

So long as you have set up passwordless login by ssh to the servers to collect /etc/passwd from (or want to just enter the password for each when prompted), you don't need pslurp or any other canned utility, all you need is ssh and awk. For example to retrieve and combine all /etc/passwd files from a list of servers removing all entries that contain nologin or false (for /bin/false) and writing them out in a format of:

server1
list of login users

server2
list of login users

...

All you need is a simple array containing the server names and the following, e.g.

#!/bin/bash

servers=(server1 server2 server3)   # list of servers to get /etc/passwd from

{ # brace enclosed operation
for i in "${servers[@]}"; do        # loop over each server
    # ssh in and on success outut server name and contents without nologin or false
    ssh "$i" "printf '\n%s\n' '$i'; awk '/nologin/||/false/{next}1' /etc/passwd"
done
} > newfile                         # redirect output to new file

That will create newfile with the format and contents you describe without using any temporary files locally.

If you want the flexibility to redirect the output to stdout or any file of your choosing, just remove the brace-enclosed grouping an redirection within the script and redirect the output (or let it print to stdout) when you run the script.

There are endless ways to approach this, but this is one that is relatively straight-forward. For instance, if you do not have bash on the servers, but are limited to POSIX shell, you can use a simple list of servers instead of an array and the remainder would work as is.

Answer 2

Given some fake password files like this:

$ tree
.
├── svrA
│   └── passwd
├── svrB
│   └── passwd
└── svrC
    └── passwd

One awk command is sufficient:

awk -F: '
    FNR==1 {
        split(FILENAME, f, "/")
        printf "\n%s\n\n", f[1]
    }
    $NF !~ /\/(nologin|false)$/
' */*

outputs this, that you can redirect into a file

svrA

root:x:0:0:root:/root:/bin/bash
user1:x:1000:1000:Parallels:/home/user1:/bin/bash

svrB

root:x:0:0:root:/root:/bin/bash
user2:x:1000:1000:user2:/home/user2:/bin/bash

svrC

root:x:0:0:root:/root:/bin/bash
user3:x:1000:1000:user3:/home/user3:/bin/bash

Do you want to ignore /bin/false and /usr/bin/false as login shells as well?