Dealing with securing an ICE server to allow only allowed IP addresses to access it

Question

I am trying to figure out the best way to secure an ICE server for WebRTC calls. At the moment, the server allows all traffic through the necessary ports (say, 4000:8000) that are required for Coturn to work. Currently, when a source user calls a target user, the negotiation takes place directly with the ICE server:

I would like to ensure that that ports 4000:8000 are available only to the source and target user, not to the outside world. To do this, I've shut down all ports, including 4000:8000 with ufw and spun up a small Nodejs server script on the same ICE server script on a different port (say, 3000). I've opened this port up to my backend server's IP address. In other words, only one port (other than port 22) is opened, and the only traffic allowed in is from my backend.

With this in mind, I've come up with a "hacky" way to ensure that the source user and target user can access my ICE server. When the source user initiates the call, a request is made to the backend, which, knowing the IPs of both users, in turn sends a request to port 3000 to my ICE server. Then, the ICE server dynamically creates rules for these two IP addresses, allowing access to ports 4000:8000. Upon success, the target user then tries to directly access the ICE server (as in the first diagram) and, ultimately, create a negotiation with the target user:

Upon disconnection, a similar request is made from my backend to the ICE server to remove these IPs from the list of firewall rules. How do I dynamically update these rules?

// creating rules
const { execSync } = require('child_process');

const addToFw = async ({ ips }) => {
    // in the diagrams, I use 'sourceUserIpAddress' and 'targetUserIpAddress', but
    // here it's just an `string[]` of IPs. Same idea, though.
    try {
        for (let i = 0; i < ips.length; i++) {
            const ip = ips[i];
            if (ip) {
                execSync(
                    `echo "y" | ufw allow from ${ip} to any port 3478`
                );
                execSync(
                    `echo "y" | ufw allow from ${ip} to any port 4000:8000 proto tcp`
                );
                execSync(
                    `echo "y" | ufw allow from ${ip} to any port 4000:8000 proto udp`
                );
            }
        }
    } catch (err) {
        console.log('Error in addToFw', err);
    }
};

// removing rules
const { execSync } = require('child_process');

const getResults = (ip) =>
    execSync(
        `ufw status numbered |(grep '${ip}'|awk -F"[][]" '{print $2}')`,
        ''
    )
        .toString()
        .split('')
        .filter(Number);

const removeRulesForIp = (ip) => {
    let results = getResults(ip);

    const occurences = results.length;
    for (let i = 0; i < occurences; i++) {
        let ruleNumber = null;
        if (i === 0) {
            ruleNumber = results[0];
        } else {
            results = getResults(ip);
            ruleNumber = results[0];
        }

        // should always be defined
        if (results[0]) {
            execSync(`echo "y" | sudo ufw delete ${results[0]}`);
        }
    }
};

const removeFromFw = ({ ips }) => {
    try {
        for (let i = 0; i < ips.length; i++) {
            const ip = ips[i];
            if (ip) {
                removeRulesForIp(ips[i]);
            }
        }
    } catch (err) {
        console.log('Error in removeFromFw', err);
    }
};

Why am I doing let results = . . . and using execSync? I've noticed that if I use an asynchronous approach and Promise.all, then ufw seems to not be able to handle that. I guess ufw can handle only one rule creation/deletion operation at once.

So, with all that out of the way, onto the actual question. Will this scale with hundreds of thousands of concurrent users? What if, say, 10,000 people simultaneously click the "call" button? Will ufw skip some requests to create/delete rules?

Also, I'm aware that I can store (in a database on my backend) and check whether or not a user's IP has changed since their last call (and remove the old IP address from my ICE server only in the event that it did), but let's say I don't do that. Are there any caveats?

Is this a feasible solution and approach to the problem or is there a better way? Or perhaps this is a feasible approach, but there are some things that I should change.