In Rust, I have a large number of receiver objects I'd like manage, however I'm running into lifetime issues using Select

Question

Due to the possibility of there being a large number of objects, I'd like to have a way to add them to the select list, remove them for processing and then add them back. All, without having to rebuild the select list each time an object is added back for waiting. It looks something like this:

use std::collections::HashMap;
use crossbeam::{Select, Sender, Receiver};

struct WaitList <'a> {
    sel: Select<'a>,
    objects: HashMap<u128, Object>,
    sel_index: HashMap<usize, u128>,
}
impl<'a> WaitList<'a> {
    fn new () -> Self { Self { sel: Select::new(), objects: HashMap::new(), sel_index: HashMap::new() } }
    fn select(&self) -> &Object {
        let oper = self.sel.select();
        let index = oper.index();
        let id = self.sel_index.get(&index).unwrap();
        let obj = self.objects.get(&id).unwrap();
        obj.cmd = oper.recv(&obj.receiver).unwrap();
        self.sel.remove(index);
        obj
    }

    fn add_object(&self, object: Object) {
        let id = object.id;
        self.objects.insert(id, object);
        self.add_select(id);
    }

    fn add_select(&self, id: u128) {
        let idx = self.sel.recv(&self.objects.get(&id).unwrap().receiver);
        self.sel_index.insert(idx, id);
    }
}

Over time the select list will contain more dead entries, then live, and I'll rebuild it at that time. But, I'd like to not have to rebuild it every time. Here's the detailed error message:

    Checking test-select v0.1.0 (/Users/bruce/Projects/rust/examples/test-select)
error[E0495]: cannot infer an appropriate lifetime for autoref due to conflicting requirements
  --> src/main.rs:28:47
   |
28 |         let idx = self.sel.recv(&self.objects.get(&id).unwrap().receiver);
   |                                               ^^^
   |
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 27:5...
  --> src/main.rs:27:5
   |
27 | /     fn add_select(&self, id: u128) {
28 | |         let idx = self.sel.recv(&self.objects.get(&id).unwrap().receiver);
29 | |         self.sel_index.insert(idx, id);
30 | |     }
   | |_____^
note: ...so that reference does not outlive borrowed content
  --> src/main.rs:28:34
   |
28 |         let idx = self.sel.recv(&self.objects.get(&id).unwrap().receiver);
   |                                  ^^^^^^^^^^^^
note: but, the lifetime must be valid for the lifetime `'a` as defined on the impl at 9:6...
  --> src/main.rs:9:6
   |
9  | impl<'a> WaitList<'a> {
   |      ^^
note: ...so that the types are compatible
  --> src/main.rs:28:28
   |
28 |         let idx = self.sel.recv(&self.objects.get(&id).unwrap().receiver);
   |                            ^^^^
   = note: expected `&mut crossbeam::Select<'_>`
              found `&mut crossbeam::Select<'a>`

While I believe I understand the issue, that the borrow of the receiver from the hash table doesn't live long enough, I'm having a difficult time trying to come up with an alternative -- and I'm not seeing a clean way to borrow the information. I considered creating a struct to contain the borrow, and using that instead of a plain id in the wait sel_index however that runs into the same lifetime problem.

struct SingleWaiter<'a> {
    id: u128,
    receiver: &'a Receiver::<Command>
}

I feel like I'm missing something or not understanding something, as it seems like it shouldn't be that difficult to do what I want to do. I can imagine that the choice of HashMap for holding object might be the issue, a Vec felt wrong, as I'm adding and inserting. BTW, the HashMap isn't normally part of the waitlist. It is part of something else, but the problem remains the same irregardless of where the HashMap lives.