Use a maximum of 2 colors from a given palette to update a list of any RGB pixels

Question

I need to mimic the way GIF images work: I have a list of 6 RGB pixels, an "allowed colors" palette (RGB, too) and a maximum number of 2 different colors I can pick from that palette. How can I update my list of RGB pixels to look as close as possible as the source?

Here is the (very basic) palette:

interface RGBColor {
  r: number;
  g: number;
  b: number;
}

const palette: RGBColor[] = [
  { r: 0, g: 0, b: 0 },       // Black
  { r: 255, g: 255, b: 255 }, // White
  { r: 255, g: 0, b: 0 },     // Red
  { r: 0, g: 255, b: 0 },     // Green
  { r: 0, g: 0, b: 255 },     // Blue
  { r: 255, g: 255, b: 0 },   // Yellow
  { r: 0, g: 255, b: 255 },   // Cyan
  { r: 255, g: 0, b: 255 },   // Magenta
];

It might seem obvious at first sight, but I struggle to get it working properly. I thought about finding the closest color for every pixel, then keeping only the two most used, and updating the other ones with the closest from the 2 I kept, but I'm sure there's something "visual" I'm missing.

For example, we'd probably want a single black pixel with 2 yellows and 3 whites to still be there after conversion, because it needs to stand out more.

This is a TypeScript project, but any language/algorithm can help.