Material-UI: Controls next to vertical ToggleButtonGroup

Question

Using Material UI with TS/React my goal is to have a vertical ToggleButtonGroup with controls next to the buttons which should be en- or disabled depending on the state of the ToggleButtons, similiar to this:

My initial thought was straight forward:

const MyCtrl = () => {
    const [enableMinWidth, setEnableMinWidth] = useState(true);
    const [enableMaxWidth, setEnableMaxWidth] = useState(false);

    return <ToggleButtonGroup orientation="vertical">
        <ToggleButton
            size="small"
            value="enableMinWidth"
            selected={enableMinWidth}
            onChange={setEnableMinWidth(!enableMinWidth)}>
            <Compress fontSize="small" sx={{ transform: "rotate(90deg)" }} />
        </ToggleButton>
        <ToggleButton
            size="small"
            value="enableMaxWidth"
            selected={enableMaxWidth}
            onChange={setEnableMaxWidth(!enableMaxWidth)}>
            <ExpandIcon fontSize="small" sx={{ transform: "rotate(90deg)" }} />
        </ToggleButton>
    </ToggleButtonGroup>;
};

However this way I don't see a way to properly position controls next to the ToggleButtons.

What would the ideal approach be to properly align the other controls, in this case a Slider and TextField? By "properly" I mean not eye-balling margins or paddings or anything of the sorts.