R Shiny Collapsible Boxes in sidebar panel / Sidebar panel with multiple collapsible boxes over full size

Question

I want to create a Shiny app, which consists of different openable boxes on one side. In the boxes should then be buttons etc..

I have used the following solution so far:

library(shinythemes)
library(shinyBS)

fluidPage(
  theme = shinytheme("cosmo"),
  titlePanel(# app title/description
    "ShinyApp"),
  sidebarLayout(
    mainPanel(plotOutput("plot1")),
    
    sidebarPanel(
      bsCollapsePanel(
        "Color Selection",
        actionButton("f1blue", "Blue"),
        actionButton("f1red", "Red"),
        actionButton("f2blue", "Blue"),
        actionButton("f2red", "Red"),
        style = "success"
      ),
      
     
    )
   
  )
)

Here is a visualization: I would like to avoid this "box within the box". This is what my current code generates:

And this is how it should look: Only collapsibles, no border and no margin through the sidebar panel:

Unfortunately this solution creates only a box in the sidebar panel. I would like to use collapsible boxes, which are as big as the sidebar panel or replace the panel with several boxes standing under each other. What should I do to achieve this?

Thanks a lot in advance!