Web api routing use only RoutePrefix

Question

I have default routing in my webapi which looks like

 config.Routes.MapHttpRoute(
                "Web API Data type action",
                "api/v{Version}/{siteId}/{controller}"
                );

Now i want to add routing to my controller which will be different something like

api/v{Version}/{controller}

So what i did i added

[RoutePrefix("api/v{Version}/{controller}")]

It works, but i can access my controller also by default routing. IS this any way to to limit routing if controller has routeprefix then use only this one. I also have idea to devide my controller to 2 namespaces and then i can limit in routing by namespace