I am very new to ASP.NET Core.
If I create 4 projects like this:
MyApplication
|-- MyApplication.DataAccess
|-- MyApplication.Provider
|-- MyApplication.Service
|-- MyApplication.ViewModel
DataAccess project is for getting data from the database, Provider for the queries, Service project for business logic, and lastly ViewModel for the data representation.
The dependency is like this:
MyApplication.DataAccess -> MyApplication.Provider -> MyApplication.Service
and also:
MyApplication.ViewModel -> MyApplication.Service
My target is to later create an application named MyApplication.Web.UI for the Razor pages.
I am very confused on how to create a dependency injection in the case of class library in C#. Is there any way for me to use DI Service or is it better to use using scope for my DBContext?
And lastly, is there a way where I can convert the DataAccess models into view models in the Service project without making the Service project depending on the DataAccess project?

You don't want your db context to "bleed" outside of your Provider project (that's what your ViewModel project is for).
Use Microsoft.Extensions.DependencyInjection with an interface registration for each service in your Provider and Service projects. The usual pattern is for implementations of those interfaces to remain internal to the project and add a public extension method in the project which registers the service interface using the internal implementation such as this:
Then your web project (or other client) takes a reference to the project and configures its DI container using the extension method (setup is different for different clients [ASP.Net for example has a specific place where services are to be registered, you don't create the service collection or build the service provider], but the gist is this...)
The result of all this is that you have service interfaces which can be easily unit tested (by replacing dependencies with a framework such as Moq) and your implementation remains suitably encapsulated. Changing to a different implementation for a service now just entails providing a suitably named alternative extension method which supplies a different implementation class to the DI container and the client systems are unaware of the change: a clean separate of concerns.