GPT FileSystem after exiting boot services in C/C++

Question

I am creating an operating system and have strugled to find a way to implement a filesystem into it for the past several months. I can read files in boot services but after exiting boot services in UEFI the function can't read the file. I am trying to create a filesystem driver outside boot services using GPT (though if possible MBR but based on what I've seen near impossible). All source code and examples I've seen use multiboot with GRUB but I do not use the GRUB bootloader. Instead I followed the one from Poncho's OSDev 2 series. I have seen an example from WYOOS where he uses the MSDos partition system but it depends on multiboot therefore it doesn't work in my scenario. All help will be appreciated. Thanks.

Answer 1

You don't give much information in the question but I assume you want an hard-drive driver that will be able to read files from drives on a filesystem. In the end, you don't really implement a filesystem driver. You implement an hard-drive driver that will be able to read/write an hard-drive. The filesystem logic comes on top of that driver.

Below that, you want an ACPI interpreter and a PCI subsystem that will manage PCI devices. It is really complex and isn't just like calling a function that gets the job done. Once you got a PCI subsystem along with an ACPI interpreter and a filesystem driver, you got quite an advanced OS. This isn't going to be one function call. It requires planning and reading specifications.

With that said, I am currently writing an x86-64 OS myself and what I'm planning to do might be of interest (or not). From UEFI, I use the following to load my kernel from disk:

    EFI_GUID FSPGuid = EFI_SIMPLE_FILE_SYSTEM_PROTOCOL_GUID;
    EFI_HANDLE* Handles = NULL;   
    UINTN HandleCount = 0;
    Status = BS->LocateHandleBuffer(ByProtocol, &FSPGuid, NULL, &HandleCount, &Handles);
    if (EFI_ERROR(Status)){
        SystemTable->ConOut->OutputString(SystemTable->ConOut, L"Kernel Panic!!: Could not locate the handle buffer.\n");
        goto DONE;
    }else{
        SystemTable->ConOut->OutputString(SystemTable->ConOut, L"Located the handle buffer for file system protocol.\n");
    }
    
    EFI_SIMPLE_FILE_SYSTEM_PROTOCOL* FS = NULL; 
    EFI_FILE_PROTOCOL* Root = NULL;
    EFI_FILE_PROTOCOL* Token = NULL;
    for (UINTN index = 0; index < (UINTN)HandleCount; index++)
    {
            Status = BS->HandleProtocol(Handles[index], &FSPGuid, (void**)&FS);
        Status = FS->OpenVolume(FS, &Root);
        Status = Root->Open(Root, &Token, L"startup.elf", EFI_FILE_MODE_READ, EFI_FILE_MODE_WRITE);
        if(!EFI_ERROR(Status))
            break;
    }
    if (Token == NULL){
        SystemTable->ConOut->OutputString(SystemTable->ConOut, L"Kernel Panic!!: Could not locate startup file.\n");
        goto DONE;  
    }else
        SystemTable->ConOut->OutputString(SystemTable->ConOut, L"Located the startup file.\n");
    UINTN BufferSize = 50000;
    CHAR8 StartupBuffer[50000];
    Status = Token->Read(Token, &BufferSize, StartupBuffer);
    if(EFI_ERROR(Status)){
        SystemTable->ConOut->OutputString(SystemTable->ConOut, L"Kernel Panic!!: Located startup.elf, but could not read from it.\n");
        goto DONE;
    }else
        SystemTable->ConOut->OutputString(SystemTable->ConOut, L"Could read startup.elf properly now jumping to it's entry point.\n");
        
    
    Status = Root->Open(Root, &Token, L"kernel.elf", EFI_FILE_MODE_READ, EFI_FILE_MODE_WRITE);
    if(EFI_ERROR(Status)){
        SystemTable->ConOut->OutputString(SystemTable->ConOut, L"Kernel Panic!!: Could not locate kernel file.\n");
    }else
        SystemTable->ConOut->OutputString(SystemTable->ConOut, L"Could locate the kernel file.\n");
    if (Token == NULL){
        SystemTable->ConOut->OutputString(SystemTable->ConOut, L"Kernel Panic!!: Could not locate kernel file.\n");
        goto DONE;  
    }else
        SystemTable->ConOut->OutputString(SystemTable->ConOut, L"Located the kernel file.\n");
    EFI_PHYSICAL_ADDRESS PhysicalBuffer;
    UINT64* KernelBuffer;
    Status = BS->AllocatePages(AllocateAnyPages, EfiBootServicesData, 4096, &PhysicalBuffer);
    KernelBuffer = (UINT64*)(UINTN)PhysicalBuffer;
    if(EFI_ERROR(Status)){
        SystemTable->ConOut->OutputString(SystemTable->ConOut, L"Kernel Panic!!: Could not allocate pages for the kernel.\n");
        goto DONE;
    }else
        SystemTable->ConOut->OutputString(SystemTable->ConOut, L"Allocated pages for the kernel\n");
    BufferSize = 4096 * 4096;
    Status = Token->Read(Token, &BufferSize, KernelBuffer);
    if(EFI_ERROR(Status)){
        SystemTable->ConOut->OutputString(SystemTable->ConOut, L"Kernel Panic!!: Located kernel.elf, but could not read from it.\n");
        goto DONE;
    }else
        SystemTable->ConOut->OutputString(SystemTable->ConOut, L"Could read kernel.elf properly.\n");

I load a file called startup.elf that will set up a high-half mapping of virtual memory for the kernel. Then I allocate lots of pages (4096) for the kernel which is much smaller than this. The startup.elf executable is then getting the address of the first byte of the kernel from a fixed address and mapping the kernel's loadable segments starting at 0x400000 in physical memory. The upper 2GB of the virtual address space is thus mapped from 0x0 to 2GB in physical memory. The kernel's location in virtual memory is thus 0xffff_ffff_8040_0000. This is where I jump to from the startup.elf file. There are probably several ways to do that but I prefer to have 2 separate files for my kernel (the startup executable and the kernel's executable).

From there, what I am planning to do is to implement a memory management subsystem. Memory management is the first thing you want to implement because every other subsystem is using memory. You want to be able to allocate memory for the kernel's needs.

Afterwards, I plan on implementing an ACPI subsystem that will parse all ACPI tables and interpret the AML language that will allow to gather information on what is present on the motherboard that I need to drive. To do that, I get the RSDP from UEFI and I find the other tables from the RSDP (see https://wiki.osdev.org/RSDP). The ACPI tables allow to gather all information about the hardware on your computer. For example, the MCFG tells you where the configuration space of PCI devices will start (see https://wiki.osdev.org/PCI).

On top of ACPI, I am planning to implement a PCI subsystem that will allow some kind of generic interface to read and write PCI registers and some kind of generic interrupt number allocator. PCI mostly works with MSI/MSI-X today. I am planning to forget about everything that is legacy and just work with the most modern stuff and just assume the presence of all this hardware (that is present on 99% of desktop x64 computers today). Windows does the same with their most recent requirements. They simply give some basic requirements for running their operating-system and the rest isn't supported. This is unlike the Linux kernel which attempts to support everything down to very old dinosaur computers. This makes the code very bloated and not very useful most of the time.

The PCI subsystem will be used by the AHCI driver to manage hard-disks. The AHCI is a PCI device which triggers interrupts using MSI (see https://www.intel.ca/content/www/ca/en/io/serial-ata/serial-ata-ahci-spec-rev1-3-1.html). UEFI firmware comes with an AHCI driver. This is why it manages to write/read hard-drives.

You see that a lot has to be done before you can actually start loading stuff from the hard-drive. You'd need to write a memory management scheme, interpret AML and ACPI tables, parse the PCI buses to find the devices and their configuration spaces and finally write an AHCI driver. Even then, you are probably not finished because modern SATA SSDs have very high throughput. They thus load and store data in the several MBs at once. You'd need some kind of efficient algorithm to cache some portions of the hard-disk and make sure to not load/store all the time as it takes a lot of RAM space. For an older hard-disk, it is going to be easier but not easy.