Using 'fetch' or other methods instead of 'fs' to read markdownfiles in NextJS

Question

I am making a blog contains math formula on markdown pages. I have the markdown files locally stored in a folder. Here is my Blogpage.tsx-

import React from 'react'

import fs from "fs"

import Markdown from "react-markdown"
import rehypeKatex from "rehype-katex"
import remarkMath from "remark-math"
import rehypeRaw from "rehype-raw"
import matter from "gray-matter"
import 'katex/dist/katex.min.css'


// export const runtime = 'edge'
const getPostContent = (postID: string) => {

  const folder = "src/blogPosts";
  const file = `${folder}/${postID}.md`;
  const content = fs.readFileSync(file, "utf8");
  const matterResult = matter(content);
  return matterResult;
};

export const generateStaticParams = async () => {

  return [{ postID: "blog-post1" }]
};



const BlogPage = (props: any) => {



  const postID = props.params.postID;
  const post = getPostContent(postID)



  return (

    <>
      <h1>{post.data.title}</h1>

      <Markdown remarkPlugins={[remarkMath]} rehypePlugins={[rehypeKatex, rehypeRaw]}>
        {post.content}</Markdown>

    </>

  )
}

export default BlogPage

It works perfectly on localhost. But getting an error when trying to deploy on Cloudflare pages.

19:18:58.294    ⚡️ Completed `npx vercel build`.
19:18:58.342    ⚡️ Invalid prerender config for /allblogs/[postID]
19:18:58.343    ⚡️ Invalid prerender config for /allblogs/[postID].rsc
19:18:59.959    
19:18:59.960    ⚡️ ERROR: Failed to produce a Cloudflare Pages build from the project.
19:18:59.961    ⚡️ 
19:18:59.961    ⚡️  The following routes were not configured to run with the Edge Runtime:
19:18:59.961    ⚡️    - /allblogs/[postID]
19:18:59.961    ⚡️ 
19:18:59.962    ⚡️  Please make sure that all your non-static routes export the following edge runtime route segment config:
19:18:59.962    ⚡️    export const runtime = 'edge';
19:18:59.962    ⚡️ 
19:18:59.962    ⚡️  You can read more about the Edge Runtime on the Next.js documentation:
19:18:59.962    ⚡️    https://nextjs.org/docs/app/building-your-application/rendering/edge-and-nodejs-runtimes
19:18:59.962    
19:18:59.992    Failed: Error while executing user command. Exited with error code: 1
19:19:00.002    Failed: build command exited with code: 1
19:19:01.023    Failed: error occurred while running build command

Adding the export const runtime = 'edge' on the page conflicts with 'fs' module and shows the error-

Error: Module not found: Can't resolve 'fs'

It seems the edge runtime conflicts with nodejs module. As and alternative to 'fs' I tried using fetch like this

const getPostContent = async (postID: string) => {
  const resource = await import(`${folder}/${postID}.md`);
  const res = await fetch(resource.default);
  console.log(res)
  if (!res.ok) {
      throw new Error(`${resource.default}: ${res.status} ${res.statusText}`);
  }
  return res.text();
  };

But it is throwing an error-

⨯ unhandledRejection: Error: Cannot find module 

At this point I am stuck. Can anyone please help me with reading the markdown files?

Also is there any alternative way to import markdown file that will not conflict with edge runtume?

Thanks in advance.

Update

I was able to solve the problem I was facing with uploading to cloudflare pages.

Since my blogpages were static markdown pages, I was able to use NextJS static export option. Also getStaticProps() and getStaticPaths() will shows error if you are using NextJS 13 or newer version. After a lot of searching I have found that I need to use generateStaticParams() instead. You can find the changes I have made here. I am just adding this update to help other who are having issue with uploading cloudflare pages.

But I did not receive a good alternative for fetching markdown files from local directory without using 'fs' which was my original question. So, I am going to leave it open.

Answer 1

It seems you're facing a common issue when deploying Next.js applications that use Node.js-specific modules, like fs, to static hosting providers like Cloudflare Pages.

The fs module is not available in the browser or in serverless functions that Cloudflare Pages use for dynamic routes.

To solve your problem, you should use Next.js's getStaticProps and getStaticPaths functions to read the markdown files at build time, rather than at runtime.

This way, you can pre-render each blog post as a static page, which can be served directly by Cloudflare Pages without needing to access the file system.

import React from 'react';
import fs from 'fs';
import path from 'path';
import Markdown from 'react-markdown';
import rehypeKatex from 'rehype-katex';
import remarkMath from 'remark-math';
import rehypeRaw from 'rehype-raw';
import matter from 'gray-matter';
import 'katex/dist/katex.min.css';

const postsDirectory = path.join(process.cwd(), 'src/blogPosts');

export async function getStaticPaths() {
  // Read all markdown files in the postsDirectory
  const filenames = fs.readdirSync(postsDirectory);

  // Create a route for each markdown file
  const paths = filenames.map((filename) => {
    return {
      params: {
        postID: filename.replace(/\.md$/, ''),
      },
    };
  });

  return {
    paths,
    fallback: false,
  };
}

export async function getStaticProps({ params }) {
  // Read the markdown file for the current postID
  const fullPath = path.join(postsDirectory, `${params.postID}.md`);
  const fileContents = fs.readFileSync(fullPath, 'utf8');

  // Parse the post metadata section
  const matterResult = matter(fileContents);

  // Pass the post data to the page via props
  return {
    props: {
      post: matterResult,
    },
  };
}

const BlogPage = ({ post }) => {
  return (
    <>
      <h1>{post.data.title}</h1>
      <Markdown
        remarkPlugins={[remarkMath]}
        rehypePlugins={[rehypeKatex, rehypeRaw]}
      >
        {post.content}
      </Markdown>
    </>
  );
};

export default BlogPage;

Let me explain the above code:

1. getStaticPaths reads all markdown files in your src/blogPosts directory and generates paths for them. This tells Next.js which pages to generate at build time.
2. getStaticProps takes the postID provided by getStaticPaths, reads the corresponding markdown file, and passes the content to your page as props.
3. The BlogPage component receives the post content as props and renders it.

Hope this can help you!!!