HTML from DB not injecting into hunchentoot route

Question

I have many routes in my app that get information from a database. In one particular situation, I am getting HTML content from a database to render it with cl-who.

I am not sure why the content from the DB will not render. The rest of the page works fine.

When I test adding html into the route manually (hard coded html tags) it works fine. In the case when I use the (second (get-content)) (see below) the html from the DB is not added to the template.

What am I missing?

Database macro

(defmacro access-db(db &body query)
  `(progn
     (clsql:connect ,db)
     (unwind-protect (progn ,@query)
      (disconnect-db ,db)))) 

Page Macro

(defmacro defpage ((&key title) &body content)
  `(cl-who:with-html-output-to-string 
       (*standard-output* nil :prologue t :indent t)
     (:html
      :xmlns "http://www.w3.org/1999/xhtml"
      :xml\:lang "en"
      :lang "en"
      (:head
       (:meta
    :http-equiv "Content-Type"
    :content "text/html;charset=utf-8")
       (:title
    ,(format nil "~A" title))
       (:link :type "text/css" 
          :rel "stylesheet"
          :href "/styles.css"))
      (:body :class "whole-page" 
         (:div :class "site-articles"
           (:div :class "article-body"
             (:div :class "on-page-title"
                   (:h1 :class "main-header" ,title))
             ,@content))))))

Hunchentoot route

(define-easy-handler (test-page :uri "/wow") ()
  (let ((content
      (bt:make-thread
       (lambda ()
         (second (get-content))))))
    (defpage (:title "Main One") 
      (bt:join-thread content))))

get-content uses access-db function and returns a list the first item from the database. A title and HTML. The html is second in that list.

Get content

(defun get-content ()
  (let ((blogs
      (access-db *PSQL-CONNECT-INFO*
        (clsql:query
         "select * from content"))))
    (first blogs)))

Answer 1

[Disclaimer: I can't get CLSQL to work and anyway don't want anything to do with databases, so this answer is about the problems with the other things.]

I think you may probably be confused about macros in general, but certainly your defpage macro is confused.

What CL-WHO's with-html-output does is to bind a stream to a variable to which you can print, and then treat its body as the implicit 'html-lisp' language that it and all similar macros (I think HTOUT was the first that really did this) understand. That means that, if you want to send output into this stream you need to print it to that stream. with-html-output-to-string just captures that output into a string in the obvious way.

Here is a cleaned-up version of that macro with a better name (defpage smells to me like 'define page' which is not at all what it does).

(defmacro with-output-to-standard-page ((&key (title "page")
                                              (stream '*standard-output*)
                                              (string-form 'nil))
                                        &body content)
  (let ((tv (make-symbol "TITLE")))
    `(let ((,tv ,title))
       (with-html-output-to-string (,var ,string-form :prologue t :indent t)
         (:html
          :xmlns "http://www.w3.org/1999/xhtml"
          :xml\:lang "en"
          :lang "en"
          (:head
           (:meta
            :http-equiv "Content-Type"
            :content "text/html;charset=utf-8")
           (:title (princ ,tv ,var))
           (:link :type "text/css" 
            :rel "stylesheet"
            :href "/styles.css"))
          (:body :class "whole-page" 
           (:div :class "site-articles"
            (:div :class "article-body"
             (:div :class "on-page-title"
              (:h1 :class "main-header"
               (princ ,tv ,var)))
             ,@content))))))))

This has behaviour which is both more conventional than yours:

(let ((title "foo"))
  (with-output-to-standard-page (:title title)
    ...))

will work, and

(with-output-to-standard-page (:title (complicated-function-with-state))
  ...)

will call complicated-function-with-state just once.

Additionally it lets you define what the stream variable and the string form are, if you want:

(with-output-to-standard-page (:var s)
  (princ "foo" s))

will work.

Finally, it puts the content underneath the initial h1, not in it (and outside the on-page-title div in fact), and uses the title for the h1.

All this is also done in a package which uses CL-WHO, HUNCHENTOOT and BORDEAUX-THREADS to avoid the awful package-prefix-everwhere horror and make the code readable.

So then if we replace your get-content function by a shim because no SQL:

(defun get-content ()
  (list 'nothing "my page content"))

Then, correcting your handler to print to the stream, which is the crucial problem

(define-easy-handler (test-page :uri "/wow") ()
  (let ((content-thread
         (make-thread
          (lambda ()
            (second (get-content))))))
    (with-output-to-standard-page (:var p :title "Main One") 
      (princ (join-thread content-thread) p))))

Everything will now work.

---

Looking at your access-db macro there are also, almost certainly, problems with it.

First of all I'll call it something like with-db-access because with-* is one conventional name for macros like this (another possibility might be accessing-db). Then it has, at least, the problem that it will multiply evaluate its first argument. It should instead be something like

(defmacro with-db-access (db &body query)
  (let ((dbv (make-symbol "DB")))
    `(let ((,dbv ,db))
       (connect ,dbv)
       (unwind-protect
           (progn ,@query)
         (disconnect-db ,dbv)))))

to avoid that problem.

Note that this very common pattern in macros (and the corresponding very common problem when people forget this) can be made a lot simpler using Tim Bradshaw's metatronic macros. Using that, this would be:

(defmacro/m with-db-access (db &body query)
  `(let ((<db> ,db))
     (connect <db>)
     (unwind-protect
         (progn ,@query)
       (disconnect-db <db>))))

My with-output-to-standard-page macro above would be similarly simplified using metatronic macros.