Comparison returns expected value call function directory, but it is not so at process on list

Question

I am creating a simple elisp tester. However, I am getting the wrong behavior (which I can not understand) as seen below.

I think that testers should return t test cases (:eq 'a 'a) and (:eq (return-symbol) 'a) naturally as my tester also precedes the following code. Actually it is not so.

The following code has been lengthened beyond necessity, but for the most part it is checking the obvious behavior.

I think that my tester should also return those expected return values.

Are there any good ideas? Even explaining the reason for this behavior would be a help to improve my tester. I would appreciate it if you give me something.

;; return-symbol is always return 'a
(defun return-symbol ()
  'a)
;; => return-symbol

;; operation check for return-symbol
(return-symbol)
;; => a

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

;; compare 'a and 'a by eq
(eq 'a 'a)
;; => t

;; compare 'a and 'a by equal
(equal 'a 'a)
;; => t

;; compare (return-symbol) and 'a by eq
(eq (return-symbol) 'a)
;; => t

;; compare (return-symbol) and (return-symbol) by eq
(eq (return-symbol) (return-symbol))
;; => t

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

;; comparison by funcalled eq
(funcall 'eq 'a 'a)
;; => t

(funcall 'eq (return-symbol) 'a)
;; => t

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

;; funcall with interned symbol
(funcall (intern "eq") 'a 'a)
;; => t

(funcall (intern "eq") (return-symbol) 'a)
;; => t

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

;; define universal comparison function
(defun multi-comp (key a b)
  "KEY is funcname symbol such as :FUNCNAME"
  (let ((funcname (replace-regexp-in-string "^:+" "" (symbol-name key))))
    (funcall (intern funcname) a b)))
;; => multi-comp

;; operation check for multi-comp
(multi-comp :eq 'a 'a)
;; => t

(multi-comp :eq (return-symbol) 'a)
;; => t

(multi-comp :equal (return-symbol) 'a)
;; => t

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

;; Define function to apply sequentially
(defun run-test (tests)
  "TESTS is list such as (([str]TESTNAME ([sym]:FUNC [sexp]A [sexp]B))
                          ([str]TESTNAME ([sym]:FUNC [sexp]A [sexp]B))...)"
  (dolist (x tests)
    (let* ((testname (car x))
           (values   (cadr x))
           (key      (nth 0 values))
           (a        (nth 1 values))
           (b        (nth 2 values)))
      (if (multi-comp key a b)
          (princ (format "%s is passed\n" testname))
        (princ (format "%s is failed\n" testname))))))
;; => run-test

;; operation check of run-test
(run-test '(("eq1" (:eq 'a 'a))
            ("eq2" (:eq (return-symbol) (return-symbol)))
            ("equal1" (:equal 'a 'a))
            ("equal2" (:equal (return-symbol) 'a))
            ("equal3" (:equal (return-symbol) (return-symbol)))))
;; =>
;; eq1 is failed       ; <= ??
;; eq2 is failed       ; <= ??
;; equal1 is passed
;; equal2 is failed    ; <= ??
;; equal3 is passed
;; nil

Answer 1

Your argument to run-test is evaluated once, so "eq1" sees 'a which is (quote a) (a list of length 2) which is, of course, fails under eq. Similarly, (return-symbol) is not evaluated and "eq1" sees the lists of length 1 which are not identical under eq.

You would have discovered that by simply adding print to multi-comp.

Your code would probably work is you replace (multi-comp key a b) with (multi-comp key (eval a) (eval b)).

Please note that the fact that you need eval is a very strong indicator that you are doing something horribly wrong.

PS. I urge you to use ERT instead of rolling your own.