Unpredictable behaviour with drakma-async and cl-async

Question

I'm trying to use drakma-async in my small project. But I just can't understand what's happening. (I use emacs + slime + ccl). I need to get data with http(s) and parse it in a callback. I assume I can get wrong data that cannot be parsed, so I want to make a retry. But when I tried to make some tests I just can't understand what's happening...

(defun my-callback (data)
  (prin1 data)
  (restart-case
      (error "Some error parsing data...")
    (just-continue () (prin1 "Continue..."))))

(defun simple-test ()
  (let ((future (asf:make-future)))
    (as:delay #'(lambda () (asf:finish future "Some data")) :time 2)
    (prin1 (asf:future-finished-p future))
    (asf:attach future #'my-callback)))

(defun drakma-test ()
  (asf:alet ((response (das:http-request "http://www.google.com")))
    ;(prin1 (asf:future-finished-p response))
    (asf:attach response #'my-callback)))

(defun drakma-test-let ()
  (let ((response (das:http-request "http://www.google.com")))
    ;(prin1 (asf:future-finished-p response))
    (asf:attach response #'my-callback)))

(defun run-test (test)
  (as:start-event-loop test))

1) So I will that's what I have with my simple example (that's what I've planned)

? (run-test #'simple-test)
NIL"Some data"    ;I get debugger here with simple-error and choose my restart
Invoking restart: #<RESTART JUST-CONTINUE #x7F0578EC20AD>
"Continue..."
1

2) Here what I get in second test:

? (run-test #'drakma-test)
"<A LOT OF HTML>
"
1

Where are my debugger and my restart?

3) Uncomment the ;(prin1 (asf:future...)) line in drakma-test

? (run-test #'drakma-test)
1

No finished/unfinished bool, No Data is not printed, I don't get a restart, I just get 1 as result.

4) I assume if i write (let ((reponse (das:http-request "http://www.google.com"))) ... ) instad of (asf:alet ...) the response will contain not future object, but will block until the request will be finished and the response will contain the data.

? (run-test #'drakma-test-let)
1

5) Uncomment the ;(prin1 (asf:future...)) line in drakma-test-let

? (run-test #'drakma-test-let)
NIL   ;future is not finished
1

Data is not printed, just that is not finished and the result of run-test.

I've run tests for cl-async and they all passed except the ipv6 test. So I just don't know where to start to understand whats happening... Why I get no debugger and restart in 2nd test? Why nothing happens in 3rd test (it's the same as 2nd, but with prin1). Why nothing happens in 5th and 5th tests?

P.S. Don't have enough reputation to create drakma-async or cl-async tags for this libraries. I know that drakma-async is built over drakma so I put this tag.

Answer 1

Thanks for m-n's comment that made the situation clearer and explained shortly the situation.

I made some examples and want to show what happens in each case:

Example:

(defun my-callback (&rest data)
  (format t "Echo from callback: ~A~%" data)
  (restart-case
      (error "Some error parsing data...")
    (just-continue () (prin1 "Continue..."))))

(defun my-errback (e)
  (format t "Echo from errback: ~A~%" e))

(defun make-example-future ()
  (let ((future (asf:make-future))) ;creating future
    (as:delay #'(lambda ()          ;finishing future in 2 seconds
                  (asf:future-handler-case ;wrapping asf:finish
                    (asf:finish future
                                "Result data")
                    (t (e) (asf:signal-error future e)))) ;signal future an error
              :time 2)
    future))

(defun simple-test-2 ()
  (let ((future (make-example-future)))
    (format t "Is future?: ~A~%Finished?: ~A~%"  
            (asf:futurep future) (asf:future-finished-p future))
    (asf:alet ((result future))
      (asf:attach-errback future #'my-errback)
      (format t "Finished? ~A~%" (asf:future-finished-p future))
      (asf:future-finished-p result)
      (asf:attach result #'my-callback))))

And here is what's happening:

? (as:start-event-loop #'simple-test-2)
Is future?: T 
Finished?: NIL
;<here we have a 2 sec pause>
Finished? T
Echo from errback: There is no applicable method for the generic function:
#<STANDARD-GENERIC-FUNCTION CL-ASYNC-FUTURE:FUTURE-FINISHED-P #x302001B67A8F>
when called with arguments:
("Result data")

A) asf:alet wait for result and bind the result value to the variable. So I was wrong thinking that asf:alet bind a future.

B) In make-example-future we wrap asf:finish with asf:future-handler-case and use asf:signal-error to send error to future. That means that error is handled and the errback will be called. Even if the callback is attached later in the code. Moreover, the error with (asf:future-finished-p result) was handled with future-handler-case because it was wrapped in asf:alet (At least I think so).

C) Comment the (asf:future-finished-p result) and the result is

Is future?: T
Finished?: NIL
Finished? T
Echo from callback: (Result data) ;here is my data
Echo from errback: Some error parsing data... ;;here is my error
1

In drakma-async there is similar future-handler-case wrapper that wraps asf:finish.

So this explains the #2 test result. I got the data and asf:alet returned me the string. The error from callback was passed to errback, which I didn't have. Moreover. In drakma-test using only asf:alet I just can't attach errback because I don't have access to future. I need to call http-request in let, not in alet.

Also this explains the result of the #3 test: I got error in (future-finished-p) which was sent to errback.

If we look at the result of #4 and #5 test with new my-callback: It can be seen, that cl-async try to call my callback with all values the drakma returned. There are 7 of them (the values that drakma:http-request return). So I tried to attach wrong number of arguments callback and my #4 and #5 tests were signalling an error that was simply handled by that future-hander-case and send it to errback.

Result: Anyway, it seems impossible to use restarts with drakma-async without removing that future-handler-case because it send error to errback, but lose all restarts.

Hope this post helps if somebody fill face up with my question.