MySQL explain shows "Using Temporary"

Question

I am trying to optimize my mySQL query and I am coming across a strange problem.

If I run the following query:

SELECT `survey_visit`.`id` 
FROM `survey_visit` 
LEFT JOIN `campaign` ON `survey_visit`.`campaign_alphanumeric` = `campaign`.`alphanumeric` 
ORDER BY `visit_date` DESC 
LIMIT 5

The EXPLAIN shows that I read 5 rows in the survey_visit table. However, if I do a second join as follows:

SELECT `survey_visit`.`id` 
FROM `survey_visit` 
LEFT JOIN `campaign` ON `survey_visit`.`campaign_alphanumeric` = `campaign`.`alphanumeric` 
LEFT JOIN `client` ON `campaign`.`fk_clientID` = `client`.`id`
ORDER BY `visit_date` DESC 
LIMIT 5

then it uses a temporary table, uses filesort and reads through all records. Note that the problem is very specific to joining the "client" table. I have tried joining other tables and it all works fine. I have checked the indexes and all seems to be well.

Any advice is most appreciated.

Here is the EXPLAIN on the second query:

[0] => Array
    (
        [id] => 1
        [select_type] => SIMPLE
        [table] => survey_visit
        [type] => ALL
        [possible_keys] => 
        [key] => 
        [key_len] => 
        [ref] => 
        [rows] => 1564
        [Extra] => Using temporary; Using filesort
    )

[1] => Array
    (
        [id] => 1
        [select_type] => SIMPLE
        [table] => campaign
        [type] => eq_ref
        [possible_keys] => alphanumeric
        [key] => alphanumeric
        [key_len] => 12
        [ref] => bpharma.survey_visit.campaign_alphanumeric
        [rows] => 1
        [Extra] => 
    )

[2] => Array
    (
        [id] => 1
        [select_type] => SIMPLE
        [table] => client
        [type] => index
        [possible_keys] => PRIMARY,id_UNIQUE
        [key] => PRIMARY
        [key_len] => 4
        [ref] => 
        [rows] => 2
        [Extra] => Using where; Using index; Using join buffer (Block Nested Loop)
    )

EDIT: Here is the EXPLAIN for query 1. Thanks

[0] => Array
    (
        [id] => 1
        [select_type] => SIMPLE
        [table] => survey_visit
        [type] => index
        [possible_keys] => 
        [key] => survey_visit_visit_date_index
        [key_len] => 6
        [ref] => 
        [rows] => 5
        [Extra] => 
    )

[1] => Array
    (
        [id] => 1
        [select_type] => SIMPLE
        [table] => campaign
        [type] => eq_ref
        [possible_keys] => campaign_alphanumeric_uindex
        [key] => campaign_alphanumeric_uindex
        [key_len] => 12
        [ref] => bpharma.survey_visit.campaign_alphanumeric
        [rows] => 1
        [Extra] => Using index
    )