What is the best way to assign nil to a default association?

Question

So if I have a factory such as this (address is another factory) and I want to create a test that creates a person with no address

require 'faker'

FactoryGirl.define do
  factory :person do
    address
    first        { Faker::Name.first_name }
    last         { Faker::Name.last_name }
  end
end

I would expect the following to work (but doesn't):

create(:person, address: nil)

What is the best way to set this up properly?