Make voice call using Twilio Client JS, a subaccount and access token

Question

I am trying to make a voice call from a subaccount and keep getting the error:

callerId must be provided for TwilioClient and SIP calls when using Dial

I have verified my application SID, Account SID, api key SID and secret and all match up. The body of the response logged in the Twilio debugger is:

<?xml version="1.0" encoding="UTF-8"?>
<Response>
    <Dial answerOnBridge="true" callerId="PHONE NUMBER OF PARENT ACCOUNT">
        <Number>THE PHONE NUMBER IM TRYING TO CALL (A REAL PHONE)</Number>
    </Dial>
</Response>

I cant find anywhere in the docs that show this with the JS library. I have tried setting the From field in the options for Device.connect with no luck. Do i have to specify anything for the voice grants?

Another odd thing, everything works if I dial from the parent account's number, but When i go and look at the parent account, it doesnt have voice enabled.