I am building a google form with apps script and am unable to duplicate functionality found in the builder.
I cannot find methods to set the option on a GridItem called "limit to one response per column" or "shuffle row order"
both which are found in an advanced tab at the bottom of the grid edit block.
Is there a way to do this?
It sounds like you are using the Forms service to programmatically build a form - you weren't clear in your question.
Those advanced settings have not been exposed through the Google Apps Script Forms service. Since there's no other API for Forms, your only recourse is to request an API extension. Issue 6393 is asking for it now, and you can star that issue to add your vote for it.
There is one relevant open issue, Issue 3654, which is an enhancement-request for control over the number of selections allowed per line in a griditem.