Sensitivity Analysis with PyNetLogo - scatterplot shows strange values

119 Views Asked by At

OBJECTIVE: Use SALib and PyNetLogo to perform a global sensitivity analysis on my NetLogo model. The objective of the model is to measure the change in forest-extent after 160 ticks, i.e., the main model output is forest-extent.

PROBLEM DESCRIPTION: I'm following the PyNetLogo tutorial, specifically Example 2 (https://pynetlogo.readthedocs.io/en/latest/_docs/SALib_ipyparallel.html#Running-the-experiments-in-parallel-using-ipyparallel), adjusted only to match my model instead of the wolf-sheep-predation model from the tutorial. I performed the Sobol' global SA on three test parameters and when looking at the bivariate scatterplot (see below) I noticed that a lot of the data points show a forest-extent of ~14,500, which is equivalent to the initial forest-extent value of the model. Based on thousands of model runs, the forest-extent value is expected to go down to ~1,800 patches. I ran single iterations of the model through PyNetLogo to see if this anomaly occurs again, but the model behaved normally. Same goes for running the model in NetLogo directly. I realized that the scatterplot uses data from each tick, so some values with forest-extent ~14,500 are to be expected, but there should be much fewer. Code for the scatterplot and experiment definition below, the rest is pretty much exactly like in the tutorial / documentation. I'm currently using a Jupyter Lab notebook (not sure if that's relevant).

QUESTION: Is this an issue with how PyNetLogo runs the NetLogo model, causing it to generate the rogue values, or is it a problem of how the scatterplot is generated in matplotlib? I'm new to Python, so apologies if I'm not realizing something very obvious.

enter image description here

def simulation(experiment):

    #Set the input parameters
    for i, name in enumerate(problem['names']):
            netlogo.command('set {0} {1}'.format(name, experiment[i]))

    netlogo.command('setup')
    #This is the original description: Run for 100 ticks and return the number of sheep and wolf
    # agents at each time step; I adjusted it to forest-extent and 160 ticks.
    counts = netlogo.repeat_report(['forest-extent'], 160)

    results = pd.Series([counts['forest-extent'].values.mean()],
                         index=['Avg. forest-extent'])

    return results

[...]

import scipy

nrow=1
ncol=3

fig, ax = plt.subplots(nrow, ncol, sharey=True)

y = results['Avg. forest-extent']

for i, a in enumerate(ax.flatten()):
    x = param_values[:,i]
    sns.regplot(x, y, ax=a, ci=None, color='k',scatter_kws={'alpha':0.2, 's':4, 'color':'gray'})
    pearson = scipy.stats.pearsonr(x, y)
    a.annotate("r: {:6.3f}".format(pearson[0]), xy=(0.15, 0.85), xycoords='axes fraction',fontsize=13)
    if divmod(i,ncol)[1]>0:
        a.get_yaxis().set_visible(False)
    a.set_xlabel(problem['names'][i])
    a.set_ylim([0,1.1*np.max(y)])

fig.set_size_inches(9,9,forward=True)
fig.subplots_adjust(wspace=0.2, hspace=0.3)
  
plt.show()
0

There are 0 best solutions below