Auto-fill empty coordinates and smooth data with geom_raster

Question

Code:

ggplot(dat, aes(dat$x, dat$y)) + 
  xlim(-3,3) + 
  ylim(-5,5) +
  geom_raster(aes(fill = dat$density),
              hjust = 0, vjust = 0, interpolate = TRUE, alpha =1)

Data:

x  y    density
1  -1  1 0.07589831
2   2  1 0.01999884
3   1  0 0.14752616
4  -1  0 0.10468884
5  -1  1 0.07589831
6  -1  0 0.10468884
7  -2  0 0.05619694
8   1  0 0.14752616
9  -2  1 0.05072076
10  1  0 0.14752616
11  1 -1 0.04915273
12 -2 -1 0.01330193
13 -1  1 0.07589831
14 -1  0 0.10468884
15  0  0 0.14332413
16  1 -1 0.04915273
17  0  0 0.14332413
18  0  0 0.14332413
19  3  0 0.03155999
20 -2  1 0.05072076
21  0  1 0.07323680
22  2 -1 0.03560643
23  0  0 0.14332413
24 -1  0 0.10468884
25  1  0 0.14752616
26  2  0 0.09192561
27  0  1 0.07323680
28  0  1 0.07323680
29 -3  1 0.03381405
30  0 -1 0.04760234
31  0  1 0.07323680
32 -1  1 0.07589831
33  1  2 0.01242018
34 -3  2 0.02415452
35  1  0 0.14752616
36 -3  2 0.02415452
37  0  0 0.14332413
38  0 -1 0.04760234
39  0  0 0.14332413
40  0  0 0.14332413
41 -2  0 0.05619694
42 -3  1 0.03381405
43 -1  0 0.10468884
44  2 -1 0.03560643
45  2  0 0.09192561
46  1  1 0.04103001
47  1  0 0.14752616
48  0 -1 0.04760234
49  1  0 0.14752616
50  1  0 0.14752616
51  2  0 0.09192561
52 -2  0 0.05619694
53  2  0 0.09192561

I plotted x & y from data frame dat. How can I smooth the geom_raster to look almost fluid? I've added interpolate, but the plot still looks boxy? I would also like to fill in the empty spaces with the least dense color. Thank you!!