uproot: processing a TH2D using the uproot method .pandas()

Question

I am very new to uproot and Python, but hopefully catching up quickly. I am wondering why the method .pandas() is creating such a weird table from a TH2D histogram:

myhisto = file["angular_distr_el/ID3_mol_e0_valid/EN_gate/check_cthetaEE_x"]
type(myhisto)

outputs:

uproot.rootio.TH2D

Finally, myhisto.pandas() returns:

        count   variance
cos(theta)  electron energy [eV]        
[-inf, -1.0)    [-inf, 10.0)    0.0 0.0
[10.0, 10.15)   0.0 0.0
[10.15, 10.3)   0.0 0.0
[10.3, 10.45)   0.0 0.0
[10.45, 10.6)   0.0 0.0
... ... ... ...
[1.0, inf)  [24.4, 24.549999999999997)  0.0 0.0
[24.549999999999997, 24.7)  0.0 0.0
[24.7, 24.85)   0.0 0.0
[24.85, 25.0)   0.0 0.0
[25.0, inf) 0.0 0.0
2244 rows × 2 columns

and myhisto.columns returns:

Index(['count', 'variance'], dtype='object')

Where can I find the documentation of the method .pandas() to understand what it is doing? Is there a way to reorganise myhisto in a DataFrame with the right columns?

Answer 1

After some fun but desperate browsing, I understand which type of object it is. It is a very clever way of creating sorted MultiIndex DataFrames. Just typing myhisto.index is possible to see it directly:

MultiIndex([([-inf, -1.0),                [-inf, 10.0)),
            ([-inf, -1.0),               [10.0, 10.15)),
            ([-inf, -1.0),               [10.15, 10.3)),
            ([-inf, -1.0),               [10.3, 10.45)),
            ([-inf, -1.0),               [10.45, 10.6)),
            ([-inf, -1.0),               [10.6, 10.75)),
            ([-inf, -1.0),               [10.75, 10.9)),
            ([-inf, -1.0),               [10.9, 11.05)),
            ([-inf, -1.0),               [11.05, 11.2)),
            ([-inf, -1.0),               [11.2, 11.35)),
            ...
            (  [1.0, inf), [23.65, 23.799999999999997)),
            (  [1.0, inf), [23.799999999999997, 23.95)),
            (  [1.0, inf),               [23.95, 24.1)),
            (  [1.0, inf),               [24.1, 24.25)),
            (  [1.0, inf),               [24.25, 24.4)),
            (  [1.0, inf),  [24.4, 24.549999999999997)),
            (  [1.0, inf),  [24.549999999999997, 24.7)),
            (  [1.0, inf),               [24.7, 24.85)),
            (  [1.0, inf),               [24.85, 25.0)),
            (  [1.0, inf),                 [25.0, inf))],
           names=['cos(theta)', 'electron energy [eV]'], length=2244)

The solution is to unstack or create a pivot table of the DataFrame. For this specific object, a pivot table is better, because of the presence of counts AND variance as columns in the original DataFrame. As an example:

myhisto.unstack()

count   ... variance
electron energy [eV]    [-inf, 10.0)    [10.0, 10.15)   [10.15, 10.3)   [10.3, 10.45)   [10.45, 10.6)   [10.6, 10.75)   [10.75, 10.9)   [10.9, 11.05)   [11.05, 11.2)   [11.2, 11.35)   ... [23.65, 23.799999999999997) [23.799999999999997, 23.95) [23.95, 24.1)   [24.1, 24.25)   [24.25, 24.4)   [24.4, 24.549999999999997)  [24.549999999999997, 24.7)  [24.7, 24.85)   [24.85, 25.0)   [25.0, inf)
cos(theta)                                                                                  
[-inf, -1.0)    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
[-1.0, -0.9)    0.0 1.0 1.0 0.0 0.0 2.0 0.0 2.0 0.0 1.0 ... 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0
[-0.9, -0.8)    0.0 0.0 3.0 3.0 0.0 0.0 0.0 0.0 1.0 1.0 ... 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 1.0 0.0
[-0.8, -0.7)    0.0 0.0 1.0 2.0 0.0 1.0 1.0 2.0 1.0 1.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
[-0.7, -0.6)    0.0 0.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 ... 1.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 0.0 0.0
[-0.6, -0.5)    0.0 1.0 1.0 1.0 0.0 0.0 2.0 1.0 0.0 3.0 ... 0.0 1.0 0.0 1.0 1.0 

**22 rows × 204 columns**

vs.

pivot_pipanda = pipanda.pivot_table(values="count", index="cos(theta)", columns="electron energy [eV]")

electron energy [eV]    [-inf, 10.0)    [10.0, 10.15)   [10.15, 10.3)   [10.3, 10.45)   [10.45, 10.6)   [10.6, 10.75)   [10.75, 10.9)   [10.9, 11.05)   [11.05, 11.2)   [11.2, 11.35)   ... [23.65, 23.799999999999997) [23.799999999999997, 23.95) [23.95, 24.1)   [24.1, 24.25)   [24.25, 24.4)   [24.4, 24.549999999999997)  [24.549999999999997, 24.7)  [24.7, 24.85)   [24.85, 25.0)   [25.0, inf)
cos(theta)                                                                                  
[-inf, -1.0)    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
[-1.0, -0.9)    0.0 1.0 1.0 0.0 0.0 2.0 0.0 2.0 0.0 1.0 ... 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0
[-0.9, -0.8)    0.0 0.0 3.0 3.0 0.0 0.0 0.0 0.0 1.0 1.0 ... 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 1.0 0.0
[-0.8, -0.7)    0.0 0.0 1.0 2.0 0.0 1.0 1.0 2.0 1.0 1.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
[-0.7, -0.6)    0.0 0.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 ... 1.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 0.0 0.0
[-0.6, -0.5)    0.0 1.0 1.0 1.0 0.0 0.0 2.0 1.0 0.0 3.0 ... 0.0 1.0 0.0 1.0 1.0 0.0 1.0 0.0 0.0 0.0
[-0.5, -0.3999999999999999) 0.0 0.0 2.0 0.0 1.0 1.0 3.0 2.0 3.0 1.0 ... 3.0 0.0 0.0 0.0 0.0 2.0 0.0 1.0 1.0 0.0

and from here the standard methods of pandas are available!

(To play with slicing techniques like loc[] and iloc[]: https://www.youtube.com/watch?v=tcRGa2soc-c)