Get bounding rectangle for CGGlyphs of Characters other then letters or numbers

Question

I am looking for a way to equire character's glyph descent as indicated on the picture:

The method needs to work for any given character (or at least all common unicode characters).

Here is my actual approach (in Swift, inspired by this and this question):

let char = "a"
let ctFont = CTFontCreateWithNameAndOptions("HelveticaNeue", 12, nil, nil)
var ctGlyph = CTFontGetGlyphWithName(ctFont, char)
let boundingBox = withUnsafePointer(&ctGlyph) { pointer -> CGRect in
    return CTFontGetBoundingRectsForGlyphs(ctFont, CTFontOrientation.OrientationDefault, pointer, nil, 1)
}

Descent I need is then simply equal to -boundingBox.origin.y. This approach works nicely for letter and number and number characters (see this answer for graphical representation).

The problem is that for everything other then letters or numbers (for example: .,#')*) it I get the same bounding rectangle: {x:0.612, y:0.012, w:4.908, h:8.532}. That is obviously incorrect.

How can I get the bonding rectangle of the descent directly for all characters?