Embedded Perl FREETMPS not cleaning up array of hash references properly

61 Views Asked by At

I have a project with several macros to help me write Perl functions, and basically "convert" C structs into Perl hashes. Some of these hashes nest each other, but I'll get to the point.

My program's memory usage rises drastically over time, it's a social media frontend written in C and Embedded Perl (odd language choice, but it's whatever). I'm 100% certain I've found the root cause of code, as removing it and other things causes it to go away, and I've profiled it as well, and every issue points to my array of hashes.

if (notifs && notifs_len)
{
    // Leak occurs here
    mXPUSHs(newRV_noinc((SV*)perlify_notifications(notifs, notifs_len)));
}
else ARG_UNDEFINED();


// Run function
PERL_STACK_SCALAR_CALL("base_page");
char* dup = PERL_GET_STACK_EXIT;

Where PERL_GET_STACK_EXIT is a macro which equates to

#define PERL_GET_STACK_EXIT savesharedsvpv(POPs);   \
    PUTBACK;                                        \
    FREETMPS;                                       \
    LEAVE;                                          \
    perl_unlock()

// Macros for storing data
#define hvstores_str(hv, key, val) if (val) { hv_stores((hv), key, newSVpv((val), 0)); }
#define hvstores_int(hv, key, val) hv_stores((hv), key, newSViv((val)))
#define hvstores_ref(hv, key, val) if (val) {  hv_stores((hv), key, newRV_noinc((SV*)(val))); }

perlify_notifications is created using another macro.

// Calls one of the `perlify_X` functions for each element in an array
// Note: I was not using av_push before, I only temporarily switched thinking that was the issue or if I was misreading the docs
// EX: PERLIFY_MULTI(notification, notifications, notification_t
#define PERLIFY_MULTI(type, types, mstype) AV* perlify_##types(const struct mstype* const types, size_t len) { \
    if (!(types && len)) return NULL;                               \
    AV* av = newAV();                                               \
    for (size_t i = 0; i < len; ++i)                                \
        av_push(av, newRV_noinc((SV*)perlify_##type(types + i)));   \
    return av;                                                      \
}

Taking a step further, here is the hash functions. Such as perlify_notification in the code.

// Converts it into a perl struct
HV* perlify_notification(const struct mstdnt_notification* const notif)
{
    if (!notif) return NULL;
    HV* notif_hv = newHV();

    // Profiler doesn't show much data leaking for these 4, so I am uncertain
    hvstores_str(notif_hv, "id", notif->id);
    hvstores_int(notif_hv, "created_at", notif->created_at);
    hvstores_str(notif_hv, "emoji", notif->emoji);
    hvstores_str(notif_hv, "type", mstdnt_notification_t_to_str(notif->type));
    // The functions below seem to leak the most (and they store a bunch of data), if I comment them out, most of the
    // memleak is gone (because these output a lot of data). The functions
    // being called look equivalent to this one, so it's no secret.
    hvstores_ref(notif_hv, "account", perlify_account(notif->account));
    hvstores_ref(notif_hv, "pleroma", perlify_notification_pleroma(notif->pleroma));
    hvstores_ref(notif_hv, "status", perlify_status(notif->status));
    
    return notif_hv;
}

I tried to compile all of Perl from source since I'm on Gentoo to further debug this issue and look into the FREETMPS internals, and tried to see what was happening. I tried dumping all the references I could, yet every object had a REFCNT of 1. No luck

When I ran this with massif, viewed with Top, I would just watch the memory increase slowly. I tested my luck with Trial and error: It starts at around 9 MB, then slowly increases up to 15 MB to 25 MB memory usage, as long as I keep calling these functions over and over. Of course, if I comment FREETMPS out, or just run the functions without anything else, memory usage increases at the same rate as before. I'm still certain this is the root issue.

I'm really at a loss here, am I doing something wrong with how I'm "perlifying" my C structs?

0

There are 0 best solutions below