c Pthreads + valgrind = Speicherleck: warum?

Question

Ich beginne mit Pthreads in C und ich bin auch ein Wahnsinniger, meinen Code als "fehlerfrei" zu schreiben, wie ich nur kann.

Trotz versucht, besonders vorsichtig zu sein, mich valgrind ist zu sagen, dass ich Speicher bin undicht, und zwar unabhängig Wetter:

1. ich joinable Threads erstellen, die ich nach Abschluss (Code-Schnipsel 1) beitreten
2. Ich schaffe joinable Themen, die ich
3. ich schaffen, frei stehende Fäden (Code-Schnipsel 3) nach der Erstellung (Code-Schnipsel 2)

ich weiß, das wird bereits diskutiert (siehelösen, this und auch this), aber ich bin immer noch neugierig, wie:

1. Warum bei bestimmten läuft ich ohne Fehler am Ende?
2. Warum scheint es eine zufällige Anzahl von insgesamt mallocs() im Umgang mit abgelösten Threads zu sein? < < Antwort zur Verfügung gestellt von Nos, Code-Snippet "behoben" mit einer zusätzlichen Verzögerung in der Main()
3. Warum bleibt das "Speicher-Leck" auch bei losem Thread bestehen? < < gleiche wie 2.

wie ich aus den früheren Antworten und der valgrind trace verstehen, pthread_create() ist die Ursache, den Stapel von Fäden nach Bedarf verwendet erstreckt, und es manchmal wiederzuverwenden, so dass ein paar fehlende befreit. Was jedoch weniger klar ist, ist, warum es auf den Ausführungslauf ankommt und warum es auch beim Erstellen von getrennten Threads passiert. Wie ich aus bestimmten Antworten, Kommentaren und auch von dem Mann gesehen habe, werden die Ressourcen von einem gelösten Thread nach der Fertigstellung des Threads freigegeben. Ich habe verschiedene Tweaks ausprobiert, um dies zu umgehen (hinzugefügt eine Ruhezeit vor dem Ende jedes Threads, vor dem Ende des Hauptthreads, erhöht die Stapelgröße, fügen Sie mehr "Arbeit" ...), aber es änderte nicht die Endergebnis von viel. Warum gibt es eine zufällige Anzahl von allgemeinen "mallocs()", wenn es sich um losgelöste Threads handelt, verliert Valgrind die Spur einiger gelöster Threads? Dies scheint auch nicht von der Stapelgröße abhängig zu sein.

Der angegebene Code ist ein Pseudo-Beispiel für ein Manager/Worker-Modell, für das ein Joinable/Join() -Ansatz zur Thread-Verwaltung besser geeignet ist.

Danke für jede Erleuchtung, die Sie vielleicht bieten könnten! Ich hoffe auch, dass diese (über-kommentierten) Codeschnipsel jedem helfen werden, der mit Pthreads beginnen möchte.

- swappy

PS Sys info: gcc auf debian 64bit arch

-Code Schnipsel 1 (joinable Gewinde verbunden):

/* Running this multiple times with valgrind, I sometimes end with : 
    - no errors (proper malloc/free balance) 
    - 4 extra malloc vs free (most frequently) 
    The number of mallocs() is more conservative and depends on the number of threads. 
*/ 

#include <stdlib.h>    /* EXIT_FAILURE, EXIT_SUCCESS macros & the likes */ 
#include <stdio.h>    /* printf() & the likes */ 
#include <pthread.h>   /* test subject */ 

#define MAX_THREADS 100   /* Number of threads */ 
pthread_attr_t tattr;   /* Thread attribute */ 
pthread_t workers[MAX_THREADS]; /* All the threads spawned by the main() thread */ 

/* A mock container structure to pass arguments around */ 
struct args_for_job_t { 
    int tid; 
    int status; 
}; 

/* The job each worker will perform upon creation */ 
void *job(void *arg) 
{ 
    /* Cast arguments in a proper container */ 
    struct args_for_job_t *container; 
    container = (struct args_for_job_t *)arg; 

    /* A mock job */ 
    printf("[TID - %d]\n", container->tid); 

    /* Properly exit with status code tid */ 
    pthread_exit((void *)(&container->status)); 
} 

int main() 
{ 
    int return_code;       /* Will hold return codes */ 
    void *return_status;      /* Will hold return status */ 
    int tid;         /* Thread id */ 
    struct args_for_job_t args[MAX_THREADS]; /* For thread safeness */ 

    /* Initialize and set thread joinable attribute */ 
    pthread_attr_init(&tattr); 
    pthread_attr_setdetachstate(&tattr, PTHREAD_CREATE_JOINABLE); 

    /* Spawn detached threads */ 
    for (tid = 0; tid < MAX_THREADS; tid++) 
    { 
     args[tid].tid = tid; 
     args[tid].status = tid; 
     return_code = pthread_create(&workers[tid], &tattr, job, (void *)(&args[tid])); 
     if (return_code != 0) { printf("[ERROR] Thread creation failed\n"); return EXIT_FAILURE; } 
    } 

    /* Free thread attribute */ 
    pthread_attr_destroy(&tattr); 

    /* Properly join() all workers before completion */ 
    for(tid = 0; tid < MAX_THREADS; tid++) 
    { 
     return_code = pthread_join(workers[tid], &return_status); 
     if (return_code != 0) 
     { 
      printf("[ERROR] Return code from pthread_join() is %d\n", return_code); 
      return EXIT_FAILURE; 
     } 
     printf("Thread %d joined with return status %d\n", tid, *(int *)return_status); 
    } 

    return EXIT_SUCCESS; 
} 

-Code Schnipsel 2 (freistehende Fäden nach der Erstellung):

/* Running this multiple times with valgrind, I sometimes end with : 
    - no errors (proper malloc/free balance) 
    - 1 extra malloc vs free (most frequently) 
    Most surprisingly, it seems there is a random amount of overall mallocs 
*/ 

#include <stdlib.h>    /* EXIT_FAILURE, EXIT_SUCCESS macros & the likes */ 
#include <stdio.h>    /* printf() & the likes */ 
#include <pthread.h>   /* test subject */ 
#include <unistd.h>   

#define MAX_THREADS 100   /* Number of threads */ 
pthread_attr_t tattr;   /* Thread attribute */ 
pthread_t workers[MAX_THREADS]; /* All the threads spawned by the main() thread */ 

/* A mock container structure to pass arguments around */ 
struct args_for_job_t { 
    int tid; 
}; 

/* The job each worker will perform upon creation */ 
void *job(void *arg) 
{ 
    /* Cast arguments in a proper container */ 
    struct args_for_job_t *container; 
    container = (struct args_for_job_t *)arg; 

    /* A mock job */ 
    printf("[TID - %d]\n", container->tid); 

    /* For the sake of returning something, not necessary */ 
    return NULL; 
} 

int main() 
{ 
    int return_code;       /* Will hold return codes */ 
    int tid;         /* Thread id */ 
    struct args_for_job_t args[MAX_THREADS]; /* For thread safeness */ 

    /* Initialize and set thread joinable attribute */ 
    pthread_attr_init(&tattr); 
    pthread_attr_setdetachstate(&tattr, PTHREAD_CREATE_JOINABLE); 

    /* Spawn detached threads */ 
    for (tid = 0; tid < MAX_THREADS; tid++) 
    { 
     args[tid].tid = tid; 
     return_code = pthread_create(&workers[tid], &tattr, job, (void *)(&args[tid])); 
     if (return_code != 0) { printf("[ERROR] Thread creation failed\n"); return EXIT_FAILURE; } 
     /* Detach worker after creation */ 
     pthread_detach(workers[tid]); 
    } 

    /* Free thread attribute */ 
    pthread_attr_destroy(&tattr); 

    /* Delay main() completion until all detached threads finish their jobs. */ 
    usleep(100000); 
    return EXIT_SUCCESS; 
} 

-Code Schnipsel 3 (freistehende Fäden bei der Erstellung):

/* Running this multiple times with valgrind, I sometimes end with : 
    - no errors (proper malloc/free balance) 
    - 1 extra malloc vs free (most frequently) 
    Most surprisingly, it seems there is a random amount of overall mallocs 
*/ 

#include <stdlib.h>    /* EXIT_FAILURE, EXIT_SUCCESS macros & the likes */ 
#include <stdio.h>    /* printf() & the likes */ 
#include <pthread.h>   /* test subject */ 

#define MAX_THREADS 100   /* Number of threads */ 
pthread_attr_t tattr;   /* Thread attribute */ 
pthread_t workers[MAX_THREADS]; /* All the threads spawned by the main() thread */ 

/* A mock container structure to pass arguments around */ 
struct args_for_job_t { 
    int tid; 
}; 

/* The job each worker will perform upon creation */ 
void *job(void *arg) 
{ 
    /* Cast arguments in a proper container */ 
    struct args_for_job_t *container; 
    container = (struct args_for_job_t *)arg; 

    /* A mock job */ 
    printf("[TID - %d]\n", container->tid); 

    /* For the sake of returning something, not necessary */ 
    return NULL; 
} 

int main() 
{ 
    int return_code;       /* Will hold return codes */ 
    int tid;         /* Thread id */ 
    struct args_for_job_t args[MAX_THREADS]; /* For thread safeness */ 

    /* Initialize and set thread detached attribute */ 
    pthread_attr_init(&tattr); 
    pthread_attr_setdetachstate(&tattr, PTHREAD_CREATE_DETACHED); 

    /* Spawn detached threads */ 
    for (tid = 0; tid < MAX_THREADS; tid++) 
    { 
     args[tid].tid = tid; 
     return_code = pthread_create(&workers[tid], &tattr, job, (void *)(&args[tid])); 
     if (return_code != 0) { printf("[ERROR] Thread creation failed\n"); return EXIT_FAILURE; } 
    } 

    /* Free thread attribute */ 
    pthread_attr_destroy(&tattr); 

    /* Delay main() completion until all detached threads finish their jobs. */ 
    usleep(100000); 
    return EXIT_SUCCESS; 
} 

Valgrind Ausgang für Codefragment 1 (verbundene Fäden & mem-Leck)

==27802== 
==27802== HEAP SUMMARY: 
==27802==  in use at exit: 1,558 bytes in 4 blocks 
==27802== total heap usage: 105 allocs, 101 frees, 28,814 bytes allocated 
==27802== 
==27802== Searching for pointers to 4 not-freed blocks 
==27802== Checked 104,360 bytes 
==27802== 
==27802== 36 bytes in 1 blocks are still reachable in loss record 1 of 4 
==27802== at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) 
==27802== by 0x400894D: _dl_map_object (dl-load.c:162) 
==27802== by 0x401384A: dl_open_worker (dl-open.c:225) 
==27802== by 0x400F175: _dl_catch_error (dl-error.c:178) 
==27802== by 0x4013319: _dl_open (dl-open.c:639) 
==27802== by 0x517F601: do_dlopen (dl-libc.c:89) 
==27802== by 0x400F175: _dl_catch_error (dl-error.c:178) 
==27802== by 0x517F6C3: __libc_dlopen_mode (dl-libc.c:48) 
==27802== by 0x4E423BB: pthread_cancel_init (unwind-forcedunwind.c:53) 
==27802== by 0x4E4257B: _Unwind_ForcedUnwind (unwind-forcedunwind.c:130) 
==27802== by 0x4E4069F: __pthread_unwind (unwind.c:130) 
==27802== by 0x4E3AFF4: pthread_exit (pthreadP.h:265) 
==27802== 
==27802== 36 bytes in 1 blocks are still reachable in loss record 2 of 4 
==27802== at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) 
==27802== by 0x400B7EC: _dl_new_object (dl-object.c:161) 
==27802== by 0x4006805: _dl_map_object_from_fd (dl-load.c:1051) 
==27802== by 0x4008699: _dl_map_object (dl-load.c:2568) 
==27802== by 0x401384A: dl_open_worker (dl-open.c:225) 
==27802== by 0x400F175: _dl_catch_error (dl-error.c:178) 
==27802== by 0x4013319: _dl_open (dl-open.c:639) 
==27802== by 0x517F601: do_dlopen (dl-libc.c:89) 
==27802== by 0x400F175: _dl_catch_error (dl-error.c:178) 
==27802== by 0x517F6C3: __libc_dlopen_mode (dl-libc.c:48) 
==27802== by 0x4E423BB: pthread_cancel_init (unwind-forcedunwind.c:53) 
==27802== by 0x4E4257B: _Unwind_ForcedUnwind (unwind-forcedunwind.c:130) 
==27802== 
==27802== 312 bytes in 1 blocks are still reachable in loss record 3 of 4 
==27802== at 0x4C29DB4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) 
==27802== by 0x4010B59: _dl_check_map_versions (dl-version.c:300) 
==27802== by 0x4013E1F: dl_open_worker (dl-open.c:268) 
==27802== by 0x400F175: _dl_catch_error (dl-error.c:178) 
==27802== by 0x4013319: _dl_open (dl-open.c:639) 
==27802== by 0x517F601: do_dlopen (dl-libc.c:89) 
==27802== by 0x400F175: _dl_catch_error (dl-error.c:178) 
==27802== by 0x517F6C3: __libc_dlopen_mode (dl-libc.c:48) 
==27802== by 0x4E423BB: pthread_cancel_init (unwind-forcedunwind.c:53) 
==27802== by 0x4E4257B: _Unwind_ForcedUnwind (unwind-forcedunwind.c:130) 
==27802== by 0x4E4069F: __pthread_unwind (unwind.c:130) 
==27802== by 0x4E3AFF4: pthread_exit (pthreadP.h:265) 
==27802== 
==27802== 1,174 bytes in 1 blocks are still reachable in loss record 4 of 4 
==27802== at 0x4C29DB4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) 
==27802== by 0x400B57D: _dl_new_object (dl-object.c:77) 
==27802== by 0x4006805: _dl_map_object_from_fd (dl-load.c:1051) 
==27802== by 0x4008699: _dl_map_object (dl-load.c:2568) 
==27802== by 0x401384A: dl_open_worker (dl-open.c:225) 
==27802== by 0x400F175: _dl_catch_error (dl-error.c:178) 
==27802== by 0x4013319: _dl_open (dl-open.c:639) 
==27802== by 0x517F601: do_dlopen (dl-libc.c:89) 
==27802== by 0x400F175: _dl_catch_error (dl-error.c:178) 
==27802== by 0x517F6C3: __libc_dlopen_mode (dl-libc.c:48) 
==27802== by 0x4E423BB: pthread_cancel_init (unwind-forcedunwind.c:53) 
==27802== by 0x4E4257B: _Unwind_ForcedUnwind (unwind-forcedunwind.c:130) 
==27802== 
==27802== LEAK SUMMARY: 
==27802== definitely lost: 0 bytes in 0 blocks 
==27802== indirectly lost: 0 bytes in 0 blocks 
==27802==  possibly lost: 0 bytes in 0 blocks 
==27802== still reachable: 1,558 bytes in 4 blocks 
==27802==   suppressed: 0 bytes in 0 blocks 
==27802== 
==27802== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2) 
--27802-- 
--27802-- used_suppression:  2 dl-hack3-cond-1 
==27802== 
==27802== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2) 

Valgrind Ausgang für Code-Schnipsel 1 (kein Mem-leak, einige läuft später)

--29170-- Discarding syms at 0x64168d0-0x6426198 in /lib/x86_64-linux-gnu/libgcc_s.so.1 due to munmap() 
==29170== 
==29170== HEAP SUMMARY: 
==29170==  in use at exit: 0 bytes in 0 blocks 
==29170== total heap usage: 105 allocs, 105 frees, 28,814 bytes allocated 
==29170== 
==29170== All heap blocks were freed -- no leaks are possible 
==29170== 
==29170== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2) 
--29170-- 
--29170-- used_suppression:  2 dl-hack3-cond-1 
==29170== 
==29170== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2) 

Answer 1

Sie haben einen Fehler, wenn Ihre Threads losgelöst sind und zu undefiniertem Verhalten führen.

In Haupt haben Sie diese Codezeile:

struct args_for_job_t args[MAX_THREADS]; 

, die Sie Hand von Zeigern auf Ihre Arbeitsthreads.

Dann main() erreicht dieser Teil

pthread_exit(NULL); 

Und main() nicht mehr existiert, aber man kann immer noch um Worker-Threads haben, dass die oben genannten args Array zugreift, die von Haupt auf dem Stapel ist() - was existiert nicht mehr. Ihre Worker-Threads enden möglicherweise alle, bevor main() in einigen Läufen endet, aber nicht in anderen Läufen.