I did two things that apparently solved the problem (>50k episodes without issue). 1. Move matplotlib.use("agg") to the __init__.py, because I was actually using Qt5Agg. This is a necessary fix regardless of this particular problem. 2. I "un-threaded" the saving of figures. This is likely the fix, but I still don't know why... Will mark as resolved regardless.

Idea: Currently we're making .savefig threaded, maybe creating a little helper that does .savefig and then .clear in a thread could help?

Running python -m pdb myscript.py now gave out the following RuntimeWarning:

I can confirm that the segfault is in fact the problem. Running without visdom logging still results in segfault, but much later (sample size 1, ca. 13000 episodes). The EOFError is then only the result of the dead sender. Random idea from the internet: Stack memory limit too low? Mine is currently at 8192MB (ulimit -s)