gpg-agent "forgetting" password for key, when getting too many requests

I’m running Ubuntu (via Regolith) and my gpg key is unlocked when I log in. I’m running multiple decrypt operations in parallel and I noticed, that if I get above 7, gpg-agent will "forget" that the key is already unlocked and I get prompted for a pinentry.

❯ gpg --version
gpg (GnuPG) 2.2.27
libgcrypt 1.10.1

I made a minimal working example to demonstrate this in python.

Create a test file to decrypt with: echo "something" | gpg --encrypt -o test.gpg. Running gpg --decrypt test.gpg in the shell does not prompt for the passphrase.

Using the below script, if the WORKERNUM is set below 8 (on my machine, but setting it to 1 I guess should work for any machine) the script is happily decrypting without asking for a passphrase. But if I crank it up to 8 or above I start getting requests for putting in my password, although it seems like not from every process, only some of them. The execution of the processes also obviously start to hang (I’m assuming they are waiting on gpg-agent).

import subprocess
import multiprocessing as mp
import time



the_queue = mp.Queue()
WORKERNUM = 7

def worker_main(queue):
    while True:
        msg = queue.get(True)
        print(time.time(), msg)
        out = subprocess.run(["gpg", "--decrypt", "test.gpg"], capture_output=True)
        print(msg, time.time(), out.stdout)


the_pool = mp.Pool(WORKERNUM, worker_main, (the_queue,))

counter = 0
while True:
    counter += 1
    the_queue.put(counter)
    print(the_queue.qsize())
    while the_queue.qsize() > 10:
        time.sleep(0.1)

I tried passing --batch to the decrypt command, but that didn’t change anything. I’ve been looking through the man pages for gpg and gpg-agent to see if anything is mentioned that could pertain to this, but I couldn’t really find anything. I have two questions:

a) why does this happen and
b) is there something I can configure so that instead of having to figure out the max size of the processing pool to avoid this, gpg handles this and I don’t get a pinentry

Asked By: fbence

||

After careful monitoring of journalctl I found that the actual error was Cannot allocate memory. Based on this discussion, gpg-agent can run out of secure memory if multiple thread are trying to access it. Setting auto-expand-secmem 100 in ~/.gnupg/gpg-agent.conf resolves the issue.

       --auto-expand-secmem n                                                                                                                                  Allow Libgcrypt to expand its secure memory area as required. The optional
value n is a non-negative integer with a suggested size in bytes of each
additionally allocated secure memory area. The value is rounded up to the next
32 KiB; usual C style prefixes are allowed. For an heavy loaded gpg-agent with
many concurrent connection this option avoids sign or decrypt errors due to out
of secure memory error returns.

Answered By: fbence
Categories: Answers Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.