Fix memory leak if ossl_bn_new() fails#1010
Open
ndossche wants to merge 2 commits intoruby:masterfrom
Open
Conversation
When that call fails, the `bn` BIGNUM is never freed in asn1integer_to_num(). To solve this, use rb_protect(). Example Valgrind report: ``` 32 (24 direct, 8 indirect) bytes in 1 blocks are definitely lost in loss record 11,113 of 25,910 malloc (at /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) CRYPTO_zalloc (at /usr/lib/x86_64-linux-gnu/libcrypto.so.3) BN_new (at /usr/lib/x86_64-linux-gnu/libcrypto.so.3) BN_bin2bn (at /usr/lib/x86_64-linux-gnu/libcrypto.so.3) <unknown stack frame> *asn1integer_to_num (ossl_asn1.c:136) *asn1integer_to_num_i (ossl_asn1.c:165) rb_protect (at /usr/lib/x86_64-linux-gnu/libruby-3.2.so.3.2.3) *decode_int (ossl_asn1.c:356) *int_ossl_asn1_decode0_prim (ossl_asn1.c:777) *ossl_asn1_decode0 (ossl_asn1.c:936) *ossl_asn1_decode_all (ossl_asn1.c:1058) <unknown stack frame> <unknown stack frame> <unknown stack frame> rb_vm_exec (at /usr/lib/x86_64-linux-gnu/libruby-3.2.so.3.2.3) <unknown stack frame> <unknown stack frame> rb_catch_obj (at /usr/lib/x86_64-linux-gnu/libruby-3.2.so.3.2.3) <unknown stack frame> <unknown stack frame> <unknown stack frame> rb_vm_exec (at /usr/lib/x86_64-linux-gnu/libruby-3.2.so.3.2.3) rb_yield (at /usr/lib/x86_64-linux-gnu/libruby-3.2.so.3.2.3) rb_ary_each (at /usr/lib/x86_64-linux-gnu/libruby-3.2.so.3.2.3) <unknown stack frame> <unknown stack frame> <unknown stack frame> rb_vm_exec (at /usr/lib/x86_64-linux-gnu/libruby-3.2.so.3.2.3) rb_yield (at /usr/lib/x86_64-linux-gnu/libruby-3.2.so.3.2.3) rb_ary_each (at /usr/lib/x86_64-linux-gnu/libruby-3.2.so.3.2.3) <unknown stack frame> <unknown stack frame> <unknown stack frame> rb_vm_exec (at /usr/lib/x86_64-linux-gnu/libruby-3.2.so.3.2.3) rb_yield (at /usr/lib/x86_64-linux-gnu/libruby-3.2.so.3.2.3) rb_ary_each (at /usr/lib/x86_64-linux-gnu/libruby-3.2.so.3.2.3) <unknown stack frame> <unknown stack frame> <unknown stack frame> rb_vm_exec (at /usr/lib/x86_64-linux-gnu/libruby-3.2.so.3.2.3) <unknown stack frame> <unknown stack frame> rb_catch_obj (at /usr/lib/x86_64-linux-gnu/libruby-3.2.so.3.2.3) <unknown stack frame> <unknown stack frame> <unknown stack frame> rb_vm_exec (at /usr/lib/x86_64-linux-gnu/libruby-3.2.so.3.2.3) rb_vm_invoke_proc (at /usr/lib/x86_64-linux-gnu/libruby-3.2.so.3.2.3) rb_proc_call_kw (at /usr/lib/x86_64-linux-gnu/libruby-3.2.so.3.2.3) ```
rhenium
reviewed
Feb 26, 2026
ext/openssl/ossl_asn1.c
Outdated
Comment on lines
140
to
143
| @@ -137,9 +144,11 @@ asn1integer_to_num(const ASN1_INTEGER *ai) | |||
Member
There was a problem hiding this comment.
I think a cleaner solution may be to prepare a wrapped BIGNUM first and then let ASN1_*_to_BN() write into it directly. ossl_pkey_ec.c does this in a few places.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When that call fails, the
bnBIGNUM is never freed in asn1integer_to_num(). To solve this, use rb_protect().Example Valgrind report:
This was found by a hybrid static-dynamic analyser that looks for inconsistent handling of error checks in bindings.