FFI Memory Leak

[James Edward Gray II]

I've found a memory leak in some FFI code. I'll give example code =
below, but I can explain the issue easily enough first.

When calling a function provided by Tokyo Cabinet, it returns a value in =
the form of a pointer and a length to read at that address. The =
documentation for the function (tchdbget() at =
http://1978th.net/tokyocabinet/spex-en.html#tchdbapi, if you are =
interested) warns that the returned region was allocated with malloc() =
and needs to be free()d. I can't figure out how to free the value, so =
there is a memory leak. What I would like to know is how to free the =
value at the end of the pointer.

Here's some code showing what I just explained.

#!/usr/bin/env ruby -wKU

require "rubygems"
require "ffi"

# map the C interface
module Lib
extend FFI::Library
ffi_lib(
*Array(
ENV.fetch(
"TOKYO_CABINET_LIB",
Dir["/{opt,usr}/{,local/}lib{,64}/libtokyocabinet.{dylib,so*}"]
)
)
)
=20
attach_function :tchdbnew, [ ], =
ointer
attach_function :tchdbopen, [ointer, :string, :int], =
:bool
attach_function :tchdbput, [ointer, ointer, :int, ointer,
:int], =
:bool
attach_function :tchdbget, [ointer, ointer, :int, ointer], =
ointer
attach_function :tchdbclose, [ointer], =
:bool
end

# translate the interface to Ruby
class TokyoCabinet
def self.open(*args)
db =3D new(*args)
yield db
ensure
db.close if db
end
=20
def initialize(path)
@db =3D Lib.tchdbnew
Lib.tchdbopen(@db, path, (1 << 1) | (1 << 2)) # write create mode
end
=20
def []=3D(key, value)
k, v =3D key.to_s, value.to_s
Lib.tchdbput(@db, k, k.size, v, v.size)
end
=20
def [](key)
k =3D key.to_s
size =3D FFI::MemoryPointer.newint)
value =3D Lib.tchdbget(@db, k, k.size, size)
value.address.zero? ? nil : value.get_bytes(0, size.get_int(0))
ensure
size.free if size
# FIXME: How do I free value here?
end
=20
def close
Lib.tchdbclose(@db)
end
end

# show the problem
def show_memory
3.times { GC.start } # try to clean up
mem =3D `ps -o rss -p #{Process.pid}`[/\d+/]
puts "Current memory: #{mem}"
end

TokyoCabinet.open("leak.tch") do |db|
db[:some_key] =3D "X" * 1024
10.times do
5000.times do
db[:some_key] # reading causes the memory leak
end
show_memory
end
end

# Sample Run:
#=20
# Current memory: 30324
# Current memory: 37828
# Current memory: 45364
# Current memory: 52896
# Current memory: 60428
# Current memory: 67960
# Current memory: 75488
# Current memory: 83020
# Current memory: 90552
# Current memory: 98080

__END__

Thanks in advance for any help provided.

James Edward Gray II

 

[John Mettraux]

I've found a memory leak in some FFI code. =A0I'll give example code belo=

w, but I can explain the issue easily enough first.

When calling a function provided by Tokyo Cabinet, it returns a value in =

the form of a pointer and a length to read at that address. =A0The document=
ation for the function (tchdbget() at http://1978th.net/tokyocabinet/spex-e=
n.html#tchdbapi, if you are interested) warns that the returned region was =
allocated with malloc() and needs to be free()d. =A0I can't figure out how =
to free the value, so there is a memory leak. =A0What I would like to know =
is how to free the value at the end of the pointer.

Hello,

maybe

http://gist.github.com/263033

or

http://gist.github.com/263034

Seems to work.


Merry Christmas,

--=20
John Mettraux - http://jmettraux.wordpress.com

 

[James Edward Gray II]

in the form of a pointer and a length to read at that address. The =
documentation for the function (tchdbget() at =
http://1978th.net/tokyocabinet/spex-en.html#tchdbapi, if you are =
interested) warns that the returned region was allocated with malloc() =
and needs to be free()d. I can't figure out how to free the value, so =
there is a memory leak. What I would like to know is how to free the =
value at the end of the pointer.

=20
Hello,
=20
maybe
=20
http://gist.github.com/263033
=20
or
=20
http://gist.github.com/263034
=20
Seems to work.

Yeah, those do seem to work. Awesome. Thanks for the help!

James Edward Gray II=

 

[John Mettraux]

Yeah, those do seem to work. =A0Awesome. =A0Thanks for the help!

Great news, I will then release a fresh rufus-tokyo with the fix (I'll
use free() directly, as in Tokyo Cabinet's C examples).


Merry Xmas,

--=20
John Mettraux - http://jmettraux.wordpress.com

 

[James Edward Gray II]

=20
Great news, I will then release a fresh rufus-tokyo with the fix (I'll
use free() directly, as in Tokyo Cabinet's C examples).

Yeah, there doesn't seem to be any advantage to using tcfree() that I =
can glean from the documentation.

Merry Xmas,

Same to you.

James Edward Gray II=