Torture Testing for Ruby

Torture testing can be a very helpful technique for making your code more stable, especially when you're dealing with ruby libraries of which parts are written in C or C++. Torture testing basically means that you generate random input for your code and then see if your program behaves strange (eg generates a core dump).

Here is a trivial example of how to use the class TortureTesting with an array. It will call the specified methods in random order and with random parameters.

obj = [];
TortureTesting.run(n=2000,
[obj, :unshift, [:any]], # Random numeric, float, or string parameter
[obj, :unshift, [:string]], # Only random strings
[obj, :unshift, [:numeric]], # Only random numbers
[obj, :pop, [] ],
[obj, :unshift, ["foo"]], # Parameter not randomized
[obj, :unshift, [23]] # Parameter not randomized
)

torture.rb:

$stdout.sync = true
$big = 2**34
class TortureTesting
class << self
def string(size=rand(20))
@cs ||= (0..255).to_a
(0...size).collect { @cs[Kernel.rand(256)].chr }.join
end
def float(max=$big) rand * max end
def numeric(max=$big) rand(max) end
def any() self.send([:numeric, :float, :string][rand(3)]) end

def run(n=2000, *args) @calls = args; torture(n) end
def rand_param(k) return self.send(k) if Symbol === k; k end
def randomize_parameters(d) d.map {|p| rand_param(p) } end

def torture(n=2000)
puts "Starting torture: #{n}"
n.times {|c|
$stdout.write "#{c}\r"
obj, method, parameter_description = @calls[rand(@calls.size)]
begin
obj.send(*([method] + randomize_parameters(parameter_description)))
rescue Exception => e
end
}
end
end
end

(Thanks to Lincoln D. Stein's who first explained me how to use this idea for testing cgi programs.)