[TODO] Dir.glob is slow
Reported by Kevin Clark | January 8th, 2008 @ 04:04 PM | in 1.0 preview
It didn't start slow, but it got slow. Here's my jumping around the tree benching.
Odysseus:zss kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } }"
2.542118 0.000000 2.542118 ( 2.542121)
3.178957 0.000000 3.178957 ( 3.178955)
3.289544 0.000000 3.289544 ( 3.289537)
3.076772 0.000000 3.076772 ( 3.076762)
3.381311 0.000000 3.381311 ( 3.381303)
Odysseus:zss kev$ # The stuff above was from HEAD
Odysseus:zss kev$ # Next stuff from 645f30882c9 (when I checked in)
Odysseus:zss kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } }"
0.216285 0.000000 0.216285 ( 0.216286)
0.216778 0.000000 0.216778 ( 0.216774)
0.216331 0.000000 0.216331 ( 0.216321)
0.216141 0.000000 0.216141 ( 0.216133)
0.225699 0.000000 0.225699 ( 0.225686)
Odysseus:zss kev$ # HEAD from before was b804fce737247
Odysseus:zss kev$ # from 5299125844b0Odysseus:zss kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } }"
0.227294 0.000000 0.227294 ( 0.227290)
0.217481 0.000000 0.217481 ( 0.217476)
0.217489 0.000000 0.217489 ( 0.217471)
0.217344 0.000000 0.217344 ( 0.217336)
0.215922 0.000000 0.215922 ( 0.215913)
Odysseus:zss kev$ # from b373e9f2826Odysseus:zss kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } }"
0.217015 0.000000 0.217015 ( 0.217016)
0.217152 0.000000 0.217152 ( 0.217142)
0.228356 0.000000 0.228356 ( 0.228346)
0.216791 0.000000 0.216791 ( 0.216782)
0.216279 0.000000 0.216279 ( 0.216270)
Odysseus:zss kev$ # from 65998d601a
Odysseus:zss kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } }"
0.263701 0.000000 0.263701 ( 0.263707)
0.258497 0.000000 0.258497 ( 0.258491)
0.222697 0.000000 0.222697 ( 0.222687)
0.219437 0.000000 0.219437 ( 0.219428)
0.218945 0.000000 0.218945 ( 0.218934)
Odysseus:zss kev$ # from 8ffd9dc06c51Odysseus:zss kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } }"
0.232451 0.000000 0.232451 ( 0.232449)
0.223635 0.000000 0.223635 ( 0.223629)
0.222265 0.000000 0.222265 ( 0.222257)
0.222324 0.000000 0.222324 ( 0.222316)
0.221149 0.000000 0.221149 ( 0.221139)
Odysseus:zss kev$ # from 9976301ba0
Odysseus:zss kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } }"
0.264747 0.000000 0.264747 ( 0.264752)
0.258253 0.000000 0.258253 ( 0.258246)
0.223186 0.000000 0.223186 ( 0.223177)
0.219662 0.000000 0.219662 ( 0.219654)
0.219553 0.000000 0.219553 ( 0.219543)
Odysseus:zss kev$ # from da83dd80169
Odysseus:zss kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } }"
0.265317 0.000000 0.265317 ( 0.265311)
0.258382 0.000000 0.258382 ( 0.258375)
0.223628 0.000000 0.223628 ( 0.223618)
0.220182 0.000000 0.220182 ( 0.220173)
0.219035 0.000000 0.219035 ( 0.219027)
Odysseus:zss kev$ # from 30f20cfbd67
Odysseus:zss kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } }"
0.229125 0.000000 0.229125 ( 0.229127)
0.228557 0.000000 0.228557 ( 0.228547)
0.241010 0.000000 0.241010 ( 0.241006)
0.228068 0.000000 0.228068 ( 0.228059)
0.227721 0.000000 0.227721 ( 0.227732)
Odysseus:zss kev$ # from 90a3189
Odysseus:zss kev$ # segfault
Odysseus:zss kev$ # from 205c51283
Odysseus:zss kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } }"
An exception has occurred:
Unknown field d_name (RuntimeError)
Backtrace:
FFI::Struct(Dir::DirEntry)#[] at kernel/platform/ffi.rb:229
Dir#read at kernel/core/dir.rb:203
Dir.foreach {} at kernel/core/dir.rb:167
Dir.open at kernel/core/dir.rb:144
Dir.foreach at kernel/core/dir.rb:166
Dir.glob at kernel/core/dir.rb:25
main.__eval_script__ {} at (eval):1
Benchmark.measure at /Users/kev/code/rbx/stdlib/benchmark.rb:296
main.__eval_script__ {} at (eval):1
Integer(Fixnum)#times at kernel/core/integer.rb:15
main.__eval_script__ at (eval):1
CompiledMethod#activate at kernel/core/compiled_method.rb:124
Compile.execute at kernel/core/compile.rb:51
main.__script__ at kernel/loader.rb:171
Odysseus:zss kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } }"
An exception has occurred:
Unknown field d_name (RuntimeError)
Backtrace:
FFI::Struct(Dir::DirEntry)#[] at kernel/platform/ffi.rb:229
Dir#read at kernel/core/dir.rb:203
Dir.foreach {} at kernel/core/dir.rb:167
Dir.open at kernel/core/dir.rb:144
Dir.foreach at kernel/core/dir.rb:166
Dir.glob at kernel/core/dir.rb:25
main.__eval_script__ {} at (eval):1
Benchmark.measure at /Users/kev/code/rbx/stdlib/benchmark.rb:296
main.__eval_script__ {} at (eval):1
Integer(Fixnum)#times at kernel/core/integer.rb:15
main.__eval_script__ at (eval):1
CompiledMethod#activate at kernel/core/compiled_method.rb:124
Compile.execute at kernel/core/compile.rb:51
main.__script__ at kernel/loader.rb:171
Odysseus:zss kev$ # from a1a29f1b2
Odysseus:zss kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } }"
An exception has occurred:
Unknown field d_name (RuntimeError)
Backtrace:
FFI::Struct(Dir::DirEntry)#[] at kernel/platform/ffi.rb:229
Dir#read at kernel/core/dir.rb:203
Dir.foreach {} at kernel/core/dir.rb:167
Dir.open at kernel/core/dir.rb:144
Dir.foreach at kernel/core/dir.rb:166
Dir.glob at kernel/core/dir.rb:25
main.__eval_script__ {} at (eval):1
Benchmark.measure at /Users/kev/code/rbx/stdlib/benchmark.rb:296
main.__eval_script__ {} at (eval):1
Integer(Fixnum)#times at kernel/core/integer.rb:15
main.__eval_script__ at (eval):1
CompiledMethod#activate at kernel/core/compiled_method.rb:124
Compile.execute at kernel/core/compile.rb:51
main.__script__ at kernel/loader.rb:171
Odysseus:zss kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } }"
An exception has occurred:
Unknown field d_name (RuntimeError)
Backtrace:
FFI::Struct(Dir::DirEntry)#[] at kernel/platform/ffi.rb:229
Dir#read at kernel/core/dir.rb:203
Dir.foreach {} at kernel/core/dir.rb:167
Dir.open at kernel/core/dir.rb:144
Dir.foreach at kernel/core/dir.rb:166
Dir.glob at kernel/core/dir.rb:25
main.__eval_script__ {} at (eval):1
Benchmark.measure at /Users/kev/code/rbx/stdlib/benchmark.rb:296
main.__eval_script__ {} at (eval):1
Integer(Fixnum)#times at kernel/core/integer.rb:15
main.__eval_script__ at (eval):1
CompiledMethod#activate at kernel/core/compiled_method.rb:124
Compile.execute at kernel/core/compile.rb:51
main.__script__ at kernel/loader.rb:171
Odysseus:zss kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } }"
An exception has occurred:
Unknown field d_name (RuntimeError)
Backtrace:
FFI::Struct(Dir::DirEntry)#[] at kernel/platform/ffi.rb:229
Dir#read at kernel/core/dir.rb:203
Dir.foreach {} at kernel/core/dir.rb:167
Dir.open at kernel/core/dir.rb:144
Dir.foreach at kernel/core/dir.rb:166
Dir.glob at kernel/core/dir.rb:25
main.__eval_script__ {} at (eval):1
Benchmark.measure at /Users/kev/code/rbx/stdlib/benchmark.rb:296
main.__eval_script__ {} at (eval):1
Integer(Fixnum)#times at kernel/core/integer.rb:15
main.__eval_script__ at (eval):1
CompiledMethod#activate at kernel/core/compiled_method.rb:124
Compile.execute at kernel/core/compile.rb:51
main.__script__ at kernel/loader.rb:171
Odysseus:zss kev$ # from 83fb4decc
Odysseus:zss kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } }"
An exception has occurred:
Unknown field d_name (RuntimeError)
Backtrace:
FFI::Struct(Dir::DirEntry)#[] at kernel/platform/ffi.rb:229
Dir#read at kernel/core/dir.rb:205
Dir.foreach {} at kernel/core/dir.rb:169
Dir.open at kernel/core/dir.rb:146
Dir.foreach at kernel/core/dir.rb:168
Dir.glob at kernel/core/dir.rb:25
main.__eval_script__ {} at (eval):1
Benchmark.measure at /Users/kev/code/rbx/stdlib/benchmark.rb:296
main.__eval_script__ {} at (eval):1
Integer(Fixnum)#times at kernel/core/integer.rb:15
main.__eval_script__ at (eval):1
CompiledMethod#activate at kernel/core/compiled_method.rb:126
Compile.execute at kernel/core/compile.rb:51
main.__script__ at kernel/loader.rb:171
Odysseus:zss kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } }"
An exception has occurred:
Unknown field d_name (RuntimeError)
Backtrace:
FFI::Struct(Dir::DirEntry)#[] at kernel/platform/ffi.rb:229
Dir#read at kernel/core/dir.rb:205
Dir.foreach {} at kernel/core/dir.rb:169
Dir.open at kernel/core/dir.rb:146
Dir.foreach at kernel/core/dir.rb:168
Dir.glob at kernel/core/dir.rb:25
main.__eval_script__ {} at (eval):1
Benchmark.measure at /Users/kev/code/rbx/stdlib/benchmark.rb:296
main.__eval_script__ {} at (eval):1
Integer(Fixnum)#times at kernel/core/integer.rb:15
main.__eval_script__ at (eval):1
CompiledMethod#activate at kernel/core/compiled_method.rb:126
Compile.execute at kernel/core/compile.rb:51
main.__script__ at kernel/loader.rb:171
Odysseus:zss kev$ cd ../rbx/Odysseus:rbx kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; Dir.chdir('../zss') { 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } } }"
0.333827 0.000000 0.333827 ( 0.333828)
0.257736 0.000000 0.257736 ( 0.257731)
0.249342 0.000000 0.249342 ( 0.249334)
0.250615 0.000000 0.250615 ( 0.250606)
0.251102 0.000000 0.251102 ( 0.251093)
Odysseus:rbx kev$ # NEED TO REVIEW THIS, break in FFI?
Odysseus:rbx kev$ # from 9371bb2965bOdysseus:rbx kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; Dir.chdir('../zss') { 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } } }"
0.331061 0.000000 0.331061 ( 0.331059)
0.255369 0.000000 0.255369 ( 0.255363)
0.248674 0.000000 0.248674 ( 0.248665)
0.248669 0.000000 0.248669 ( 0.248660)
0.249097 0.000000 0.249097 ( 0.249089)
Odysseus:rbx kev$ # from d21810882
Odysseus:rbx kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; Dir.chdir('../zss') { 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } } }"
0.322466 0.000000 0.322466 ( 0.322462)
0.239516 0.000000 0.239516 ( 0.239507)
0.239084 0.000000 0.239084 ( 0.239069)
0.233033 0.000000 0.233033 ( 0.233024)
0.237686 0.000000 0.237686 ( 0.237677)
Odysseus:rbx kev$ # from 5ce7bf934
Odysseus:rbx kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; Dir.chdir('../zss') { 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } } }"
0.362550 0.000000 0.362550 ( 0.362547)
0.286700 0.000000 0.286700 ( 0.286691)
0.286995 0.000000 0.286995 ( 0.286986)
0.286668 0.000000 0.286668 ( 0.286668)
0.286512 0.000000 0.286512 ( 0.286502)
Odysseus:rbx kev$ # from 6e4f65103
Odysseus:rbx kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; Dir.chdir('../zss') { 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } } }"
0.329941 0.000000 0.329941 ( 0.329935)
0.253551 0.000000 0.253551 ( 0.253543)
0.245977 0.000000 0.245977 ( 0.245968)
0.247011 0.000000 0.247011 ( 0.247004)
0.249790 0.000000 0.249790 ( 0.249782)
Odysseus:rbx kev$ # from 71d9a4144
Odysseus:rbx kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; Dir.chdir('../zss') { 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } } }"
0.341340 0.000000 0.341340 ( 0.341336)
0.263697 0.000000 0.263697 ( 0.263687)
0.265676 0.000000 0.265676 ( 0.265666)
0.261816 0.000000 0.261816 ( 0.261809)
0.261817 0.000000 0.261817 ( 0.261809)
Odysseus:rbx kev$ # from 91d46b86a8
Odysseus:rbx kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; Dir.chdir('../zss') { 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } } }"
0.343331 0.000000 0.343331 ( 0.343328)
0.263947 0.000000 0.263947 ( 0.263941)
0.265847 0.000000 0.265847 ( 0.265839)
0.261642 0.000000 0.261642 ( 0.261630)
0.265849 0.000000 0.265849 ( 0.265840)
Odysseus:rbx kev$ # from 6f448f0dd72b
Odysseus:rbx kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; Dir.chdir('../zss') { 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } } }"
0.327299 0.000000 0.327299 ( 0.327297)
0.240275 0.000000 0.240275 ( 0.240266)
0.239644 0.000000 0.239644 ( 0.239632)
0.238283 0.000000 0.238283 ( 0.238275)
0.233701 0.000000 0.233701 ( 0.233692)
Odysseus:rbx kev$ # from f957b5a5881
Odysseus:rbx kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; Dir.chdir('../zss') { 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } } }"
Toplevel exception detected.
Error: An unhandled exception has terminated this VM.
=> wrong number of arguments (got 1, required 2) (ArgumentError)
Unable to run /Users/kev/code/rbx/runtime/platform
Odysseus:rbx kev$ # from 82bf3156236
Odysseus:rbx kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; Dir.chdir('../zss') { 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } } }"
2.201456 0.000000 2.201456 ( 2.201455)
2.095360 0.000000 2.095360 ( 2.095355)
2.107633 0.000000 2.107633 ( 2.107625)
2.110066 0.000000 2.110066 ( 2.110058)
2.122973 0.000000 2.122973 ( 2.122965)
Odysseus:rbx kev$ # from 057a9304d9d
Odysseus:rbx kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; Dir.chdir('../zss') { 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } } }"
0.329986 0.000000 0.329986 ( 0.329985)
0.241954 0.000000 0.241954 ( 0.241955)
0.240005 0.000000 0.240005 ( 0.239998)
0.238918 0.000000 0.238918 ( 0.238910)
0.233695 0.000000 0.233695 ( 0.233688)
Odysseus:rbx kev$ # from e09d7f3f681c67
Odysseus:rbx kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; Dir.chdir('../zss') { 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } } }"
2.340409 0.000000 2.340409 ( 2.340408)
2.262365 0.000000 2.262365 ( 2.262362)
2.269435 0.000000 2.269435 ( 2.269428)
2.270313 0.000000 2.270313 ( 2.270308)
2.279050 0.000000 2.279050 ( 2.279042)
Odysseus:rbx kev$ # from 0e0722db7d0
Odysseus:rbx kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; Dir.chdir('../zss') { 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } } }"
Toplevel exception detected.
Error: An unhandled exception has terminated this VM.
=> wrong number of arguments (got 1, required 2) (ArgumentError)
Unable to run /Users/kev/code/rbx/runtime/platform
Odysseus:rbx kev$ # from 34a425dbd87Odysseus:rbx kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; Dir.chdir('../zss') { 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } } }"
0.326095 0.000000 0.326095 ( 0.326093)
0.239172 0.000000 0.239172 ( 0.239165)
0.238133 0.000000 0.238133 ( 0.238124)
0.237303 0.000000 0.237303 ( 0.237296)
0.232216 0.000000 0.232216 ( 0.232208)
Odysseus:rbx kev$ # from 761154bOdysseus:rbx kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; Dir.chdir('../zss') { 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } } }"
0.329522 0.000000 0.329522 ( 0.329530)
0.241832 0.000000 0.241832 ( 0.241825)
0.240477 0.000000 0.240477 ( 0.240469)
0.249282 0.000000 0.249282 ( 0.249272)
0.238762 0.000000 0.238762 ( 0.238762)
Odysseus:rbx kev$ # from 38ee82398
Odysseus:rbx kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; Dir.chdir('../zss') { 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } } }"
Toplevel exception detected.
Error: An unhandled exception has terminated this VM.
=> wrong number of arguments (got 1, required 2) (ArgumentError)
Unable to run /Users/kev/code/rbx/runtime/platform
Odysseus:rbx kev$ # from 3094b9dc37ab2Odysseus:rbx kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; Dir.chdir('../zss') { 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } } }"
0.323946 0.000000 0.323946 ( 0.323943)
0.239257 0.000000 0.239257 ( 0.239251)
0.238281 0.000000 0.238281 ( 0.238273)
0.236703 0.000000 0.236703 ( 0.236695)
0.231414 0.000000 0.231414 ( 0.231404)
Odysseus:rbx kev$ # from 0308bd69f6
Odysseus:rbx kev$ ~/code/rbx/shotgun/rubinius -e "require 'benchmark'; Dir.chdir('../zss') { 5.times { puts Benchmark.measure { Dir.glob '**/*.rb' } } }"
Toplevel exception detected.
Error: An unhandled exception has terminated this VM.
=> wrong number of arguments (got 1, required 2) (ArgumentError)
Unable to run /Users/kev/code/rbx/runtime/platform
Comments and changes to this ticket
-
Eero Saynatkari January 20th, 2008 @ 01:58 PM
- → State changed from “new” to “open”
- → Title changed from “Dir.glob is slow” to “[TODO] Dir.glob is slow”
Any news, Kevin?
-
Evan Phoenix March 12th, 2008 @ 03:38 PM
- → State changed from “open” to “invalid”
Eric rewrote glob, so this ticket is stale.
Please Login or create a free account to add a new comment.
You can update this ticket by sending an email to from your email client. (help)
Create your profile
Help contribute to this project by taking a few moments to create your personal profile. Create your profile »
