I finally got around to reading "How does Sidekiq really work?" by Dan Svetlov dansvetlov.me. Dang, it did not disappoint. What an enjoyable and in-depth read! A few things that stuck out to me 1/11 🧵
dansvetlov.me
1) @mike.contribsys.com is a exceedingly clever! 2) I've used Ruby for 12+ years now, and I learned new Hash methods from this article 3) The design of Ruby thread interrupts seem heavily influenced by linux signals 4) My next article "when good threads go bad" will pair nicely with this! 2/11 🧵 1) @mike.contribsys.com you clever devil, that Signal.trap approach is ingenious! In my post on thread interrupts, I mentioned that you cannot use rails.logger inside of a signal.trap: bit.ly. Similarly, you cannot use the Sidekiq logger in trap, which is inconvenient. 3/11 🧵
The full reasons for that are explained in docs.ruby-lang.org, and have to do with a concept called "reentrancy". But mike bypasses all of that by using using IO.pipe. The io writer is safe to write to in the trap, so that's all it does, it writes the signal. 4/11 🧵
docs.ruby-lang.org
Meanwhile, the main loop for sidekiq is always waiting on the io reader, calling a blocking method, `wait_readable` in the while loop. Signal received -> write the signal -> wait_readable returns truthy -> while loop runs and calls `handle_signal` with the signal read 5/11 🧵 `handle_signal` can do what it wants! No reentrancy requirements! No special requirements at all in fact - we're back in the regular flow of our program! Super clever. 2) Two weird hash methods big Hash doesn't want you to know... 6/11 🧵 You can change the default return value of a hash by using `default=` on the hash. docs.ruby-lang.org. Now instead of nil, you'll return whatever `default` you provided! Very handy. 7/11 🧵
`compare_by_identity` sets the hash to compare keys by their reference instead of their value. Not day-to-day useful but a fascinating option, and utilized nicely in Sidekiq 8/11 🧵 3) The design of Ruby thread interrupts seem heavily influenced by linux signals When referring to the linux signal docs, I noticed a startling resemblance - it included `raise` and `kill` functions, a list of pending signals, and a way to block signals using bitmasks with pthread_sigmask 9/11 🧵 If you read my article on ruby thread interrupts and bitmasks, this will be very familiar! jpcamara.com I asked koichi if signals were the inspiration, and he confirmed it. It was fun to see the connection and what can be an inspiration for central language apis! 10/11 🧵
4) "When good threads go bad" coming soon! Dan shows how jobs are executed in the `process` method, which has this hefty section utilizing `handle_interrupt`. He does a great job describing how it works, and my next article will dig deeper into if you're interested! 11/11 🧵