What Hooks does Ruby have for Metaprogramming?
Ruby lets you hook in and see (and change!) a lot of behavior of the core language. Methods, constants, classes, variables... Ruby lets you query them all, and change a lot about them.
These are just hooks -- things Ruby calls in response to something happening. That's different from, say, all the methods you can call to find our what's defined and how -- like instance variables, or method bindings, or...
Here's summaries and links for all the hooks I could find (thanks to Google and StackOverflow!):
Methods
- respond_to_missing?: a way to make sure your dynamic methods defined with method_missing also handle respond_to? correctly
- method_missing: called when a method cannot be found, potentially to allow dynamically defining one instead
- method_added: called whenever a method is added... which can be used to modify the method
- method_removed: called whenever a method is removed
- singleton_method_added: method added to the singleton class of the object, to be callable just on this one instance
- singleton_method_removed: method removed from singleton class
- method_undefined: a method has been undefined, with undef_method. Undef_method is different from remove_method because remove_method may still allow superclasses to define the method -- undef_method means it's gone entirely. singleton_method_undefined: called when a singleton method is undefined entirely
- initialize_copy: an optional callback when cloning any Object
Classes
Modules
- append_features: a Module is included, and its constants, methods and variables used
- included: a Module is included, which usually obsoletes "append_features"
- extend_object: a Module extends an Object
- extended: an Object is extended by a module, which mostly obsoletes extend_object
- const_missing: a constant isn't already present
Marshalling
- marshal_dump: called on an object to have it dump itself in Ruby Marshal format
- marshal_load: called on an object to have it load itself in Ruby Mashal format
Coercion
- coerce: called by the first type in a two-argument operator on the second argument, to make it turn into something the first argument can recognize
- induced_from: deprecated, please don't use
- to_i, to_f, to_s, to_a, to_hash, to_proc and others: conversions, indicating that the object is being used as a type and should try to convert itself to that type
Does Ruby Have a Metaobject Protocol?
There's a semi-famous book, The Art of the Metaobject Protocol by Kiczales, des Rivieres and Bobrow. Alan Kay, the guy who invented SmallTalk and the phrase "Object Oriented", called it the best book in ten years.
But it's takes some describing.
What is a Metaobject Protocol?
You know how Ruby has a class called "Class"? And how all classes are instances of it? And how Class is a subclass of Module?
The Metaobject protocol asks, "what if there were more subclasses of Class? And you could make classes from them, instead of plain old Class?"
Also, it includes what we'd now call introspection functions -- they didn't usually call it that twenty years ago when this was published.
But what, specifically, does that mean?
It includes:
- introspection on classes, methods and instances
- runtime definition of classes, methods and instances
- specialized initializers -- constructors, effectively
- subclasses of "Class" with special behavior, called Metaclasses
- hooks for computing the class precedence list, for things like variable definition and what methods are called when parent classes conflict
Most of this doesn't look too surprising. Clearly Ruby already has a lot of it.
How much of it?
Ruby Equivalents for Metaobject Protocol Hooks
Ruby already lets you do some really interesting things with your classes using hooks, as you metaprogrammers already know.
In Ruby, instead of a metaclass, you would make a subclass of Object that defined some hooks, and then classes that inherit from that class would get that behavior. Ruby's inherited and method_defined hooks, for instance, are very similar to redefining the inheritance or method definition behavior in a metaclass.
But the Metaobject protocol lets you do a few things with classes that Ruby doesn't (easily).
It defines a hook for "the class is all done being defined" ("finalize-inheritance") -- Ruby never prevents you from including modules or defining methods, so it has no such callback. It's never "final."
The Metaobject protocol lets you say "here is the order to look through parent classes when defining variables and what method gets called." That's not impossible in Ruby, but you'd have to do a lot of undefining and redefining in your classes -- it's clearly not easy.
The other big things that the Metaobject Protocol allows are mostly about generic functions, a very powerful Lisp construct that Ruby has no equivalent for. While we could debate whether Ruby should allow generic functions, it doesn't... And so there's no equivalents for them.
So... Does Ruby Have One, Or Not?
Ruby doesn't let you create a subclass of Class -- try it in irb!
So in the simple, literal sense it definitely does not.
And it doesn't easily let you calculate a class precedence list, so it's missing at least one major piece of functionality from the Lisp Metaobject Protocol...
But amusingly, you could absolutely write a gem for that! You'd implement all the inheritance and method_definition hooks and then calculate the method precedence as soon as somebody called a function...
So Ruby has nearly all the power of a Metaobject protocol, except for a lack of generic functions. Which could also be written as a gem (I'm so getting flamed for this).
Why Ruby Should Stay a Laughing Stock
Five minutes on the Internet will find you somebody who thinks Ruby shouldn't exist and is clearly inferior (try it!). Or pick a big tech company, especially an "Enterprise" company. Same thing -- you'll find a Ruby-hater in five minutes if you're actually trying.
If Rubyists keep doing the right thing, this will be true forever. Let me explain.
What Ruby Rocks At
There's a set of things Ruby is really good at. I'll list some of them:
- Prototyping new ideas
- Anything web or HTTP
- Stuff that doesn't need to be fast
- Reading other people's code - Ruby is short and readable.
- Building tools and libraries, especially DSLs.
There are some other things Ruby is "good" at. Like, it's good at them but many people think they're a bad idea. I'll list some of those, too:
- Taking on a massive technical debt "loan", quickly
- Using lots of other people's code with minimal examination (gems!)
- Monkeypatching
- Patching up somebody else's code dynamically, in an un-maintainable way.
If you put list one and list two together, what should Ruby be really, really good at?
Ruby and Valley Culture
How about this?
"Ruby is absolutely amazing at quickly prototyping ideas for web sites and web apps, based on other people's code, with huge technical debt, when they haven't had to scale yet."
If you're working in a big enterprise company, that should sound like damning Ruby with faint praise.
If you're building a little Silicon Valley startup, you should get dollar signs in your eyes when you read that.
And that's the difference. That's why the ads on LinkedIn think Rubyists make about $15,000/yr more than Pythonistas right now, despite the two languages' parse trees looking like lost twins.
The Dark Side
Of course, that mad phat l3wt doesn't come without a price.
If thousands of Silicon Valley founders build their dubious ideas on Ruby, you'll see 90% of those projects sink -- they were bad ideas in the first place. And that's fine -- they failed cheap, and that's good.
But worse, 10% of those ideas will succeed, built in Ruby, and get bought by larger companies for their customers and engineers.
Which means that a flood of badly-built Ruby apps is coming, and it's coming because building in Ruby was exactly the right thing. And a whole generation of grizzled enterprise veterans are going to see nothing but Ruby apps which were (intentionally, correctly, appropriately) built with the absolute maximum amount of technical debt, in absolute minimum time, assuming it would all get rebuilt.
If Ruby keeps being the best, most powerful, most appropriate way to validate your app idea, it will remain forever a technical laughingstock.
The kind of engineers who ignore technical and business merits and just go based on what their Enterprise buddies say will always think Ruby is a ridiculous toy language and all Ruby code sucks.
Thank God.
I won't need to work with them ever again.
On Hiring a Developer
I got mail from somebody with a vision for an app, wanting to know how much it would cost to build their idea.
This is my advice to him, with a few specifics redacted:
Hey, (Name).
I only have an approximate answer, but I'll try. First off: 800-1000 hours at $55/hr isn't an unreasonable estimate at all for a mid-sized app. There are people who can get this done in a lot less than 800 hours... And they'll all charge more than $55/hr. So you're unlikely to get it done well and much cheaper than that. You could get it done badly, usually overseas, for much cheaper than that, but there are a lot of obvious pitfalls -- it could work, but it often doesn't.
You have a full, rich site idea -- you have a lot of ideas for "extra" features like categorizing shared (data). You haven't trimmed your idea down to its leanest form. The very best developers will work with you to refine the vision you have for the site and figure out what's important to your users. As a side effect, they may be able to do it faster be removing parts that aren't necessarily vital, such as the categorization... Or put off features that are good, but aren't 100% necessary at launch (following other people's shares?). Or find a way to use an existing network and base the friend/follow feature off that (Twitter account? Facebook?).
Some part of what you're trying to do is its "most unique" part, something that makes it different from other sites. Otherwise why would you spend thousands of dollars to build it? In that way, a really good developer can "tease out" that vision and build just the smallest thing that tests it. And they can do that in a lot less than 800-1000 hours. Then you can build only the later features that makes sense. Or if your initial idea didn't work, you can refine that instead of building "extra" features that will only matter when and if your first idea is really working.
But $55/hr would be very, very cheap for a really good developer, Rails or not. You can get a Rails dev for $55/hr, and they will eventually be able to do the things you list there on the features. Usually you should expect to pay more for a CA Rails developer, though, more like $75/hr.
But the best thing you can do is to refine your vision down to something smaller -- Startup guys call it an MVP ("minimum viable product"). Because it's much cheaper to test your ideas with an MVP than with a full built-out site... And until you have something fun that feels good, building a bigger site just burns your money and slows you down.
The other thing you should consider for a (data)-sharing site is design. How pretty do you need this to be? How good does it need to "feel" to users? Do you need nice visual transitions? If you're going to need a designer, do NOT have a dev build the site and then try to "paint" on the design afterward. The designer will have strong ideas about how to do things and you can get the most value from a good (and expensive!) designer by asking before you have the whole site decided.
Also, a designer may produce nice mockups that will show your developer exactly what to build. Communicating exactly what you want to a developer is very hard -- the feature list above could be any of hundreds of different site layouts. Single-page? One big Twitter-style feed? Separate SoundCloud-like sub-pages? A developer could do exactly what you said there and still disappoint you, or just produce something ugly and unusable.
If you're going to need a designer, figure that out early. If you aren't, make sure your developer has good work in their portfolio that they have designed. Also, expect to pay more.
The Ruby Community is a Loud, Bright, Wonderful Mess
Ruby and Python, so similar and so different. By which I mean, "the languages let you do the same things in most of the same ways, but the communities are utterly different."
Community Mores
Python says, there should be one — and preferably only one — obvious way to do it.
Ruby, like Perl, says there's more than one way to do it. More specifically, the community is okay with there being more than one way to do it.
Python tries very hard to keep a single consensus. The Ruby community says, "have fun, come back, show us what you built!"
And, sure, sometimes there are problems.
Problems and Solutions
If you take the best 2% of utter chaos, you get some chaos in the results. "Best" is hard to judge, right?
So you get wildly different tools and ideas every few years — RubyGems, Hoe, Bundler, RVM, Haml, Decent_Exposure)... And somehow, the community takes it all in good humor. There's a little fashion-mongering — "nobody's still using RVM, dahling, it's all rbenv these days!". But mostly, we take the best, adopt it and keep building great software.
The Ruby community reminds me, over and over, of Warren Ellis' Transmetropolitan. It's about a wonderfully dystopian future and a whacked-out journalist as your view into it...
This city never allowed itself to decay or degrade. It's wildly, intensely growing. It's a loud bright stinking mess.
It takes strength from its thousands of cultures, and the thousands more that grow anew each day.
— Spider Jerusalem, Transmetropolitan
The Ruby community is exactly that kind of loud, bright, stinking mess.
And as he says... "but God help me, I can't imagine living anywhere else."
The Art of the MetaObject Protocol
I'm reading a fine classic of programming, The Art of the MetaObject Protocol.
It's interesting how dated some of the assumptions in it are. For instance, it reads as though they're being intentionally obtuse, but only because I'm now so used to these ideas from Ruby -- when the book was written, this was obscure, fringe stuff and there wasn't a good vocabulary for it.
I had somehow gotten the impression that the book was about designing extended REST APIs, or what we'd now call a HyperMedia API. That impression now appears to be ten kinds of wrong.
Instead, it's about extending a language upwards, using macros, in many different directions at once.
Suddenly, I'm seeing possibilities that Ruby hasn't got, or has screwed up pretty thoroughly. Interesting.
Though it seems like the commentary on metaobject protocol in Ruby is pretty weak in general.
The Programmer's Secret Weapon for Code Spelunking
A lot of the time, it’s hard to find where Ruby has defined something. When you want to know, “what code did that?”, some languages have debugger watchpoints, Java has Eclipse, Ruby has…? What?
Ruby isn’t terribly friendly to IDEs. Dynamic method definition, highly configurable inheritance, libraries like ActiveRecord that reconfigure at runtime… IDEs have a lot of trouble finding the code you want in a highly dynamic language. With great power comes great responsibility. Which often means you’re suddenly on your own.
In Ruby, my go-to solution is command-line code search. I’ve used Ack for many years. While recursive find would take seconds or minutes, Ack would take well under ten seconds, consistently. It’s very fast, recursive, uses Perl-compatible regexps, very fast, lets you search by language, very fast, doesn’t require an IDE or project file and did I mention very fast? I used it constantly until about April.
And then I found Ag ("https://github.com/ggreer/the_silver_searcher"). It is THREE TIMES as fast as Ack. You know that jaw-hits-floor moment? When I first searched for something in a large code tree with Ag… It was faster than fast -- instant. I hit the return key and the output was just there. As fast as the next prompt popped up, so did a page and a half of search results.
When you’re looking for code, stop waiting on anything. Don’t wait on GNU find or your IDE. When you’re used to instant search you’ll have your next query half-composed by the time you hit return on your first one. Searching for code becomes a Read-Eval-Print Loop like in any good language. Like the OODA loop, you can tighten it up and everything just feels better.
Some quick gotchas in Ruby -- often a method is dynamically defined, so you can’t always find it by its name. Sometimes you’re looking for a table and ActiveRecord changes from snake_case to CamelCase. Regexps and case-insensitive search (-i) can really help you out here -- get used to using them.
Sometimes files aren’t searched by default. For instance, Rakefiles and .erb files may not be in Ack’s default idea of Ruby files. You can add an .ackrc and configure Ack exactly to your liking. That’s one of the better advantages over Ag, in fact. Consider using both of them, especially if you like speed but also need regexps occasionally -- and use a tool like Sack to automatically switch between them so you don’t have to remember when to use each.
Digging Into the Rails Source
You’ve made it mostway through your free Ruby on Rails Internals class. Today we’ll be poking around the Rails source code. If you have money but not time, Rebuilding Rails lets you find a lot quickly… But in case you have more time, we’ll see where in the Rails source you can learn the same things for yourself.
First off, Rails has a combination of terse, pithy code (easy to read!) and good, solid tests which help explain what a given piece of code does. I recommend switching back and forth between implementation and tests often when reading.
Head to Rails on GitHub. My screenshots are branch 3.2-stable, or you can look at latest. You can already see the division into gems with the various directories: actionmailer, actionpack, activemodel, activerecord, activeresource and activesupport.
You can see the controller and view code in “actionpack” (note: actionview has been extracted in Rails 4). Rebuilding Rails has an easy simplified version, but you can see the real thing in there with some work. Inside actionpack/lib are subdirectories like action_controller which provide a great start for a deep dive.
Rack
Later chapters of Rebuilding Rails talk a lot about Rack, including Rack Middleware. For that, you should look at http://github.com/rack/rack instead of Rails. Luckily, most of the Rack code you care about is quite short. Unluckily, it’s poorly documented. Again, make sure to read tests (where they exist) for best understanding. I don’t know another introduction as good as Chapter 8 of Rebuilding Rails, but you can get some of the same content from a my Ruby Hangout interview on Rack frameworks.
Dependencies
Rails also depends on a number of other libraries. It’s hard to understand ActiveRecord without Arel, for instance. Read through the .gemspec files when you want to know what Rails depends on. That’s also great when thinking, “huh, what else should I add to my own framework?” If you don’t know what a gem does, Google something like “arel gem” or “tzinfo gem” or “ruby mail gem”. You can also check out rubygems.org to track down info and documentation if you know its name.
In general, look at dependencies like Gemfile and .gemspec files for every library you look at. You can learn a lot about any library from what gems it uses.
Get the Code Locally
GitHub is a nice interface and all, but it’s no substitute for having the code locally where you can use it. Make sure to clone the code for the things you care about, and “bundle install” the dependencies you won’t be drilling into yet. If you use a Ruby IDE like RubyMine that may help. We’ll talk about what I use for exploration in the next lesson.
And Now, a Word From...
Incidentally - if you prefer the guided tour, considering buying Rebuilding Rails for your workplace. Companies pay more for your time than just your salary -- Rebuilding Rails is probably a good investment if it saves you just half an hour.
Will it, though? Start with the free chapters, and see if anything in there could save you ten minutes. Or just buy it, and if you’re not delighted, I’ll refund your money.
Web Servers and Application Servers
A reader recently asked me why Ruby web app deploys usually have a web server (NGinX, Apache) and an application server (Unicorn, Thin, Puma, Racer, Mongrel, Passenger, Jakarta, TorqueBox or whatever I've forgotten this week).
Actually, he asked, "why do I need to run NGinX and Unicorn?" It's a fair question.
My Answer
In some sense you don't need it. It's 100% possible to just run Unicorn for your site. The main reasons to run a web server as a reverse proxy (NGinX or Apache):
Speed. Both of those servers handle static requests much faster than any Ruby app server. They're designed for it.
Configurability. NGinX and Apache have many options related to security, SSL, rewriting URLs and request queueing with no equivalent in the app server. They also have an easier time running multiple different sites at once (example: running five subdomains with three different Ruby versions) for similar reasons.
Stability. The web servers fail more rarely and gracefully, partly because they have a much larger audience than all Ruby app servers together.
Support. If you need to add an option (example: fair round-robin request queueing), it's much easier to find a plugin for NGinX or Apache.
With that said, when you have a small simple site, you can totally run it off Unicorn and call it a day. You won't have as many configuration options available. You'll have more trouble with SSL support. You'll serve fewer requests (static ones for CSS, JS, etc. add up). You'll have slightly more downtime. But for small sites, you may not care about any of that.
Note that Passenger is a weird hybrid here. It's a Ruby app server, but it bakes in a copy of NGinX. So it's much closer to running with NGinX and an app server than Unicorn or Thin.
StartCom SSL Certificates
I need to be careful how I recommend StartCom SSL certificates from now on.
It turns out that on page 12 of the StartCom policy document they tell you that you can't use the free certificates for any commercial activities.
Their product descriptions, product comparisons and all other documentation I've found fail to mention this. They literally pointed me at the policy document when turning me down for a renewal -- that is, they have no simpler or more straightforward explanation.
That is, they're hiding it and then expecting me to not argue on renewal.
They've otherwise been great. No problems with service, a perfectly good web site for free/cheap certificates. Overall, I've had a very reasonable user experience... Until now.
It's perfectly understandable that they want money for a certificate if I'm going to use it for commercial purposes.
They're just taking a reasonable thing and presenting it dishonestly.
And now I'm paranoid about next renewal. What other carefully-buried terms did I not see?
