“I’m noticing that the topic of empathy is starting to be discussed more and more and developer circles. In retrospect, it’s strange that it has taken so long to come to the fore.”
Early tech companies were attempts to make something possible that hadn’t previous been possible. Think of Microsoft, Intel, Palm…
Turns out we’ve mostly mined out that vein — most stuff that people want to be possible, that computers can easily do, is now possible.
So we’ve switched from the problem “how do we make this possible for the very patient?” to “how do we make it convenient for a bunch of people, mostly non-programmers?”
That’s a big change in what problem we solve. That’s also a lot of the reason for the rise of Rails. Rails doesn’t help that much with making totally-new things possible, but it helps a huge amount in seeing what people like and don’t via prototyping. A Rapid Prototyping environment is a terrible way to hug the edges of what’s just barely possible on your hardware, but it’s a great way to explore the space of now-possible things and see which ones are useful, convenient or marketable.
“Make it possible” is a goal where empathy is probably useless or even counterproductive. You are looking for things people think of as impossible as your target. You don’t (yet) care if what you’re doing is useful or desirable — thinking in those terms will probably get in your way. The question is “what can we do, even though we don’t know why we care yet?”
But “make it useful to humans” is a goal that is nothing but empathy.
Our industry went through huge changes after “Microsoft” or even “Google” stopped being the next who-you-want-to-be. Now it’s companies like SnapChat, Facebook, etc. Programmers are still collectively sneering at social media companies being worth huge valuations. But making things useful is going to be a much, much bigger deal than making them possible, in the end.
“The next Microsoft” still hasn’t been fully replaced by “the next Facebook” in our heads, at least outside of the VC ecosystem. We still sneer at things that facilitate human interaction instead of new tricks in physics or abstraction. That change is going to take awhile.
This isn’t really my blog post — it’s Brian Brushwood’s. But as his blog slowly, visibly decays, it makes me want to keep a copy somewhere I’m sure I can find it. So, here it is:
Lately a lot of young magicians have been asking me for advice, which has caused me to remember one of the most valuable correspondences of my life: one of the most mind-blowing letters I ever received, chock-full of insights that to this very day guide my career and philosophy in both creating and performing magic.
This is a pretty long post, but with Teller’s permission, I’d like to share with you the secrets he gave me 14 years ago to starting a successful career in magic.
Lots of you loved my old post on this. Thanks! But thanks to many fine questions, especially by one guy on Twitter, it’s time for an update! And also, naturally, the same things you loved last time. I’m not heartless.
Sometimes you’re sure that’s not the right place for that piece of code, but where does it go? “Refactor” is only a good answer if you know how to fix it.
In a Rails application, what kind of code goes where?
For code about your database or domain objects, the model is your first stop in Rails. Models are powerful, easy to test, reusable across applications and more like non-Rails code than most of Rails — familiar, even if you don’t know Rails yet.
If there’s a good way to put the code in your model, that’s usually a safe bet and a good idea.
Write tests for them too, of course! The model is also the easiest part of Rails to test.
With models, also remember that there’s no requirement that they be based on ActiveRecord. It’s fine for them to use a different data store (e.g. Redis, Cassandra, files.) It’s also fine for them to only exist in memory, or be loaded from a service API (e.g. Stripe) or created and sent to one (e.g. a MailGun mailer.)
It’s easy to put lots of code in your controllers, but it’s almost always a mistake. Business logic for your app should get out of the controller and into the model as quickly as possible. Logic about how things are shown to the user should go into the view or the view helpers. In general, the controller should be a tiny, thin glue layer putting together your other components.
A controversial but often good idea is to set exactly one instance variable in your controller to use in your view. That enforces single-purpose controller actions and single-purpose views… Which can be good or bad.
It also sometimes encourages dubious workarounds like serving a single “background” page that only loads a lot of AJAX requests, each one single-purpose, from the server. That’s architecturally clean, but can also be very slow. More to the point, it’s kind of a weird thing to force because it’s “clean.”
Having lots of logic in your views is a huge anti-pattern. Don’t do it. It’s hard to test, it’s hard to find, it’s hard to write sandwiched in between the HTML… Just don’t.
Instead, your views should contain HTML, variables that turn into HTML, and calls to helper methods that generate HTML — or whatever your final output format is. There should be no logic in there to test. No conditionals, minimal loops, no non-display methods. If you add an output format, there should be no code to repeat because all the interesting data transforms already happened, and no other output format cares about your HTML-only helpers. Right?
You’ll see loops used semi-frequently for lists, including lists of Partials (see later on.) That’s basically okay, but not great. The less logic, the better. Logic doesn’t just move code into a hard-to-test place. It also encourages the next programmer (like you in the future) to do it wrong too.
In Rails, Mailers are basically views that output email instead of HTML. That means that in terms of testing, code organization and helpers, they basically work the same way.
There’s some argument about this, and Rails code organization for mailers varies a bit. But when working it out, try to mostly think of it as a different view with the same kind of controller action and most things work themselves out.
If you have a repeated chunk of HTML that gets used multiple places, such as blog-style comments, forms for frequent objects or decorations around a page column, it can make sense to embed them in a Partial.
See the Rails documentation for Partials for details, but they even allow lists of objects to render the partial repeatedly (see “rendering collections” at the link above), which can eliminate pesky loops from your view code. Partials can nest other view code or other partials as well.
The big time not to use Partials when it seems like you should is for the very top-level organization of the page. For that, use a Layout.
Rails “helpers” are very specifically view helpers. They’re automatically included in views, but not in controllers or models. That’s on purpose.
Code in the application helper is included in all the views in your application. Code in other helpers is included in the corresponding view. If you find yourself writing big loops, method calls or other logic in the view but it’s clearly display logic, move it into a method in the helper.
Do you need something like a “controller helper” or “model helper”? You can put it in a parent class if that makes sense. Or you can put it into the /lib directory, or any of the alternate places for lib-type code (see below.)
A sometimes-pattern in Rails applications is presenters — view-specific objects that are created by the controller to be passed to the view.
This is another variation on the “controller should only return one thing” idea above — what if you return one complex, view-specific thing that encapsulates many different database models that you queried? Is that sufficiently “one thing?”
It’s an interesting question. But it does make it really easy to encapsulate all your view logic together, so that’s nice.
A presenter can be an excellent way to extract a big chunk of logic used to format several different models together when it’s not feeling like view helpers are a clean solution. You can put them into the view helpers directories, or under lib, or make a new subdirectory under app for them — if you do that last one, make sure to add it to the Rails search path so that you don’t have to require them all explicitly.
The /lib Directory
Every Rails app starts with a /lib directory, but not much explanation of it.
Remember that helpers are specifically view helpers? What if you wanted a controller helper? Or a model helper? Sometimes you can use a parent controller or parent model, but that’s not always the best choice.
If you want to write a helper module for non-view logic, the /lib directory is usually the best place to put it. For example, logging code or some kinds of error handling may be a cross-cutting concern like that.
Also, if you’re putting everything in the ApplicationController or ApplicationHelper, those can get big. Consider factoring some of that code out into helpers, or into modules in /lib.
Stuff in /lib isn’t automagically included for you like controllers and models. So you’ll need to explicitly require the file like you do in non-Rails Ruby, not just use the name of the class.
Shared Ruby objects like app-specific Exception types can also go here.
Often your Rails application will use Services. That can mean an external third-party service (e.g. MailGun, a hosted datastore, or HoneyBadger to catch exceptions.)
In the simplest cases, there’s a gem and you can just use it. That’s fine.
In the pretty simple cases, there’s an obvious way to tie the service in, such as an error catcher registering itself from an initializer under config/initializers.
If you’ll be dealing with some explicit object from the service, use a model. This can be an object in an external data store (e.g. Cassandra column groups, Redis hashes.)
Sometimes you’ll deal with a service through a wrapper object (e.g. the Gibbon API for MailChimp, or the OpenTok wrapper for WebRTC.) In those cases, the wrapper object can go in a gem, or in the /lib directory or (better) in a service object under app/service.
Sometimes an external service will produce objects or events. You have to be careful — sitting around and receiving events will block your Rails worker, which can be murder on the response time for a user waiting in a browser.
Rails actions are really designed for one-and-done handling where you receive a request and quickly produce a response to it. In general, this works best when you convert incoming events to HTTP requests or some similar thing Rails can respond to quickly. And the opposite direction (start a long-running process) works best when Rails can farm that out to an external server. See job queues like DelayedJob for a great example of how to do this.
Services vary a lot, is what I’m saying, and may sometimes not be a great match for how Rails does things. Rails is specifically designed around HTTP, and isn’t the best application architecture if you have some not-really-HTTP features like long-running connections, bidirectional traffic, tight response-time requirements or the need to always have a browser talk to the same stateful server. Rails is really poorly designed to act as a World of Warcraft server, for instance.
Sometimes you have reusable pieces in your application. A controller or model might be needed by multiple different Rails apps. A particular piece of logic for logging or display might be useful to a lot of different folks. You might even find a different way of doing things that most Rails apps would benefit from.
These are all cases where you want to create a new gem and have your applications use it instead of sharing a directory of code.
(Obviously, you’ll also use a lot of third-party gems. I’m not even counting those right now.)
These days it’s really easy to create a new gem, so don’t be intimidated. If you haven’t worked through the first free chapter of Rebuilding Rails, this may be a good time to do it — it’ll show you how to quickly, easily create and use a new gem.
(Brandon Hilkert also has a whole book on the subject!)
Third-Party (“vendor”) Code
Sometimes you’ll embed third-party code into your app, especially for older Rails apps. The name Rails uses for such things is “vendor”, which often goes in the path for assets (e.g. jQuery or D3) or gems if you copy them locally or libraries — often under lib/vendor or a similar path.
When you use Bundler to package gems into your app for deployment, it defaults to using a path with “vendor” in it for the same reason.
The best is not to include third-party code directly in your app, usually by keeping it in gems. But if you need or want to for some reason, include “vendor” in the path to let people know that’s what you’re doing. Also, don’t just edit code in a vendor path. As the name suggests, keep it in a form that could be downloaded from elsewhere.
So where do you put server-side code to validate or sanitize input?
If it’s simple, the controller is fine. A params[:input].strip is fine right in the controller. As it grows, though, that can fatten up your nice skinny controllers — not a good thing.
When it gets larger, you can refactor it into a controller parent class in some cases, but usually a module in /lib. Rarely it might want to be a gem. See the /lib section above for more ideas.
Jobs and Tasks
Rake tasks are generally put into lib/tasks. That’s great for tasks you’ll run from the command line. You’ll generally have one or more “topic” Ruby files organizing your tasks — the database-related tasks in one, the payment-related tasks in another, the email-related tasks in a third file and so on. Rake allows and encourages multiple tasks per namespace and multiple namespaces per file. Here’s a pretty good article on organizing Rake tasks.
Job Queue jobs (e.g. DelayedJob, Resque, Sidekiq, Sucker Punch) are usually put into classes, one file per class. That can clutter things up a bit. You can also, surprisingly often, make those be rake tasks as well. In those cases, don’t feel limited to one task per file when it could just be a whole file of “forward these ten job types to these ten rake tasks.”
I’m not a huge authority on Rails testing. I’ll start by pointing you at this excellent post on ThoughtBot about Rails testing and test types, which I refer to occasionally.
In general, tests come in multiple flavors — unit tests are very specific in what they test, don’t find integration bugs, are cheap to read, write and run and should immediately isolate exactly what the problem is. On the far other side of the spectrum, integration tests test everything, are very expensive to read, write and run and they often only give you a pretty vague idea of what went wrong.
Other tests form a spectrum between them, or just off one side or the other. That spectrum, in Rails, runs something like:
- Non-Railsy unit tests for plain Ruby objects
- Rails Model tests that mock (fake) all the database access
- Rails Model tests
- Rails Controller tests that don’t render views
- Rails Controller tests that check your actual view HTML
- Rails Integration tests
- “Smoke tests” using REST libraries that you point at a maybe-not-even-local server
Obviously I have not described every possible test configuration, but I hope it’s pretty clear what direction the spectrum runs.
If you do this for awhile, you’ll get a feel for what you like. Also for what tests catch production problems fast. Just remember that unit tests are cheaper, more exact, and catch fewer problems while integration tests are the opposite. And don’t be afraid to have tests from several different points along the spectrum.
Concerns, Exceptions and a Conclusion
Rails has a very specific, very unusual setup. I think it’s a good idea for small apps, but only use Rails until it hurts. If your application gets too big or complicated, the Rails code organization may hurt more than it helps you.
There are several “grow out of Rails” approaches to apply alternate architectures to the framework. From Hexagonal Rails to Objects on Rails to the more general Clean Ruby DCI approach. I won’t tell you which to use, but I’ll tell you that you’re better off starting with plain, simple Rails and growing out of it.
Most Rails apps, and even more Rails controllers, don’t need to get all that big. They often don’t need to change much. Why go complicated when simple is working great?
Interested in taking your Rails migration code to the next level? Welcome. You’re in the right place.
What’s that? Somebody in the back says they’re not already up on the basics? No problem. The Rails Guide on migrations is like most Rails guides — pretty darn awesome. It’ll have you adding tables, dropping columns and altering multi-column indices before you know it.
For the rest of you who know the basics, let’s learn what awesome Rails programmers already know about migrations.
Migrations can bring down a busy site in a hurry. Many things you do in a migration will lock the table you’re touching, which will block every SQL query and every Rails route that touches that table, sometimes for minutes or hours. You can’t “scale” your way out of it — you only get one database, and it’s locked for all app servers.
A question you’ll occasionally see: when do you run your migrations in production? There’s a standard answer, so let’s talk about it.
You’ll run your migrations before you deploy new code. In general, that means “every time you push code to production, you run all the migrations since the last time you pushed code to production. Do it before you replace the old code.”
Easy enough. Why?
First, because that’s the order you think about things. If you’re adding a new column to a table, you’ll want to make your code use it. If you run the migration first, that all works out nicely.
Migrations can be tricky, and it’s time-consuming to test them properly. One answer is to have other people look over your code.
In addition to the checks you can enforce with automated tools, there are a lot of things a human should look over. Let’s talk about some of the rules for code reviewing a migration that aren’t covered by the Good Migrations or Strong Migrations tools.
Keep Each Migration Small
Small migrations are (usually) happy migrations. It can be hard to roll back migrations, especially when they modify large tables. If your database supports rolling back schema changes at all, it can still take minutes or hours to do. Even a large, hard-to-divide operation like adding a column to a two-terabyte table is happiest if it’s mostly done by itself.