If you had to choose, would you rather have refinements or Active Support core extensions built-in to the standard library?

It seems like the best argument for refinements is avoiding conflicts between libraries that have slightly different implementations of methods like String#camelize. Would we still need refinements if the best of Active Support was built-in to Ruby?

The argument for refinements is making it easy for library authors to have appealing syntax without polluting the global context. The alternative is "static" methods on modules (e.g. DataMapper.not(:age, 12) vs. :age.not => 12).

Adding ActiveSupport to Ruby would remove the need for refinements *for those specific extensions* but not for other extensions.

Purely lexical refinements (refinements that apply only to code at the same textual level or below) make this sort of sugar possible while still providing a hint to the reader that a refinement is in play.

Dynamic refinements, especially those imported into a block via module_eval, have the potential to silently explode the complexity of code, without enough added value, imho.

Refinements. Pushing something like Active Support into the standard library is short-sighted and not scalable. It only solves the problem for libraries that become popular enough to bring into stdlib. AS gets a pass because Rails is popular, but what about all the other super useful monkeypatching libs out there-- do we bring those into stdlib too?

The whole point of refinements is meant to solve this exact problem. We can have modular code outside of core that has limited scope. We don't need to have discussions about "blessing" these libraries; that's a win for decentralized development.

The biggest issue with refinements seems to be related to debugging, but I'm dubious about this. It seems like a solvable problem to me.

@lsegal I would love to see a fully sketched out proposal that describes how debugging would work, both for purely lexical refinements and those imported via module_eval. The former seems doable to me. The latter, not so much.

And I don't mean "it's possible". I mean "as a user, I would know that a bug I hit was caused by a refinement".

To clarify: I think we should be focusing our efforts on "how can we make dynamic refinements easier to debug?", rather than "how can we use less monkeypatched code?"-- if you put a bunch of smart people on that problem, I'm sure we could iterate on some nice improvements.

I'm actually going to be looking at the refinements impl. this week to see if I can make the debugability aspect easier. If anyone wants to help out we should discuss.

As a data point, the JavaScript proposal for scoped object extensions (aka Refinements) is specifically limited to lexical-only.

This proposal is explicitly based on the original Ruby proposal ("Very similar to Ruby refinements which are based on ClassBlocks") with some thought given to local reasoning and performance impact.

See wiki.ecmascript.org

@wycats I guess my naive question is the following: doesn't the VM know how to map bytecode back to the file/line it parsed? It had to load and then call on that refinement code somehow. If so, it would seem possible to me to extract that information from the VM somehow. The question would then be what that performance cost is and whether the cost is worthwhile for debugability.

@lsegal An example of a critical debugging feature would be active refinements in the backtrace.

Questions:

1) Would it break compatibility with existing consumers of the backtrace
2) Would it be possible to scrape the information out in the case of an exception, or would the runtime need to keep track always. At what cost?

Relatedly, how do dynamic refinements affect more advanced JITs? It may be that relatively slow runtimes can easily handle dynamic refinements, but that they wreak havoc on more intricate JITing strategies.

Based on my discussion with @headius and the JavaScript proposal, I suspect that more advanced JITs would have to slow down the entire runtime to support dynamic refinements.

I would naturally answer outside of the spirit of the question; AS is my answer, but only because they're good to harmless, and refinements are actively bad. That said, AS doesn't replace refinements.

Here's the thing: refinements are trying to solve a social problem with technical means; those who monkey-patch String with random things aren't going to use refinements properly, either. They may (in theory) help good people write better code, but they won't stop bad people from writing bad code.

Most of the worry about this stuff is just hand-wringing; I don't remember the last time I personally ran into a big because of a bad monkey-patch.

Just to clarify, I mentioned Active Support core extensions because they're the most popular and presumably are the most frequent cause of conflicts that could be avoided with refinements.

I'm not proposing that Active Support be merged into the standard library whole hog. We could start with only the methods that are common to Active Support, Facets, and extlib. The idea is that by providing a more useful standard library, we could avoid 80% of the monkey patching of core classes. If we had an 80/20 solution, perhaps we wouldn't feel the need to add refinements to the language.

The problem is refinements are pretty much meant for the 20% case. The 80% scenario is someone does straight Rails and doesn't care that String#camelize is everywhere. The 20% case is that someone uses AS and another gem and both implement 5.days.ago (or some other 20% sugar) with subtle but important incompatible implementation differences. Refinements don't need to exist in order to make it easier for a single well defined method to be added to String, they need to be added for the edge cases where multiple libraries have competing implementations of ill-defined APIs (and need to co-exist).

You can skirt the issue by calling this a "social" problem, but I'd disagree. #camelize was ill-defined until it was heavily used/implemented, too.

And that 20% case has an impact on 100% of calls to #send and 100% of the debugging/writing code time.

And the #camelize issue (which I am admittedly not familiar with) was able to be solved without refinements, yes? How often does this actually happen? I know you're referencing the 80/20 rule, but I'm pretty sure that I can count on one or maybe two hands the number of methods that this is a problem for, and then, a camelize gem would fix the issue. Why modify the language in a global way for this edge case? Seems bad.

I think many libraries implement their own String#camelize (or whatever) because they don't want to specify an Active Support dependency. If a library is only using a couple Active Support methods, it's common to just copy and paste those methods in. This can cause conflicts as the Active Support implementation changes over time. If the most popular among these methods were added to the standard library then people wouldn't copy-and-paste from Active Support, which I suspect is the cause of the bulk of these conflicts.

The String#camelize case is a real-world example from Right::AWS, which @wycats wrote about on his blog.

@lsegal I agree that String#camelize was ill-defined until it was heavily used/implemented...but now that it's well-defined and heavily used, why not move it into stdlib?

Refinements are not about "solving a social problem", they're about making it possible for library authors to write elegant code without worrying about leaking their extensions.

Today, it's considered bad practice to extend commonly used classes (both core classes, and classes in commonly used gems), because things might leak. As a result, people don't do it, and "I don't remember the last time I personally ran into a big because of a bad monkey-patch" is the state of affairs.

Today, Rails is free to extend commonly used classes for its own internal use via ActiveSupport, but other libraries cannot. Refinements are about giving libraries that are not the 800 pound gorilla the ability to do so.

The choice is not between refinements and AS. Framing this way makes the discussion pointless.

Discussing simple tools that would address the less than 1% of the cases where there is a true clash on some name would be helpful. Names are meaningful. If there is such ambiguity in naming that two different meanings are easily possible, then get more specific. Further, there is no guarantee that one can do whatever one wants in a language. All global class/module names are also places where conflicts can happen. We deal with it.

There is also a need to fix a great deal of the broken OO design that exists in Ruby itself and in major Ruby libraries. Mirrors and capabilities are two concepts that would be far more valuable to the language and ecosystem than refinements.

@brixen Feel free to reframe and take the discussion in a different direction. I'm happy to set the Active Support question aside. I imagined it could be an 80% solution that would appease proponents of refinements, but apparently it does not, so I'm happy to shift to a discussion of refinements on the merits.

@brixen Could you talk more about mirrors and capabilities and why these concepts are preferable to refinements? What would the API for these features look like? Have you thought about implementing these features in Rubinius (or CRuby) as a proof-of-concept?

What I object to is the rampant hand-waving about what problems actually exist and how refinements fix them. Every discussion of refinements is using tiny code samples and unrealistic scenarios. Unrealistic in the sense that they are extrapolations or fabrications that could have much simpler solutions, most of which involve just simple good OO.

If you have not read Kent Beck's Implementation Patterns, please do so. It has an excellent framing of how technical discussions can fail. Most of us are talking about refinements at completely different levels. As yet, no one has made a case for refinements to me that cannot be solved with something much more simple. That is a death blow to the whole idea, for me.

I don't have time to explain the concepts, sorry. :) There is plenty of material about them on the interwebs. Sandi Metz's book is a good resource for OO refreshers.

Yes, I plan on implementing Mirrors in Rubinius and using them heavily for the tools I'm building.

And no offense to anyone here, but that's what I'm going to spend my time on. Building.

Let's flip this around: can someone provide a couple of really solid examples of situations where a refinement would make an error much more difficult to debug (in a way that is not currently possible to do without refinements)? @wycats, I know you said you could think of a number. This isn't a challenge, but a pragmatic question. If we can get really concrete on exactly what kind of pain refinements will cause, we can have a better idea about how to solve them. Solving them might even include completely different avenues, like changing the way we solve these problems in Ruby (i.e. use mirrors).

My biggest concern about the dynamic binding of refinements is that you can't tell by looking at it what the following code will do:

def foo
bar { "hello".upcase }
end

If bar does a dynamic refinement of the given block, upcase could call pretty much anything. Object#method and friends can't see the refinement (perhaps a bug, but fixing it is gross), so you now have to look at bar's code, and potentially refinements bar applies, to even reason about the code.

This is a large part of why I believe "using" should be purely lexical. Never before in Ruby has it been impossible to look at a piece of code and know it will call the same method each time. That's a real possibility with dynamic refinements.

I believe it violates the principle of least surprise by a long shot. Ruby programmers expect that method dispatch works in a consistent way and refinements turn every . into a gamble. I know I'm being heavily reactionary, but I believe it's true that it makes ruby LESS predictable rather than more.

Charlie's point about `Object#method` and friends not working anymore is huge. This means that the effects of method dispatch are no longer observable.

I know most folks don't care about the implementation difficulties, but they are huge issues too. For example, the current implementation of dynamically-applied refinements (in module_eval blocks) causes a full global cache flush *every time*. This is a total nonstarter, but nobody has been able to figure out another way to implement it.

Unless it's possible to implement refinements in such a way that non-refined calls stay 100% consistent with current behavior and refined calls suffer only a modest performance hit, I don't see how the feature is worth it.

For details on this point, see @headius's full post on refinements to the ruby-core list.

Tim Felgentreff has started an investigation into mirrors for Ruby: github.com

It would be irresponsible of me to not link [io has a very clean mirror](viewsourcecode.org) here.

I think refinements are useful for some specific tasks; String#camelize is *not* one of them. I see no real gain of using `"foo".camelize` over `StringUtils.camelize("foo")`.

No, I think refinements should be used for DSLs. As @wycats mentioned: It's the difference between DataMapper.not(:age => 12) and `:age.not => 12`. Or `5.days.ago` instead of `DateUtilts.now.subtract(:days => 5)`.

And in that context I think that *lexical* refinements makes a whole lot more sense than dynamic refinements; both in terms of debugging and in terms of sanity.