Skip to content

Add behavior registration and callback flow

Jeremy Jackson requested to merge jj-add-behavior-registration into master

Closes: #18 (closed)

One of the things that I think we've learned, is that we should be more intentional about defining our experiments up front. We can do a lot of useful validation and add clarity for developers very early in their workflow if we do this.

The implementation varies a bit from the issue, but ultimately matches the new control, candidate and variant(:name) interface that was defined in the deprecation of the use/try methods in !145 (merged).

The way this is implemented is actually using the same pattern as callbacks, and we just assign the result of any callback to an instance variable that we can return if there was nothing provided by any override blocks provided in the experiment block itself.

To define the experiment behaviors, you can use things like:

class ExampleExperiment < ApplicationExperiment
  # Calling `control` without anything will default to trying to call a
  # `control_behavior` method, which should be private.
  control

  # You can provide a block that will be instance_exec'd.
  control { 'control_block' }

  # You can specify a series of methods, and a block -- each method and
  # then the block will be called. Any return value of the last method or
  # the block will be the return value of the run call unless the
  # behavior has been overridden in the experiment block.
  control(:control_method1, :control_method2) { 'control_block' }

  # You can additionally use the standard callback options, like `if:`
  # and `unless:` if you didn't want to handle that variant under some
  # circumstance -- I'm not sure this is useful, but it's interesting
  # and plays into a couple longer term ideas that we might want to
  # consider.
  control -> { 'control_block_for_jejacks0n' }, if: -> { context.try(:user)&.username == 'jejacks0n' }

  # Right now you can only register a single callback chain for a given
  # behavior, but we might enable multiple in the future so you could do
  # things like:
  #control -> { 'control_block_for_jejacks0n' }, if: -> { context.try(:user)&.username == 'jejacks0n' }
  #control -> { 'control_block_for_everybody_else' }, if: -> { context.try(:user)&.username != 'jejacks0n' }

  # You can define and register a candidate... `candidate` is simply a
  # shortcut for calling `variant :candidate`
  candidate # will call `candidate_behavior`
  candidate :candidate_method # will call `candidate_method`
  candidate { 'candidate_block' } # block is called using instance_exec
  
  # Same with variant registration. You just have to provide a variant name.
  variant(:red) { 'red_variant' } # block is called

  # To document the order of calls, here, the blue lambda would be called,
  # the `blue_method`, and finally the blue block. The return value of
  # `experiment(:example, :blue).run` would be `"blue_block"`.
  variant(:blue, -> { 'blue_lambda' }, :blue_method) { 'blue_block' }

  # All behavior methods should be defined as `private` now.

  private

  def control_behavior; end
  def control_method1; end
  def control_method2; end
  def candidate_behavior; end
  def candidate_method; end
  def blue_method; 'blue_method'; end
end

There's a couple things that we can add as a follow up to this effort, like early validation of needing a control and at least one other variant. I think this is the foundation of a behavior registration and callback system, and we can explore what it enables as we move forward.

Edited by Jeremy Jackson

Merge request reports

Loading