Experimenting with Generators

I’ve been experimenting with the redux-saga and generators recently.

For me, the key revelation is to understand how the generator/iterator syntax gives you a 2-way communication from some code that uses the generator function and the execution of the calls. Let’s use the word controller for the code that uses the iterator, and generatorFunction for the function with the yields.

So the controller calls:

iterator.next()

This will evaluate the code up to some line containing

yield someJsExpression;

The iterator is going to get back a value of someJsExpression in the return object from calling next(). That expression could be an Object, a Function, a String, etc.

The generatorFunction can use the result from calling yield, which is the value that is passed into the second call to iterator.next(). Note, the first call to next() advances the code in the generatorFunction up to the first yield, so passing a param to next there does nothing.

On the next call to

iterator.next(someValue)

The first yield will return that value.

The best way to understand this is to use an interactive JavaScript editor.

Here’re a few JS BIN examples, that can also work in the sandbox of Webstorm if you uncomment out the line to declare lodash/fp.

Simple Generator Function Example:

// noprotect
// line above is for generators
// lodash/fp is `_`

function firstYieldParam() {
  console.log('firstYieldParam called');
  return 'What generator gives back to controller first yield';
}

function* generator(param) {
  console.log('before the line with yield in the generator, param = ', param);
  const whatControllerGaveTheGenerator = yield firstYieldParam();
  const paramToSecondYield = `whatControllerGaveTheGenerator is ${whatControllerGaveTheGenerator}`;
  const whatControllerGaveTheGenerator2 = yield paramToSecondYield;
  console.log(`whatControllerGaveTheGenerator2 is ${whatControllerGaveTheGenerator2}`);
  return 'this is the return value from the generator';
}

function controller() {
  console.log(`Running the controller(), about to call generator()`);
  const iterator = generator({a: 1});
  //console.log(iterator);
  console.log(`About to call iterator.next`); // ('param for first time next is called ==> unused')`);
  const firstResult = iterator.next(); // 'param for first time next is called ==> unused.');
  console.log(`firstResult from iterator.next()`, firstResult); 
  const secondResult = iterator.next('param for second time next is called.');
  console.log(`secondResult from iterator.next()`, secondResult);
  const thirdResult = iterator.next('param for third time next is called.');
  console.log(`thirdResult from iterator.next()`, thirdResult);
}

controller();

Here’s an older version of this demo, with an explanatory screenshot.

Generator with Yield*

The key is that the yield* basically unfolds (aka inlines) the nested generator function.

// noprotect
// line above is for generators
// lodash/fp is `_`


function* g4() {
  yield* [1, 2, 3];
  return "foo";
}

var result;

function* g5() {
  result = yield* g4();
}

var iterator = g5();

console.log(iterator.next()); // { value: 1, done: false }
console.log(iterator.next()); // { value: 2, done: false }
console.log(iterator.next()); // { value: 3, done: false }
console.log(iterator.next()); // { value: undefined, done: true }, 
                              // g4() returned { value: "foo", done: true } at this point

console.log('result is', result);          // "foo"[]

Generators and Async Operations

The key here is that the express that you use for yield can be a function (or as in redux-saga, an object that contains a function).

// noprotect

// line above is for generators
// lodash/fp is `_`

// const _ = require('lodash/fp');

// Based on http://gajus.com/blog/2/the-definitive-guide-to-the-javascript-generators

const delayedFunc = _.curry((name, callback) => {
  setTimeout(() => {
    console.log(`calling callback(${name})`);
      
    // This is where we're calling the `advancer` with the `response`
    callback(name);
  }, 1000);
});


function controller(generator){
  const iterator = generator();
  let timesNextCalled = 0;
  const advancer = (response) => {
    
    // Notice that the iterator is a closure, so it can go to all the recursive calls.
    // Also, notice that this is where we send back the result from calling yield, so the
    // SECOND time this line is called, the value goes into variable `a`
    const state = iterator.next(response);
    timesNextCalled++;
    
    console.log(`state after iterator called ${timesNextCalled} times is`, state);
    
    // state has { value, done }
    if (!state.done) {
      // Note that the value is a function, the callback, which is the 
      // curried anonymous funcs below with the "advancer". Basically, this is
      // recursive. It's critical that that the value is a curried function!
      state.value(advancer);
    }
  }

  // First time advancer is called, there is no "response"
  advancer();
  return 'All Done! (but not yet with async operations)';
}


function* myGeneratorFunc() {
  console.log('These operations run in order.')

  // This is the curried function, that will take one more argument, the callback,
  // which is the "advancer" defined above.
  const a = yield delayedFunc('A'); 
  console.log("after first yield called");
  
  const b = yield delayedFunc('B');
  const c = yield delayedFunc('C');
  console.log(`After running all 3 delayed funcs, a = ${a}, b = ${b}, c = ${c}`);
  console.log('Done with sequential async operations.')
}

function run() {
  return controller(myGeneratorFunc);
}

console.log('About to call run');
const result = run();
console.log(`Done with run, result = ${result}`);

Here’s a video I made: