Go 1.23 adds iterators. An iterator is a way to provide values that can be used in for x := range iter loops.

People are happy the iterators were added to the language.

Not everyone is happy about HOW they were implemented. This person opined that they demonstrate “typical Go fashion of quite ugly syntax”.

The ugly

Are Go iterators ugly? Here’s the boilerplate of an iterator:

func IterNumbers(n int) func(func(int) bool) {
	return func(yield func(int) bool) {
      // ... the code
	}
}

Ok, that is kind of ugly. I can’t imagine typing it from memory.

The competition

We do not live in a vacuum. How do other languages implement iterators?

C++

I recently implemented DirIter class with an iterator in C++, for SumatraPDF.

I did it to so that I can write code like for (DirEntry* e : DirIter("c:\")) { ... } to read list of files in directory c:\.

Implementing it was no fun. I had to implement a class with the following methods:

begin()
end()
DirEntry* operator*()
operator==()
operator!=()
operator++()
operator++(int)

Oh my, that’s a lot of methods to implement.

A bigger problem is that the logic is complicated.

This is an example of pull iterator where the caller “pulls” next value out of the iterator. The caller needs at least two operations from an iterator:

give me next value
do you have more values?

In C++ it’s more complicated than that because “Overcomplication” is C++’s middle name.

A function that reads a list of entries in a directory is relatively simple.

The difficulty of implementing pull iterator comes from the need to track the current state of iteration to be able to provide “give me next value” function.

A simple directory traversal turned into complicated tracking of what I have read so far, did the process finish and reading the next directory entry.

C

C# also has pull iterators but they removed incidental complexity present in C++. It reduced the interface to just 2 essential methods:

T Next() which returns next element
bool HasMore() which tells if there are more values to read

Here’s an iterator that returns integers from 1 to n:

class NumberIterator {
  private int _current;
  private int _end;
  public NumberIterator(int n) {
    _current = 0;
    _end = n;
  }
  public bool HasMore() {
    return _current < _end;
  }
  public int Next() {
    if (!HasMore()) {
       throw new InvalidOperationException("No more elements.");
    }
    return ++_current;
  }
}

Much better but still doesn’t solve the big problem: the logic is split across many calls to Next()so the code needs to track the state.

C# push iterator with yield

Later C# improved this by adding a way to implement push iterator.

An iterator is just a function that “pushes” values to the caller using a yield statement.

Push iterator is much simpler:

static IEnumerable<int> GetNumbers(int n) {
  for (int i = 1; i <= n; i++) {
    yield return i;
  }
}

Clever and elegant

Here’s a Go version:

func GetNumbers(n int) func(func(int) bool) {
	return func(yield func(int) bool) {
		for i := i; i <= n; i++ {
			if !yield(i) {
				return
			}
		}
	}
}

The clever and elegant part is that Go designers figured out how to implement push iterators in a way very similar to C#’s yield without adding new keyword.

The hard part, the logic of the iterator, is equally simple as with yield.

The yield statement in C# is kind of magic. What actually happens is that the compiler rewrites the code inside-out and turns linear logic into a state machine.

Go designers figured out how to implement it using just a function.

It is true that there remains essential complexity: iterator is a function that returns a function that takes a function as an argument.

That is a mind bend, but it can be analyzed.

Instead of yield statement pushing values to the loop driver, we have a function.

This function is synthesized by the compiler and provided to the iterator function.

The argument to that function is the value we’re pushing to the loop.

It returns a bool to indicate early exit. This is needed to implement early break out of for loop.

An iterator function returns an iterator object. In Go case, the iterator object is a new function.

This creates a closure. If function is an iterator object then local variables of the function are state of the iterator.

I don’t know why Go designers chose this design over yield.

I assume the implementation is simpler so maybe that was the reason.

Or maybe they didn’t want to add new keyword and potentially break existing code.