Iterable vs Iterator in Python

Let’s learn about iterables and iterators.

Iterator

In Python, an iterator is an object which implements the iterator protocol, which consist of the methods __iter__() and __next__() .

• An iterator is an object representing a stream of data.
• It returns the data one element at a time.
• A Python iterator must support a method called __next__() that takes no arguments and always returns the next element of the stream.
• If there are no more elements in the stream,__next__() must raise the StopIteration exception.
• Iterators don’t have to be finite.It’s perfectly reasonable to write an iterator that produces an infinite stream of data.

Iterable

In Python,Iterable is anything you can loop over with a for loop.

An object is called an iterable if u can get an iterator out of it.

1. Calling iter() function on an iterable gives us an iterator.
2. Calling next() function on iterator gives us the next element.
3. If the iterator is exhausted(if it has no more elements), calling next() raises StopIteration exception.

Topics Covered in this story

---

How to convert an object into Iterator

Using iter() function.

The built-in iter() function takes an arbitrary object and tries to return an iterator that will return the object’s contents or elements, raising TypeError if the object doesn’t support iteration.

Iterables vs Iterators.

1. Both iterables and iterators can be iterated using for loop.

https://gist.github.com/IndhumathyChelliah/326b2acd8475078896daa62f85f4ef4c

2. Iterables supports only iter() function.But iterators supports both iter() and next() function.

Iterable:

Iterator:

3. Iterators are also iterables.

We can get an iterator from an iterable by calling iter() function.Similarly, we can call iter() function on the iterator itself. It will return the iterator object itself.

https://gist.github.com/IndhumathyChelliah/62a7c932a12d2b5a63004018609dcfa7

---

Data types that support Iterators

• Text Sequence type- str

Strings support Iterator. iter() on a string returns an iterator.

https://gist.github.com/IndhumathyChelliah/7695f0955b8b929efd5272624f491efe

• Sequences in Python: List, tuple, range supports Iterator.

1.Converting list(iterable) into iterator object.

https://gist.github.com/IndhumathyChelliah/f2edb9fa82aba0c0786d0b45d27cea3b

2. Converting tuple(iterable) into iterator object.

https://gist.github.com/IndhumathyChelliah/d7d9ebadf78dd8d8c85331beb2e76e6c

3.Converting range() into iterator object.

https://gist.github.com/IndhumathyChelliah/d4bb594203af7908f6dfa0af55abea2f

• Mapping: Dictionary supports Iterator.

iter()function will loop over dictionary keys only.

https://gist.github.com/IndhumathyChelliah/2bad88dcc9272edbe7d7b894eb1680db

Dictionary has methods like 
1.values()– which will iterate over values
2.items() -which will iterate over (key, value) pairs.

https://gist.github.com/IndhumathyChelliah/36fa0a555e5adfb8520962f44a2f3263

items()-loop over (key,value) pair in the dictionary.

https://gist.github.com/IndhumathyChelliah/38953c1966d3982df840af043cf0f32e

• set

https://gist.github.com/IndhumathyChelliah/f826769f94379f4af6127d58faf8d826

File objects are implemented as iterators.

Files also support iteration by calling the readline() method until there are no more lines in the file.
readline()-Read until newline or EOF and return a single str. If the stream is already at EOF, an empty string is returned.

https://gist.github.com/IndhumathyChelliah/cde645e2a87e08decf809f897a9eb1c4

colors.txt

red
green
blue

---

How to convert an iterator to list/tuple/dict

Iterators can be materialized as lists or tuples by using the list() ortuple() constructor functions.The dict() constructor can accept an iterator that returns a finite stream of (key, value) tuples.

https://gist.github.com/IndhumathyChelliah/37867ae8fdb12d66c7865283dbf44814

---

Functions which returns an iterator

1. Generator
   Generators are a special class of functions that return an iterator that returns a stream of values.
   Any function containing a yield keyword is a generator function.

https://gist.github.com/IndhumathyChelliah/900289f58ae3a67f209638b5f1faa710

2. Generator expression
Generator expression returns an iterator object only. Refer to my story for generator expressions.

https://gist.github.com/IndhumathyChelliah/94f52a4d63fabfc3d3969fae046feb92

Built-in functions that return an iterator.

1. map()
   Return an iterator that applies a function to every item of iterable, yielding the results.

Refer to my story of map() vs starmap()

https://gist.github.com/IndhumathyChelliah/fd88a87b68615f11f42f268f39ded671

2. filter()
Returns an iterator from those elements of iterable for which function returns true. iterable may be either a sequence, a container that supports iteration, or an iterator.

Refer to my story of filter vs filterfalse

https://gist.github.com/IndhumathyChelliah/c39ce37add52680d94945bcda195761a

3. zip()
zip(iter a, iter b ..) —Returns a zip object which is an iterator.
It takes one element from each iterable and returns them in a tuple.

https://gist.github.com/IndhumathyChelliah/94a51b6141d7a1edab7435540eaf8844

4. enumerate():
enumerate(iterable, start=0)
Return an enumerate object which is an iterator. 
iterable must be a sequence, an iterator, or some other object which supports iteration. 
The __next__() method of the iterator returned by enumerate()returns a tuple containing a count (from the start which defaults to 0) and the values obtained from iterating over iterable.

https://gist.github.com/IndhumathyChelliah/6a2ae0413962e3b181f5199c53aabd25

5.itertools module:

The itertools module contains a number of commonly-used iterators as well as functions for combining several iterators.

Refer to my story for itertools module.

6.reversed()

reversed() function is used to reverse the elements in the sequences (list,tuple,range()).It is also used to reverse the key elements in the dictionary.
reversed() function returns an iterator object.

https://gist.github.com/IndhumathyChelliah/4b5ff3f251d2ce0a5a56254ed3fcb028

The built-in functions which supports Iterator.

1.max()
max() also supports a single iterator argument and returns the largest element.

https://gist.github.com/IndhumathyChelliah/bdbf4f48051dedea6de4edb53ab4cbfd

2. min()
min() also supports a single iterator argument and returns the smallest element.
If iterator is infinite,max() and min() will never return.It supports only finite iterators.

https://gist.github.com/IndhumathyChelliah/21a3f4809c28e4be3bce7c754ad202e1

Operators:

in, not in

The "in" and "not in" operators also support iterators. X in iterator is true if X is found in the stream returned by the iterator. 
If the iterator is infinite, and if the element X never appears in the stream, the "in" and "not in" operators will never return.

https://gist.github.com/IndhumathyChelliah/f19de61cbcfc5b626f1e8647d207653d

Sequence unpacking

Sequence unpacking also supports iterators.

Sequence unpacking requires the list of variables on the left to have the same number of elements in the iterator.

https://gist.github.com/IndhumathyChelliah/77b61ba15d4b495ec0e8f3cc420c3ad2

for loop

In the statement for X in Y, Y must be an iterator or some object(iterable) for which iter() can create an iterator.
 These two statements are equivalent:

for i in iter(obj):
    print(i)

for i in obj:
    print(i)

Limits of Iterator

• We can only go forward in an iterator.
• We can’t make a copy of it.
• No way to get the previous element.
• We can’t reset the iterator.

https://gist.github.com/IndhumathyChelliah/e4a4127b19b486c10b080dff7f54167c

• The iterator protocol only specifies the __next__() method. Functions may therefore consume all of the iterator’s output, and if you need to do something different with the same stream, you’ll have to create a new iterator.

https://gist.github.com/IndhumathyChelliah/0a04c083876c788ad8328c09587aad11

StopIteration

Raised by the built-in function next()and an iterator’s __next__() method to signal that there are no further items produced by the iterator.

https://gist.github.com/IndhumathyChelliah/96e4de16ddf3853025feba04721fa208

---

Conclusion

• Iterators can be infinite or finite.
• max(),min() doesn’t support infinite iterators.
• len() is not supported by iterators.
• for loop can be used to iterate over iterable and also iterator.
• Iterators:
  1. We can iterate using for loop
  2. We can use the next() method which will return the next element in the iterator.
  3. We can convert the iterator to list using list() constructor, tuple using tuple () constructor.
  4.The dict() constructor can accept an iterator that returns a finite stream of (key, value) tuples
• Difference between iterables and iterators:
  1. Iterators support iter() and next() function. Iterables support only iter() function.
  2. Iterators are also iterables but not vice versa.

---

Resources(Python Documentation):

Iterators

Iterator

Iterator Types

StopIteration