Python Itertools

Python Itertools-Functions that create Iterators for efficient looping.

Python Itertools:

This module standardizes a core set of fast, memory efficient tools that are useful by themselves or in combination. Together, they form an “iterator algebra” making it possible to construct specialized tools succinctly and efficiently in pure Python.
Python itertools module is available in Python Standard Library.

Types of Iterators provided by this module:

• Infinite Iterators
• Iterators terminating on shortest input sequence
• combinatoric iterators

Return type will be Iterator object.We can iterate through the iterator by

1. using next() function
2. using for loop
3. converting to list object using list()

Infinite Iterators:
count(),cycle(),repeat()

1. count():
   Makes an iterator that returns evenly spaced values starting with number specified in argument start.By default start is 0 and step is 1.Step can be non integer values also.This functions returns an infinite iterator.

 itertools.count(start=0,step=1)

https://gist.github.com/IndhumathyChelliah/2343b3465436ccd996006ad893f5d69a

It is also used as an argument in map() and zip() function.

zip() -zip() function is a Python built-in function which allows to combine corresponding elements from multiple sequences and return zip object which is iterator of tuples. itertools.count() can be used as sequence in zip() function.

https://gist.github.com/IndhumathyChelliah/52bb217dd9f70a25b0d9419e1fcba618

map()-Python map() function is used to apply function on all elements in the specified iterable and returns a map object which is an iterator. We can use itertools.count() as an argument in map() function .

https://gist.github.com/IndhumathyChelliah/bd5e0eef629ca42cb1c8dc8de5cb1ac7

2. cycle():
Makes an iterator returning elements from the iterable and saving a copy of each.When the iterable is exhausted,returns element from the saved copy.Repeats indefinitely.

itertools.cycle(iterable)

https://gist.github.com/IndhumathyChelliah/c66abe74a0bae892fe7eea1d72e5620c

3.repeat():
Makes an iterator that returns object over again and again.Runs indefinitely unless times argument is mentioned.
Used as an argument in map() and zip() function.

itertools.repeat(object,times)

https://gist.github.com/IndhumathyChelliah/5524f0dcbbb97637bd5c91849d8d2951

It is used as an argument in map() function.

https://gist.github.com/IndhumathyChelliah/7039a67b3e41d1af3dfcf77182780ca4

It is used as an argument in zip() function.

https://gist.github.com/IndhumathyChelliah/ad7fdaf372739ac8ef522ff144996c84

Iterators terminating on the shortest input sequence:

1. accumulate():
   Makes an iterator that returns accumulated sum or accumulated results of other binary functions which is mentioned in func-parameter.
   Usually number of elements in the output iterable matches the number of elements in the input iterable.If initial-parameter is mentioned, it gets accumulated, so the output iterable contains one element more than the input iterable.
   functools.reduce() returns only final accumulated value for similar function.
   initial parameter is introduced in Python version 3.8

itertools.accumulate(iterable,func,*,initial=None)

https://gist.github.com/IndhumathyChelliah/eceb9239f5cdae954d95e2ca20dea306

2. chain():
Makes an iterator that returns element from the first iterable until its exhausted,then proceeds to the next iterable. It will treat consecutive sequences as a single sequence.
itertools.chain(*iterables)

https://gist.github.com/IndhumathyChelliah/d2abfcc98a2d198968fdd7f8ac89c724

3. chain.from_iterable():
This function takes one iterable as an input argument and returns flattened iterable containing all elements in the input iterable. All elements in the input iterable should be iterable,otherwise it will raise TypeError.

chain.from_iterable(iterable)

https://gist.github.com/IndhumathyChelliah/0efce478a8e10b33af668599051061ff

4. compress():
Make an iterator that filters elements from the data returning only those that have a corresponding element in selectors that evaluates to True.Stops when either data or selectors iterables has been exhausted.

itertools.compress(data,selectors)

https://gist.github.com/IndhumathyChelliah/2d494af2b65b464957f45aa70ea69bcd

5.dropwhile():
Make an iterator that drops elements from the iterable as long as the predicate is True,afterwards returns every element.The iterator does not produce any output until the predicate first becomes False.

itertools.dropwhile(predicate,iterable)

6.takewhile():
Make an iterator that returns element from the iterable as long as the predicate is True.

itertools.takewhile(predicate,iterable)

https://gist.github.com/IndhumathyChelliah/686e365562a89ccd22f341612a8f3ccf

7. filterfalse():
Make an iterator that filters elements from iterable returning only those for which the predicate is False.If predicate is None,returns items that are False.
filter() method returns an iterator which contains elements of an iterable for which function returns True.

itertools.filterfalse(predicate,iterables)

https://gist.github.com/IndhumathyChelliah/34c438ed316a906baa1f5a48cb0c5957

8.zip_longest():
Makes an iterator that aggregates elements from each of the iterables. If iterables are of uneven length,missing value is filled with fillvalue. Iteration continues until longest iterable is exhausted.
In zip(),iteration continues until shortest iterable is exhausted.

itertools.zip_longest(*iterables,fillvalue=None)

https://gist.github.com/IndhumathyChelliah/bd8d4b11ceb1fb27056eaa268826cb8b

9. starmap():
Make an iterator that computes the function using arguments obtained from the iterable. Used instead of map() when argument parameters are already grouped in tuples from a single iterable (the data has been “pre-zipped”).

itertools.starmap(function,iterable)

https://gist.github.com/IndhumathyChelliah/d4a5b1873b5a5b41a6810e88c71e1dfe

10. islice():
Makes an iterator that returns selected elements from the iterable. If start is None, then iteration starts at zero. If step is None, then the step defaults to one.If stop is None,then iteration continues until the iterator is exhausted, if at all; otherwise, it stops at the specified position. islice() doesn’t support negative values for start,stop and step.

itertools.islice(iterable,stop)
itertools.islice(iterable, start, stop[, step])

https://gist.github.com/IndhumathyChelliah/b2d9559073f64c06d2337ee8e5fe07e2

11.tee():
Return n independent iterators from a single iterable.

itertools.tee(iterable,n=2)

https://gist.github.com/IndhumathyChelliah/a3917388103d8f52bc09aa324ef2f173

12. groupby():
Make an iterator that returns consecutive keys and groups from the iterable.
key is a function computing key value for each element.If key is not specified or None,key defaults to an identity function and returns the element unchanged.

itertools.groupby(iterable,key=None)

https://gist.github.com/IndhumathyChelliah/c82b252462bdd3303aeb87b97add6f5b

Combinatoric Iterators:

1. product():
   Cartesian product of input iterables.
   Cartesian product definition: The product of set X and set Y is the set that contains all ordered pairs ( x, y ) for which x belongs to X and y belongs to Y.
   To compute the product of an iterable with itself, specify the number of repetitions with the optional repeat keyword argument. For example, product(A, repeat=4) means the same as product(A, A, A, A).

itertools.product(*iterables,repeat)

https://gist.github.com/IndhumathyChelliah/6e6f6ef998621d6561255fc06c4935ac

2. permutations():
Return successive r length permutations of elements in the iterable. If r is not specified or is None,then r defaults to the length of the iterable and all possible full-length permutations are generated.The permutation tuples are emitted in lexicographic ordering according to the order of the input iterable. So, if the input iterable is sorted, the combination tuples will be produced in sorted order.

Elements are treated as unique based on their position, not on their value. So if the input elements are unique, there will be no repeat values in each permutation.

itertools.permutations(iterable,r=None)

https://gist.github.com/IndhumathyChelliah/5d4b6c5c3ac1f5dbea5d971b17e614cf

Note:In Permutations,order of the elements matters.

3. combinations():
Return r length subsequences of elements from the input iterable.

The combination tuples are emitted in lexicographic ordering according to the order of the input iterable. So, if the input iterable is sorted, the combination tuples will be produced in sorted order.

Lexicographic ordering means way of ordering of words in alphabetical order of their component letters.

Elements are treated as unique based on their position, not on their value. So if the input elements are unique, there will be no repeat values in each combination.

itertools.combinations(iterable, r)

4. combinations_with_replacement():
Return r length subsequences of elements from the input iterable allowing individual elements to be repeated more than once.

itertools.combinations_with_replacement(iterable, r)

https://gist.github.com/IndhumathyChelliah/be9ae610810f32486dc458fe64714315

Note:
1. Used as an argument in map() and zip():
count(),repeat()
repeat()- supply stream of constant values to map() or zip() function.
count()-it will supply different values to map() or zip() function.

2.Difference between cycle() and repeat():
cycle()-iterates over the same object again and again
repeat()-returns the same object again and again.

3.Difference between reduce() and itertools.accumulate():
reduce():
* It will return only the final accumulated value.
* First argument should be function and second argument should be iterable.
accumulate()
* It will return the running accumulated value.The elements in the output iterable will be equal to elements in the input iterable,if initial value is not mentioned.
* First argument should be iterable and second argument should be function.

4.Difference between filter() and itertools.compress():
* filter()function-filters the given iterable with the help of function that test each element in the iterable is True or not.
* compress():filters the given iterable based on the corresponding element in the selector. Iterable containing True /False is given as selector.

5.Difference between filter() and itertools.filterfalse():
*filter():Construct an iterator from the elements of iterable for which function returns True.
*filterfalse():Construct an iterator from the elements of iterable for which function returns False.

6.Difference between zip() and itertools.zip_longest():
*zip():Iteration continues until shortest iterable is exhausted.
*zip_longest():Iteration continues until longest iterable is exhausted.

7.Difference between list slicing and itertools.islice():
*List slicing creates new list
*islice()-Returns an iterator.We can loop through the iterator,in the way we want.

8.Difference between itertools.permutations() and itertools.combinations():
* itertools.permutations():Order of the elements does matters.
*itertools.combinations():Order of the elements doesn’t matters.
Both combinations and permutations doesn’t repeat values.

9.Difference between itertools.combinations() and itertools.combinations_with_replacement
* combinations():Order of element doesn’t matters and doesn’t repeat values.
* combinations_with_replacement():Order of element doesn’t matters and it repeats values.

10.Difference between itertools.takewhile() and itertools.dropwhile():
* takewhile():Make an iterator that returns element from the iterable as long as the predicate is True.
* dropwhile():Make an iterator that drops element from the iterable as long as the predicate is True afterwards returns every element.

Resources:

Itertools:https://docs.python.org/3/library/itertools.html